You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
shape_snapshot.execute_for_shape and shape_snapshot.query_fn spans were missing from production traces for move_in_query operations, while initial_snapshot and subset_query worked fine. This wires up a parent span so the existing child spans get emitted.
Problem
Sampling for shape_snapshot.query_fn is a plain included?(_) -> true (no rate limiting), so the absence of move_in_query spans wasn't a sampling issue — the spans were never being created.
View the top 3 failed test(s) by shortest run time
Elixir.Electric.ShapeCacheTest::test get_or_create_shape_handle/2 against real db crashes when initial snapshot query fails to return data quickly enough
Stack Traces | 0s run time
22) test get_or_create_shape_handle/2 against real db crashes when initial snapshot query fails to return data quickly enough (Electric.ShapeCacheTest)
test/electric/shape_cache_test.exs:507
** (EXIT from #PID<0.21571.0>) killed
Elixir.Electric.ShapeCacheTest::test await_snapshot_start/4 does not recursively loop forever if the snapshot fails to start
Stack Traces | 0.0982s run time
26) test await_snapshot_start/4 does not recursively loop forever if the snapshot fails to start (Electric.ShapeCacheTest)
test/electric/shape_cache_test.exs:1023
Expected truthy, got false
code: assert String.contains?(
log,
"shape_handle=#{shape_handle} [error] No consumer process when waiting on initial snapshot creation"
)
arguments:
# 1
""
# 2
"shape_handle=50617496-1779100178069616 [error] No consumer process when waiting on initial snapshot creation"
stacktrace:
test/electric/shape_cache_test.exs:1071: (test)
test/runtime-dsl.test.ts > N: wake primitives verification > N5: runFinished wake records the finished child on the parent stream
test/runtime-dsl.test.ts > M: deep researcher coordination > M3: separate researcher entities keep child results isolated across later wakes
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:4968:3
test/runtime-dsl.test.ts > K: wiki coordination > K4: create_wiki rejects switching the topic on an existing wiki
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5546:3
test/runtime-dsl.test.ts > I: peer review coordination > I1: peer review aggregates three reviewer writes through shared state
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5049:3
test/runtime-dsl.test.ts > K: wiki coordination > K8: wiki keeps durable child metadata and shared articles carry topic and author details
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5688:3
test/runtime-dsl.test.ts > K: wiki coordination > K9: idempotent wiki recreation does not duplicate shared article rows
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5760:3
test/runtime-dsl.test.ts > J: debate coordination > J3: debate with only one durable side stays partial until the missing side arrives
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5328:3
test/runtime-dsl.test.ts > I: peer review coordination > I3: peer review with one configured reviewer summarizes only that durable row
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5143:3
test/runtime-dsl.test.ts > J: debate coordination > J1: debate parent reads both sides from shared state before issuing a ruling
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5221:3
test/runtime-dsl.test.ts > K: wiki coordination > K3: get_wiki_status reports complete coverage after specialist articles land
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5521:3
test/runtime-dsl.test.ts > I: peer review coordination > I4: peer review with two configured reviewers summarizes only those durable rows
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5181:3
test/runtime-dsl.test.ts > K: wiki coordination > K10: same-topic wiki expansion adds only the missing article and updates later query coverage
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5791:3
test/runtime-dsl.test.ts > K: wiki coordination > K6: repeating create_wiki with the same topic and subtopics is idempotent
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5630:3
test/runtime-dsl.test.ts > K: wiki coordination > K2: repeating create_wiki reuses existing specialists and only spawns missing subtopics
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:5434:3
test/runtime-dsl.test.ts > M: deep researcher coordination > M1: researcher workers start from spawn initialMessage without an extra send
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:4915:3
test/runtime-dsl.test.ts > D: shared state > D9: a setup-registered shared-state effect fires on the first wake write and survives a later wake
Stack Traces | 30s run time
Error: Test timed out in 30000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
❯ test/runtime-dsl.test.ts:3396:3
test/runtime-dsl.test.ts > K: wiki coordination > K1: wiki specialists accumulate shared articles that a later query can read
Stack Traces | 30.1s run time
Error: Timeout (30000ms) waiting for shared:wiki_article x2 on shared state wiki-wiki-1
[]
❯ waitForHistory test/runtime-dsl.ts:664:11
❯ test/runtime-dsl.test.ts:5391:5
I thought 686ebdb had fixed that. Just now reviewing the changes, I see we have two functions named query_move_in_async: one in PartialModes and the other one in Effects. My commit fixed the former one (where the root span comes from the shape serving plug).
The one defined in Effects is invoked in response to a materializer_changes callback in the Consumer module. We could start a span right in the handle_info callback instead, seems like a less ad-hoc approach then creating a span right inside the inner function.
Looks like the two implementations of query_move_in_async overlap. Can we deduplicate them into one?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
shape_snapshot.execute_for_shapeandshape_snapshot.query_fnspans were missing from production traces formove_in_queryoperations, whileinitial_snapshotandsubset_queryworked fine. This wires up a parent span so the existing child spans get emitted.Problem
Sampling for
shape_snapshot.query_fnis a plainincluded?(_) -> true(no rate limiting), so the absence ofmove_in_queryspans wasn't a sampling issue — the spans were never being created.