test(gc): collect at stack depth, and gate walker liveness on every arm - #7359
Merged
Conversation
added 2 commits
August 4, 2026 10:56
The native-root walker had no probe that made it work. Every probe in the suite calls `gc()` at the end, from a shallow stack, so on macOS and Linux `04_dead_after_deep_stack` reported 7 frames visited and **zero** root locations. Both arms would have passed unchanged with a walker that visited nothing at all -- other root sources covered the probes. Windows only walked a deep stack (5,626 frames) by accident of heap sizing, which is why the `--require-locations` gate could be applied there and nowhere else. `11_collect_at_depth` makes that coverage deliberate. `descend` holds a heap value live ACROSS its recursive call and collects at the deepest point, so at collection time there is one live root per frame, all of them mid-frame rather than in the leaf. Every slot is read after the collection returns, so a walker that stops early -- or a map with a wrong base register -- produces a wrong checksum, not merely a slower run. Under `PERRY_GC_FORCE_EVACUATE=1` every survivor moves, so a stale pointer cannot be accidentally right. Measured, byte-matching the pinned Node oracle: macOS aarch64 228 frames, 222 records, 221 locations (was 7 / 0) x86-64 Linux 231 frames, 221 locations (was 7 / 0) With both Unix arms now walking a real stack, `--require-locations` moves from the Windows-only branch to the shared path and gates all three. Full ratchet suite: 11/11 byte-identical to the oracle under `PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1`. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe PR adds a deep-recursion GC probe with live roots in each stack frame. The workflow runs the probe with tracing and forced evacuation on macOS, Linux, and Windows, and requires walker telemetry on every platform. ChangesGC root validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Workflow
participant Probe
participant GC
Workflow->>Probe: run 11_collect_at_depth with tracing
Probe->>Probe: recurse with one live Payload per frame
Probe->>GC: collect at maximum depth
GC-->>Probe: relocate live roots
Probe-->>Workflow: emit checksum and memory metrics
Workflow->>Workflow: require nonzero walker telemetry
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
proggeramlug
added a commit
that referenced
this pull request
Aug 4, 2026
#7360) * test(gc): collect at stack depth, and gate walker liveness on every arm The native-root walker had no probe that made it work. Every probe in the suite calls `gc()` at the end, from a shallow stack, so on macOS and Linux `04_dead_after_deep_stack` reported 7 frames visited and **zero** root locations. Both arms would have passed unchanged with a walker that visited nothing at all -- other root sources covered the probes. Windows only walked a deep stack (5,626 frames) by accident of heap sizing, which is why the `--require-locations` gate could be applied there and nowhere else. `11_collect_at_depth` makes that coverage deliberate. `descend` holds a heap value live ACROSS its recursive call and collects at the deepest point, so at collection time there is one live root per frame, all of them mid-frame rather than in the leaf. Every slot is read after the collection returns, so a walker that stops early -- or a map with a wrong base register -- produces a wrong checksum, not merely a slower run. Under `PERRY_GC_FORCE_EVACUATE=1` every survivor moves, so a stale pointer cannot be accidentally right. Measured, byte-matching the pinned Node oracle: macOS aarch64 228 frames, 222 records, 221 locations (was 7 / 0) x86-64 Linux 231 frames, 221 locations (was 7 / 0) With both Unix arms now walking a real stack, `--require-locations` moves from the Windows-only branch to the shared path and gates all three. Full ratchet suite: 11/11 byte-identical to the oracle under `PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1`. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF * docs: changelog fragment for #7359 Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF * ci(gc): add the aarch64-Linux arm, the one uncovered corner of the map The matrix covered aarch64+Mach-O, x86-64+ELF and x86-64+PE, and the note explaining the gap said ARM64 Linux was skipped because "its two components are each covered above." That is the exact compositional fallacy `word_width_for` in `gc_map.rs` exists to warn about. `.word` is not a fixed size -- GNU `as` defines it as the target's natural machine word -- so LLVM's AArch64 ELF backend spells every 32-bit stack-map field `.word`, while both covered arms spell it `.long`: Mach-O uses `.long` on aarch64, and on x86 `.word` means *two* bytes so LLVM will not use it for a 32-bit field. The directive width is a property of the intersection, not of either component. What this arm does and does not add, stated precisely, because overclaiming in this file is how #7321's wrong explanation survived into an issue and a job name: the `.word` spelling is already unit-tested on every arm, against a hand-written sample. What no arm has ever exercised is the end-to-end chain on this target -- real LLVM asm output, real ELF linking, real runtime walking -- where the failure mode is not a parse error but a wrong answer. `ubuntu-24.04-arm` is already in use in release-packages.yml and the repo is public, so the runners are available; the "queue for hours" half of the old rationale is stale too. Also repairs a garbled sentence in the header comment, left by an edit that spliced two clauses about the pre-#7349 x86-64 refusal. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF * docs(ci): the knob ledger said PERRY_STATEPOINT_REPORT was deleted; it is internal plumbing The env *spelling* was deleted under the kill policy (#7314) and the flag is the only entry point -- but the variable itself is still how the driver hands the format to the rayon module workers, and run_pipeline.rs remove_var's it when the flag is absent so an inherited value cannot switch reporting on. That block is a knob ledger. An entry reading "deleted" for a name still greppable in the tree makes the whole list look stale to the next auditor. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com>
proggeramlug
added a commit
that referenced
this pull request
Aug 4, 2026
…tform matrix (#7367) The plan still said statepoints were aarch64-only (#7321), that the matrix "therefore runs on macos-14", that `statepoints-refuse-x86` pinned the refusal, and it spelled the knob `PERRY_STATEPOINTS` four times. None of that is true now, and this document is what the adoption decision gets made from. What actually changed: - x86-64 is unblocked. `_Unwind_GetGR(ctx, 7)` does segfault and cannot be fixed as stated -- libgcc tracks only the columns CFI restores and RSP is derived, not tracked. #7349 stopped asking for it and derives the SP-relative base from `_Unwind_GetCFA`, with a per-arch return-address adjustment (x86-64 `call` pushes one, aarch64 `bl` does not). x86-64 Linux is a first-class arm. - Windows works via RtlVirtualUnwind (#7355), the one walker with no Itanium unwinder beneath it. - aarch64+ELF is now covered too (#7360) -- the only shape where LLVM spells 32-bit stack-map fields `.word`. - One mechanism, not two: PERRY_STATEPOINTS and the plain-map bridge are deleted, so the kill-policy line about "a mode that still exists" no longer applies to this pair. - The gate proves something now. Until today the Unix arms reported 7 frames and ZERO locations -- they would have passed with a walker that visited nothing. #7359's deep-collect probe took them to 221 locations. - watchOS/visionOS are not blocked by Perry: they build on stable without `dyn-eval`, and fail three crates away in psm's Mach-O guard. So the remaining adoption gate is `llvm-inprocess` becoming a default cargo feature, plus sequencing step 2 (root density) -- adopting today would regress binary size on root-dense code. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF Co-authored-by: Ralph Küpper <ralph@skelpo.com>
proggeramlug
added a commit
that referenced
this pull request
Aug 4, 2026
…#7369) * fix(lint): split index_set.rs, over the 2000-line cap since #7342 (#7366) `scripts/check_file_size.sh` exits 1 on main HEAD: `crates/perry-codegen/src/expr/index_set.rs` is 2035 lines against a 2000 cap. It crossed in #7342. That script runs inside the `lint` job, which is a REQUIRED context -- so this is the second independent way `lint` was red on main today (the first was rustfmt on linker.rs, #7361). A required check that is red on main blocks nothing; it means every merge is a bypass. The split follows the recipe in the script's own failure message: extract a topical group into a sibling module. `lower_inline_dyn_typed_array_set` and its `emit_inline_ta_int_store` helper are the guarded inline typed-array store for a type-erased receiver -- one coherent unit, moved verbatim to `index_set_typed_array.rs`. index_set.rs drops to 1749 lines, leaving real headroom rather than landing one line under the cap. Mechanical move: the two functions are byte-identical, only the imports they need travelled with them and `lower_inline_dyn_typed_array_set` became `pub(super)` so its one caller can still reach it. cargo test -p perry-codegen --lib: 609 passed. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF Co-authored-by: Ralph Küpper <ralph@skelpo.com> * docs(plan): both statepoint adoption gates are closed; record the platform matrix (#7367) The plan still said statepoints were aarch64-only (#7321), that the matrix "therefore runs on macos-14", that `statepoints-refuse-x86` pinned the refusal, and it spelled the knob `PERRY_STATEPOINTS` four times. None of that is true now, and this document is what the adoption decision gets made from. What actually changed: - x86-64 is unblocked. `_Unwind_GetGR(ctx, 7)` does segfault and cannot be fixed as stated -- libgcc tracks only the columns CFI restores and RSP is derived, not tracked. #7349 stopped asking for it and derives the SP-relative base from `_Unwind_GetCFA`, with a per-arch return-address adjustment (x86-64 `call` pushes one, aarch64 `bl` does not). x86-64 Linux is a first-class arm. - Windows works via RtlVirtualUnwind (#7355), the one walker with no Itanium unwinder beneath it. - aarch64+ELF is now covered too (#7360) -- the only shape where LLVM spells 32-bit stack-map fields `.word`. - One mechanism, not two: PERRY_STATEPOINTS and the plain-map bridge are deleted, so the kill-policy line about "a mode that still exists" no longer applies to this pair. - The gate proves something now. Until today the Unix arms reported 7 frames and ZERO locations -- they would have passed with a walker that visited nothing. #7359's deep-collect probe took them to 221 locations. - watchOS/visionOS are not blocked by Perry: they build on stable without `dyn-eval`, and fail three crates away in psm's Mach-O guard. So the remaining adoption gate is `llvm-inprocess` becoming a default cargo feature, plus sequencing step 2 (root density) -- adopting today would regress binary size on root-dense code. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF Co-authored-by: Ralph Küpper <ralph@skelpo.com> * gc: admit two provably-leaf helpers, and measure that it buys nothing js_gc_register_global_root was the most frequent non-leaf callee in the probe suite (148 call sites) and is provably GC-leaf: its whole body is runtime_write_barrier_root_heap_word -- which js_write_barrier_root_heap_word, already CannotCollect, wraps in one line -- plus a TLS Vec::push. The "malloc count threshold" trigger does not apply to that push: the counter is MALLOC_STATE.objects.len(), a registry of Perry GC objects, and the #[global_allocator] is plain mimalloc/System with no GC hook. js_typed_feedback_maybe_dump_trace joins its already-admitted family siblings. Measured A/B on the same tree, and the result is a null: probe safepoints roots total bytes __text 06_string_retention 105 -> 100 27=27 0 -4 B 09_try_catch_roots 343 -> 339 259=259 0 -4 B 11_collect_at_depth 120 -> 117 36=36 0 -4 B Root counts are IDENTICAL. The 40 safepoints removed across the suite were all rootless, and a rootless safepoint costs essentially nothing -- which is what docs/engine-plan.md already says: "the axis is not 'statepoints are bigger', it is 'roots are bigger'". Recording it as evidence: the safepoint-count lever is not the binary-size lever, so sequencing step 2 must attack live-root SETS. Two tests come with it. One pins the wrapper's classification to the barrier it wraps. The other pins js_nanbox_string OUT of the allowlist: at 120 call sites it is the obvious next candidate and reads as pure bit manipulation, but its null guard calls js_string_from_bytes to allocate an empty string. Probe suite 11/11 byte-identical under forced evacuation + verification. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com>
proggeramlug
added a commit
that referenced
this pull request
Aug 4, 2026
…7368) * fix(codegen): statepoint report counted zero safepoints since #7348 #7348 deleted the explicit bridge and with it the only callers of note_statepoint and note_skipped -- they lived in the bridge, which counted safepoints as it emitted them. The methods survived with no callers, so statepoints, relocations, max_live_roots, skipped_non_safepoints, live_roots_histogram and both by-callee maps went structurally zero in production. A real compile printed "0 statepoints emitted" while its binary carried 120. Counting at IR-emission time cannot work any more, and that is the lesson: Perry no longer decides which calls become safepoints -- RewriteStatepointsForGC does, inside LLVM. The only honest source is the compact-map rewrite, which already parses the assembly LLVM emitted and computed these exact numbers before dropping them into log::debug!. The report reads from there now: 120 safepoints across 6 function(s) in 1 module(s) 36 live roots recorded, 0.30 per safepoint An absent measurement no longer renders as a measured zero: gc_map.modules == 0 means "never reported", the text report says UNAVAILABLE rather than printing zeros, and JSON carries gc_map separately from totals so a consumer can tell them apart. schema_version -> 2. The CI gate now asserts the counts, not just the label. --only-backend rs4gc passed throughout the regression -- the label was right, the numbers were fiction. It now also requires records > 0 and roots > 0; verified against a synthetic report with the #7348 shape, where the label check still reports 9 functions green while the count checks exit 1. Second round of dead counters here (#7362 removed four that never had a writer at all). The new test documents why the first invariant missed this one: every_rendered_counter_has_a_writer called the mutators itself, so "has a writer" passed while "is written" was false. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF * gc: admit two provably-leaf helpers, and measure that it buys nothing (#7369) * fix(lint): split index_set.rs, over the 2000-line cap since #7342 (#7366) `scripts/check_file_size.sh` exits 1 on main HEAD: `crates/perry-codegen/src/expr/index_set.rs` is 2035 lines against a 2000 cap. It crossed in #7342. That script runs inside the `lint` job, which is a REQUIRED context -- so this is the second independent way `lint` was red on main today (the first was rustfmt on linker.rs, #7361). A required check that is red on main blocks nothing; it means every merge is a bypass. The split follows the recipe in the script's own failure message: extract a topical group into a sibling module. `lower_inline_dyn_typed_array_set` and its `emit_inline_ta_int_store` helper are the guarded inline typed-array store for a type-erased receiver -- one coherent unit, moved verbatim to `index_set_typed_array.rs`. index_set.rs drops to 1749 lines, leaving real headroom rather than landing one line under the cap. Mechanical move: the two functions are byte-identical, only the imports they need travelled with them and `lower_inline_dyn_typed_array_set` became `pub(super)` so its one caller can still reach it. cargo test -p perry-codegen --lib: 609 passed. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF Co-authored-by: Ralph Küpper <ralph@skelpo.com> * docs(plan): both statepoint adoption gates are closed; record the platform matrix (#7367) The plan still said statepoints were aarch64-only (#7321), that the matrix "therefore runs on macos-14", that `statepoints-refuse-x86` pinned the refusal, and it spelled the knob `PERRY_STATEPOINTS` four times. None of that is true now, and this document is what the adoption decision gets made from. What actually changed: - x86-64 is unblocked. `_Unwind_GetGR(ctx, 7)` does segfault and cannot be fixed as stated -- libgcc tracks only the columns CFI restores and RSP is derived, not tracked. #7349 stopped asking for it and derives the SP-relative base from `_Unwind_GetCFA`, with a per-arch return-address adjustment (x86-64 `call` pushes one, aarch64 `bl` does not). x86-64 Linux is a first-class arm. - Windows works via RtlVirtualUnwind (#7355), the one walker with no Itanium unwinder beneath it. - aarch64+ELF is now covered too (#7360) -- the only shape where LLVM spells 32-bit stack-map fields `.word`. - One mechanism, not two: PERRY_STATEPOINTS and the plain-map bridge are deleted, so the kill-policy line about "a mode that still exists" no longer applies to this pair. - The gate proves something now. Until today the Unix arms reported 7 frames and ZERO locations -- they would have passed with a walker that visited nothing. #7359's deep-collect probe took them to 221 locations. - watchOS/visionOS are not blocked by Perry: they build on stable without `dyn-eval`, and fail three crates away in psm's Mach-O guard. So the remaining adoption gate is `llvm-inprocess` becoming a default cargo feature, plus sequencing step 2 (root density) -- adopting today would regress binary size on root-dense code. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF Co-authored-by: Ralph Küpper <ralph@skelpo.com> * gc: admit two provably-leaf helpers, and measure that it buys nothing js_gc_register_global_root was the most frequent non-leaf callee in the probe suite (148 call sites) and is provably GC-leaf: its whole body is runtime_write_barrier_root_heap_word -- which js_write_barrier_root_heap_word, already CannotCollect, wraps in one line -- plus a TLS Vec::push. The "malloc count threshold" trigger does not apply to that push: the counter is MALLOC_STATE.objects.len(), a registry of Perry GC objects, and the #[global_allocator] is plain mimalloc/System with no GC hook. js_typed_feedback_maybe_dump_trace joins its already-admitted family siblings. Measured A/B on the same tree, and the result is a null: probe safepoints roots total bytes __text 06_string_retention 105 -> 100 27=27 0 -4 B 09_try_catch_roots 343 -> 339 259=259 0 -4 B 11_collect_at_depth 120 -> 117 36=36 0 -4 B Root counts are IDENTICAL. The 40 safepoints removed across the suite were all rootless, and a rootless safepoint costs essentially nothing -- which is what docs/engine-plan.md already says: "the axis is not 'statepoints are bigger', it is 'roots are bigger'". Recording it as evidence: the safepoint-count lever is not the binary-size lever, so sequencing step 2 must attack live-root SETS. Two tests come with it. One pins the wrapper's classification to the barrier it wraps. The other pins js_nanbox_string OUT of the allowlist: at 120 call sites it is the obvious next candidate and reads as pure bit manipulation, but its null guard calls js_string_from_bytes to allocate an empty string. Probe suite 11/11 byte-identical under forced evacuation + verification. Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com> * fix(ci): report assertion pinned to a probe Windows cannot compile Three review fixes on #7368. The report assertion ran on 09_try_catch_roots, which contains four `try` blocks. RS4GC cannot rewrite WinEH funclet pads, so linker.rs's rs4gc_funclet_refusal rejects that probe on windows-msvc -- the probe loop above tolerates it by grepping the compile log for "funclet", but this step did not. A gate pinned to a probe that cannot compile on one arm fails for a reason unrelated to its subject. The portable assertion now uses 11_collect_at_depth (no `try`, compiles on all four arms); 09_try_catch_roots keeps its own non-Windows step so the try-specific coverage that justified deleting the bridge is not lost. The gc_map doc claimed records/roots would be ABSENT when unmeasured. They are plain u64 fields on a plain derive and always serialise; `modules` is the sentinel. Fixed to describe what the code actually does -- the same class of comment-vs-code drift this PR exists to clean up. The "map never reported" guard fired for any --require-*/--print, including fields that live in `totals` and are counted at IR-emission time whether or not the rewrite ran. Now scoped to map-backed fields: --require-positive textual_calls is answered from its measured value (verified exit 0) while --require-positive records still fails on an unreported map (exit 1). Claude-Session: https://claude.ai/code/session_01EaD6yNwoinzdW1JbYNkMMF --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The native-root stack walker had no probe that made it work.
Every probe in
benchmarks/gc_ratchet/probes/callsgc()at the end, from a shallow stack. Measured on04_dead_after_deep_stack— the probe whose name promises a deep stack — macOS and Linux report:Zero root locations. Both arms would have passed the entire suite unchanged with a walker that visited nothing at all, because other root sources cover those probes. The suite was green and proved nothing about the walker.
Windows is the only arm that walks a deep stack (5,626 frames / 5,449 records), and it does so by accident of heap sizing — its GC happens to trigger mid-recursion rather than at exit. That accident is why the
--require-locationsliveness gate from #7354 could be applied to Windows and nowhere else.This is CLAUDE.md's fourth failure mode almost exactly: "the gate runs but its subject never did."
The probe
11_collect_at_depthmakes deep-stack coverage deliberate and portable instead of incidental:Three properties make it a real test rather than a slower one:
mineis live across the recursive call, so at collection time there are ~220 roots resident, none of them in the leaf frame. A walker that stops after a few frames loses them.mine.value()runs aftergc()returns, so a missed or mis-based root is a wrong checksum, not an invisible near-miss.PERRY_GC_FORCE_EVACUATE=1moves every survivor, so a stale pointer cannot be accidentally right.Measured
Byte-matching
node --expose-gc --experimental-strip-typesat the pinned version:221 locations on both, from three separate deep cycles, then a final shallow
gc()that correctly reports 7 frames / 0 locations. The two arms agree on the root count while disagreeing on frame count by 3 — different prologue shapes abovemain, which is what you'd expect.Full ratchet suite: 11/11 byte-identical to the oracle under
PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1.The gate widening
With both Unix arms now walking a real stack,
gc_walker_trace_assert.py --require-locationsmoves out of itsif [ "$RUNNER_OS" = "Windows" ]branch onto the shared path, pointed at11_collect_at_depth. All three arms now have to prove the walker ran.I want to be explicit that the Windows-only scoping was not an oversight to be fixed by deleting the condition — it was correct at the time, because macOS and Linux could not have passed it. The probe is what makes the widening honest; without it, this PR would just be turning a gate red.
Risk
Test-only. No compiler or runtime code changes — the diff is one new probe and a moved
if.The one way this bites: if the walker regresses on a platform, the gate now fails there instead of passing silently. That is the intent.
Summary by CodeRabbit