VCR-VER-004: instrument independence — the ABI observable contract, a check that fails differently (#242) - #897
Open
avrabe wants to merge 9 commits into
Open
VCR-VER-004: instrument independence — the ABI observable contract, a check that fails differently (#242)#897avrabe wants to merge 9 commits into
avrabe wants to merge 9 commits into
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
…ls differently (#242) v0.53's VCR-DEC-001 lane proved BY MUTATION that emptying `cfg_exit_observable` — the exit contract the join-aware graph allocator and its own CFG validator SHARE — emits code leaving the return value in the wrong register, and that BOTH per-compilation validators accept it (`validate_cfg_rewrite` -> Ok, VCR-RA-003 `validate_final_allocation` -> Consistent). Only execution caught it. Two independent-LOOKING instruments, one shared blind spot. This is the first step of the response: a validator built on a different axis, not a second file on the same axis. `abi_contract::validate_abi_contract(orig, rewritten)`: - OBLIGATION CANNOT BE EMPTIED. The exit obligation is RETURN_CONTRACT_REGS = [R0, R1] — a constant of the AAPCS, hard-named here. Deleting `cfg_exit_observable` outright would not change one line of this check. - FORWARD, so it is structurally incapable of the v0.53 fail-open mode. `validate_cfg_rewrite` is a backward MUST-analysis whose obligation set is a VARIABLE, and the empty set is a fixpoint. A forward symbolic evaluation always produces exactly one value for R0 at each return, so there is always exactly one obligation per sink. There is no seed to shrink. - EVIDENCE IS A VALUE, not a register-name pair. Per side independently, build a value graph over Init(r) / Def(i,k) / Phi(b,n) nodes and compare by GREATEST-FIXPOINT BISIMULATION (partition refinement). On a deterministic term graph bisimilarity is equality of the infinite unfoldings, so loops and back edges are handled coinductively — no unrolling. - TAKES NOTHING FROM THE PASS. The signature is (orig, rewritten): the CFG is re-derived here from BOTH streams' label-form branch structure and the two must agree. The v0.50 join attempt failed by letting the pass hand the checker its own seed. - Init(r) is ONE node shared by both sides — that is the AAPCS PARAMETER half of the anchor, so reading a param out of the wrong register is a value change too. Documented in-module, not hidden: memory is not in the obligation (a store-only misrename is a named false NEGATIVE — the class `validate_cfg_rewrite` DOES cover when its seed is intact; the two instruments are complementary); `reg_effect` is still shared, so a mismodeled op remains a common blind spot; and the reason there is no SMT solver (for a renames-only rewrite the terms are ground applications of the same uninterpreted operators, so a query would decide exactly the structural equality the refinement already computes — and the synth-verify dependency edge runs the wrong way). 18 unit gates, including two FAIL-OPEN bugs this module's own red-first testing found and fixed: a `bl` that is not a block ENDER was walked past as a no-op, and every op is now classified wherever it sits. Pure addition — not yet wired into any pass, so zero emitted bytes change. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
…e gate + the mutation repro (#242) The instrument from the previous commit is now LOAD-BEARING, and v0.53's exact mutation is re-runnable as a committed artifact. - graph_alloc: both entry points (increment 1 straight-line, increment 2 joins) now require BOTH the dataflow validator AND `abi_contract::validate_abi_contract`. `NotAttempted` is a DECLINE, not an accept: if a decline counted as acceptance, a change that made the check universally inapplicable would disable it SILENTLY — the vacuity failure this lane exists to prevent. So: a function this pass applies to has been ABI-contract-certified, or it was not applied. - graph_alloc: announce the dataflow ACCEPT under SYNTH_GRAPH_ALLOC_STATS before the second gate runs. That line is what makes the new gate's rejection non-vacuous: it is an OBSERVATION that the pre-VCR-VER-004 compiler would have emitted the rewrite. - scripts/repro/mutations/v053_shared_exit_contract.patch: v0.53's mutation verbatim (empty `cfg_exit_observable` + drop the churn bias), committed so the red-first evidence is the ACTUAL mutation, not a reconstruction. - scripts/repro/vcr_ver_004_instrument_independence.py: applies that patch, rebuilds, and asserts on ONE compilation of ONE fixture that (1) validate_cfg_rewrite ACCEPTS, (2) VCR-RA-003 says Consistent, (3) VCR-VER-004 REJECTS with a concrete violation naming the ABI result register, and (4) the miscompile is therefore not emitted. It also asserts the BASELINE direction first — the unmutated compiler must still APPLY and must NOT be rejected — so a false-rejection regression fails the script before it ever reaches the mutation. Always restores the tree. MEASURED, on the whole ARM repro corpus (relocatable / label-form): functions measured 617, graph-alloc APPLIED 275, bytes 40822 -> 40776 (-46, -0.11%), 8 shrank / 1 grew which is v0.53's result to the BYTE. The gate — conservative enough to require BOTH R0 and R1 for every function, and to decline whenever it cannot analyze — costs ZERO reach on the unmutated compiler. The join execution differential is unchanged at 38/38, ENGAGED=13. Under the mutation: ENGAGED drops to 0, no wrong result is emitted, and the compiler names the reason — `Violated { sink: 81, reg: R0 }` on brif_outer_740::poll, the function that returned 0x1111 instead of its parameter. Frozen anchors unaffected: SYNTH_GRAPH_ALLOC is flag-off, so no shipped byte can move. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
…tures well-formed (#242) - ci.yml: `instrument-independence-oracle`, an ISOLATED job (the script mutates the tree and rebuilds). `set -o pipefail`, a machine-readable 4/4 assertion AND a non-zero OK-count grep (#890), plus a final `git diff --exit-code` so a mutation can never persist into the repo. - The script's own RED-FIRST evidence is now recorded in its docstring and is reproducible: change `abi_gate` to treat `Violated` as an accept, rebuild with the mutation applied, and the compiler prints `whole-function colouring APPLIED (validated)` and EMITS the miscompile (assertions (3) and (4) both fail). With the gate intact it declines. Verified by running it. - graph_alloc unit fixtures: two synthetic bodies had NO return terminator, so the new gate declined them. Fixed by making the fixtures WELL-FORMED FUNCTIONS (append the `pop {r4, pc}` epilogue), NOT by weakening the check — a stream with no return sink has no ABI contract to check, and passing it would be exactly the vacuous-green failure this lane exists to prevent. The appended epilogue sits at an index after every def index the second test reasons about, so its non-vacuity argument is untouched. This is the only false-REJECTION the gate produced anywhere: 617 real corpus functions, 275 applied, byte-identical to v0.53. cargo fmt --check / clippy --workspace --all-targets -D warnings / test --workspace: all exit 0. Frozen anchors 10/10. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
… and pin the result (#242) The gate so far guards the flag-OFF graph-colouring spike. Users do not compile with that. This asks the same question of `reallocate_function_post_exhaust` — the allocator every `synth compile` runs — and turns the answer into an enforced CI floor. MEASURED (617 corpus functions, --relocatable): Holds 376 · NotAttempted 241 · Violated 0 declines: unmodeled-op 159, call 62, indirect-call 11, numeric-offset-branch 9 So a value-level, ABI-anchored, forward check can PROVE the observable return contract on 61% of the shipping path today, with ZERO false rejections against an allocator that is frozen-pinned and execution-differentialed — i.e. known-good. - arm_backend: `SYNTH_ABI_CONTRACT_AUDIT=1`, report-only. Report-only DELIBERATELY: making it gate the default path means hard-erroring a user's compile on a checker whose false-positive rate is MEASURED, not proven. The honest sequence is measure first, flip on evidence — and this is the measurement. - scripts/repro/vcr_ver_004_shipping_path_audit.py + CI: asserts ZERO Violated (a violation is either a real shipping miscompile or a false rejection by the new instrument — neither may pass silently) and a pinned Holds FLOOR of 340, so a change that makes the checker see LESS of the shipping path is visible rather than absorbed. Coverage is the honest weak point of a checker that declines on calls and unmodeled ops, so coverage is what gets pinned. RED-FIRST by real exit code, both directions: a stub binary emitting a `Violated` line -> exit 1 naming the fixture; a stub emitting NOTHING -> exit 1 on the vacuity assertion, so a stale binary or an un-wired hook cannot pass while gating nothing. Real binary -> exit 0. cargo fmt --check / clippy --workspace --all-targets -D warnings / test --workspace: all exit 0. Report-only hook, so no emitted byte moves. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
…honest summary (#242) The FEATURE_MATRIX honest summary already recorded this exact residual — "a wrong-return-register rewrite is accepted by validate_cfg_rewrite AND VCR-RA-003 *both*, caught only by execution". It is now marked CLOSED STATICALLY, with the evidence (a CI job re-running v0.53's own mutation), and — more importantly — with what the lane did NOT close spelled out rather than implied away: * gate on the flag-OFF spike; on the DEFAULT path an audit held to a CI floor (Holds 376 / NotAttempted 241 / Violated 0 of 617), because hard-erroring a user's compile on a checker whose false-positive rate is MEASURED rather than proven is a flip that wants its own evidence; * memory is not in the obligation — a store-only misrename is a false NEGATIVE here, and is the class validate_cfg_rewrite covers when its seed is intact (complementary, not redundant); * `reg_effect` is still shared, so a MISMODELED OP remains a blind spot common to all three instruments. Until synth-verify's independent ArmSemantics is pinned against it, "three independent validators" would be an OVERCLAIM, and the summary says so. docs/status/FEATURE_MATRIX.md regenerated with the sanctioned `claim_check.py --emit-status` (never hand-edited, #805); the diff is additive only. claim_check: 34/34 hold. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
avrabe
force-pushed
the
lane/l6-instrument-independence
branch
from
July 30, 2026 17:21
3437f4b to
7dbfbc8
Compare
…242, #896) L4 (#896) taught the graph allocator to colour ACROSS CALLS, taking 57 of 68 `call` declines. My gate declined every one of them (`NotAttempted { reason: "call" }`), which would have taken increment 3's reach to zero — L4's own `colours_across_a_call` was the visible symptom. Modelled rather than exempted. `abi_gate` still treats `NotAttempted` as a DECLINE; what changed is that a call is no longer un-analyzable. - `classify` gains `Cf::CallOp`, keyed on `liveness::call_effect` returning `Some` rather than on a variant list, so this classifier and the pass's notion of "a call this allocator may colour across" CANNOT DRIFT. Reusing L4's single definition is the point: two divergent call models would be a fresh instance of the shared-blind-spot class this module attacks. - A call FALLS THROUGH (it does not end a basic block) and is required byte-identical on both sides — the same rule `validate_cfg_rewrite` enforces, so `blx`'s target register is never renamed under us. - The forward walk needed no special case, which is the design paying off: each clobbered register `{R0-R3, R12, LR}` is rebound to a FRESH `Def(call_i, k)` node whose operands are the PRE-CALL argument values, while `R4-R11`/`SP` flow through untouched. So a call result is an opaque value equal across the sides exactly when the same callee got the same arguments, and a value the rewrite parked in caller-saved scratch across the call is rebound to the call's own node — it can no longer bisimulate with what the original delivers at the return. That is L4's warning discharged as a VALUE disagreement, with no liveness reasoning anywhere: `recolouring_a_live_value_into_caller_saved_across_a_call_is_violated`. - `Call`/`CallIndirect` PSEUDO-ops still decline (`call-pseudo-op`): they expand downstream into a guard + table load + `blx`, so this stream's register footprint is not the final code's. 6 new unit gates: a call is modelled not declined; callee-saved → other callee-saved holds; the caller-saved-across-a-call recolour is VIOLATED; returning the call result directly holds; renaming what feeds an argument is VIOLATED (this is what `call_effect`'s `uses` buy); a rewritten call target declines. The pre-existing mid-block decline test was repointed at `BrTable`, since `bl` is no longer the unanalyzable case. Also documented: a call's MEMORY effect is not modelled — a restatement of the standing memory limitation at the place it is easiest to forget. Sound for a renames-only rewrite (both sides run the same callee at the same point); not a claim about the callee's effects. cargo fmt --check / clippy --workspace --all-targets -D warnings / test --workspace: all exit 0, incl. L4's `colours_across_a_call`. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
… tree (#242) Context-only rebase — the two SEMANTIC edits are byte-identical (empty `cfg_exit_observable`; drop the churn bias in `color_webs_biased`). Increment 3 (#896) moved the surrounding lines, so the old hunks no longer applied and the acceptance script FATAL'd rather than silently skipping. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
The call model changes all three headline measurements, so all three are re-derived rather than left stale. - Corpus reach: 275 applied / -46 B -> 307 applied / -100 B relocatable (-120 B self-contained). That is increment 3's result TO THE BYTE: the gate composes with #896 at zero cost, still zero false rejections. - Shipping-path audit: Holds 376 -> 422, NotAttempted 241 -> 195, Violated 0 (unchanged). ~61 % -> ~68 % of the default path now carries a proven observable return contract. The 62 `call` declines are GONE, not reclassified; `unmodeled-op` rose 159 -> 174 because functions that used to stop at the call now get further and reach an FP / i64-pair op instead. The decline MOVED, it did not vanish — said in both the script docstring and the CHANGELOG, because "declines dropped by 46" would be the flattering half of the truth. - HOLDS_FLOOR 340 -> 400. A floor left at the pre-call-model value would have silently absorbed losing the entire call model again, which is the same absorbed-regression failure the floor exists to prevent. FEATURE_MATRIX honest summary re-derived via the sanctioned `claim_check.py --emit-status` (never hand-edited, #805). claim_check: 35/35 hold. fmt / clippy -D warnings / test --workspace: exit 0. Frozen anchors 10/10. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
L1's oracle-wiring gate landed on main after this lane wrote its scripts, so they had no `# ci-status:` header and Claim Check went red: oracle wiring: 154 repro scripts — 145 wired, 7 manual, 0 unwired, 2 UNDECLARED FAIL vcr_ver_004_instrument_independence.py: UNDECLARED FAIL vcr_ver_004_shipping_path_audit.py: UNDECLARED The gate working on its first real encounter, exactly as intended. `wired` is the honest declaration here, verified rather than assumed -- each has a real `ci.yml` reference. Gate now: 154 scripts, 147 wired, 7 manual, 0 unwired, 0 UNDECLARED, exit 0. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The finding this responds to
v0.53's VCR-DEC-001 lane proved by mutation that emptying
cfg_exit_observable— the exit contract the join-aware graph allocator and its own CFG validator share — makes the compiler emit code leaving the return value in the wrong register, and that both per-compilation validators accept it:validate_cfg_rewrite(the pass's acceptance oracle)Okvalidate_final_allocationConsistentTwo independent-looking instruments, one shared blind spot. That is #872 one level up, and it undercuts a claim this project leans on publicly: that per-compilation validation is an independent check on the code generator.
The thesis, and what it produced
Independence is not obtained by writing a second checker, only by checking a different WAY. Both existing instruments reason about liveness/dataflow equations over register names, backward, seeded from a shared table. Agreeing costs them almost nothing.
abi_contract::validate_abi_contract(orig, rewritten)differs on four axes:RETURN_CONTRACT_REGS = [R0, R1]— a constant of the AAPCS, hard-named in the module's own source. Deletingcfg_exit_observableoutright would not change one line of it.validate_cfg_rewriteis a backward MUST-analysis whose obligation set is a variable, and ∅ is a fixpoint — an empty seed means zero obligations means vacuous green. A forward evaluation always produces exactly one value forR0at each return. There is no seed to shrink.Init(r)/Def(i,k)/Phi(b,n)) compared by greatest-fixpoint bisimulation. On a deterministic term graph that is equality of the infinite unfoldings, so loops and back edges are handled coinductively — no unrolling.Init(r)is one node shared by both sides: the AAPCS parameter half of the anchor.Result — the mutation is caught statically
Re-running v0.53's exact mutation (committed as
scripts/repro/mutations/v053_shared_exit_contract.patch, not reconstructed), on one compilation of one fixture:brif_outer_740::pollis the function that returned0x1111instead of its parameter. Assertion (1) is load-bearing: it is what makes (3) non-vacuous, because it observes that the pre-VCR-VER-004 compiler would have emitted this.Red-first, verified: make
abi_gatetreatViolatedas an accept, rebuild with the mutation applied, and the compiler printswhole-function colouring APPLIED (validated)and emits the miscompile again. The gate is load-bearing, not decorative.False rejection: none
The brief's other warning (v0.50's spurious
JoinValueNotAvailable{R8}) was taken seriously. Over the ARM repro corpus:Increment 3's result to the byte — and the check is conservative enough to demand both
R0andR1for every function and to treatNotAttemptedas a decline. The only false rejections anywhere were two synthetic unit fixtures with no return terminator; those were fixed by making the fixtures well-formed functions, not by weakening the check.Measured against the shipping allocator, not just the flag-off spike
SYNTH_ABI_CONTRACT_AUDIT=1onreallocate_function_post_exhaust— the allocator everysynth compileruns:The observable return contract proven on ~68 % of the shipping path, zero false rejections against a known-good allocator. Held to a CI floor (zero violations + a pinned
Holdscount, so lost coverage is visible rather than absorbed), red-first in both directions including the vacuity case where the hook is not wired at all.Follow-up: composed with L4 / increment 3 (#896) — calls are now modelled
Increment 3 taught the allocator to colour across calls, taking 57 of 68
calldeclines. This gate declined every one of them (NotAttempted { reason: "call" }), which would have taken increment 3's reach to zero — its owncolours_across_a_calltest was the visible symptom.Fixed by modelling the call, not by weakening
abi_gate(NotAttemptedis still a decline). The AAPCS effect is reused fromliveness::call_effect— the one definition the pass andvalidate_cfg_rewritealready share — because two divergent call models would be a fresh instance of the very blind-spot class this lane attacks.The forward design needed no special case: each clobbered register
{R0-R3, R12, LR}is rebound to a fresh node whose operands are the pre-call argument values, whileR4-R11/SPflow through. So a call result is opaque and equal across the sides exactly when the same callee got the same arguments, and a value the rewrite parked in caller-saved scratch across a call is rebound to the call's own node — it can no longer bisimulate with what the original delivers at the return. Increment 3's warning ("a validator treatingblas effect-free would accept a non-identity equation across it") is discharged as a value disagreement, with no liveness reasoning.Call/CallIndirectpseudo-ops still decline.Re-verified after the change:
Violated { sink: 81, reg: R0 }(the mutation patch was regenerated against increment 3's tree; the two semantic edits are byte-identical, only context moved);Holds376 → 422,NotAttempted241 → 195,Violated0 (~61 % → ~68 %). The 62 direct-call declines are gone, not reclassified;unmodeled-oprose 159 → 174 because functions that used to stop at the call now reach an FP / i64-pair op instead. The decline moved, it did not vanish.HOLDS_FLOORraised 340 → 400 so a floor left at the pre-call-model value cannot silently absorb losing the model again.6 new unit gates, including the caller-saved-across-a-call recolour (VIOLATED) and renaming what feeds a call argument (VIOLATED — this is what
call_effect'susesbuy).What this does NOT close — in FEATURE_MATRIX, not just in my head
validate_cfg_rewritedoes cover when its seed is intact. Complementary, not redundant. Pinned by a unit test so it cannot drift silently.liveness::reg_effect. A mismodeled op remains a blind spot common to all three. This closes the shared-contract hole, not the shared-op-model hole.synth-verify'sArmSemantics::encode_opis a genuinely second model; until it is pinned againstreg_effect, "three independent validators" would be an overclaim — and the honest summary says so.No SMT, no opt-out — both deliberate
A renames-only rewrite changes no opcode and no immediate, so both sides' terms are ground applications of the same uninterpreted operators; deciding equality is congruence closure with no side conditions — exactly the structural equality the refinement already computes. A solver would cost a dependency (and the
synth-verify→synth-synthesisedge runs the wrong way) to decide the same thing. An opt-out env var would be a footgun, not evidence.Verification
cargo fmt --check/clippy --workspace --all-targets -D warnings/cargo test --workspace— all exit 0 by real exit code. Frozen anchors 10/10.claim_check34/34.docs/status/FEATURE_MATRIX.mdregenerated with the sanctioned--emit-status, never hand-edited (#805). Two CI jobs added, bothset -o pipefail+ non-zero-count greps (#890).Two fail-open bugs in the new checker were found by its own red-first testing and fixed: a
blthat was not a block ender was walked past as a no-op, and unclassified control flow could slip through as "straight-line with no effect".🤖 Generated with Claude Code
https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L