fix(cache): re-anchor cached ids so a warm hit can't replay another root (#2257) - #2285
fix(cache): re-anchor cached ids so a warm hit can't replay another root (#2257)#2285Kaushik2003 wants to merge 1 commit into
Conversation
…oot (Graphify-Labs#2257) Extractors mint node ids from the path STRING they are handed (_make_id(str(path)), _file_node_id(path)), so an AST cache entry written under root A embeds A's slug in every id and edge endpoint. save_cached relativized only source_file, never the ids. Because extract()'s id-remap / final-canonicalization passes key their rewrites off the CURRENT run's paths, an A-derived id matches no key on a warm hit under root B (a clone, a moved checkout, a second mount) and the stale machine slug survives into graph.json. Distinct from Graphify-Labs#2231/Graphify-Labs#2243, which fix producers on a cold run, and from Graphify-Labs#2199 (stat-index portability). Entries are now stored root-anchored and re-anchored on read, the same store-portable/re-anchor-on-load contract source_file (Graphify-Labs#777) and the stat index (Graphify-Labs#2199) already use: _relativize_ids_in replaces the root's contribution with a $graphify-root$ marker on write, _absolutize_ids_in restores what the current run's extractor would mint on read. That is the pre-remap form every downstream pass in extract() expects, so a replay reproduces a cold run exactly and no other pass changed. The anchor is derived per entry rather than assumed equal to the scan root (normalize_id distributes over path joins), so a symlinked root or relative inputs decompose exactly; only absolute root spellings may anchor, or a relative root ("src") would rewrite an already-canonical src_utils_foo into an absolute-derived id on the semantic path. The walk covers the whole payload rather than a bucket list, since the id form is self-identifying: that also reaches raw_calls[].caller_nid, swift_extensions[].nid, edges[].target_file, bash_sources[].source_file and *_type_table.path, the last three being resolution inputs that would otherwise still point at root A. save_cached's deepcopy is now unconditional; the old truthiness gate skipped it for a payload whose only content lives outside nodes/edges, which would have let the transform mutate the caller's dict and break cold-run remapping. Pre-fix entries carry no marker and their content hash never changes, so they cannot self-heal; they are swept when the release bumps the version, since AST entries live under cache/ast/v{version}/. Tests: extract a python/C/bash/markdown corpus under root A, copy the tree and graphify-out to root B, extract under B on the warm cache, and assert the run is genuinely warm (zero extractor calls), that no node id or edge endpoint carries A's slug, that the on-disk entries hold neither A's slug nor an absolute path, that a cold run still yields canonical ids, and that warm and cold match exactly. The fixture avoids JS/TS on purpose: those suffixes bypass the cache, which would make the warm assertions vacuous.
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This pull request introduces root-portability for cache entries by storing node ids and path values relative to a sentinel $graphify-root$ marker instead of the absolute root a cache was originally written under. New helpers (_id_anchor, _portability_anchors, _rewrite_strings, _relativize_ids_in, _absolutize_ids_in) strip the current root's slug on write and re-anchor it to the active run's root on read, with save_cached/load_cached wired to call them alongside the existing source_file relativization. It also changes the deep-copy in save_cached from being gated on populated node/edge buckets to being unconditional for dict results. The accompanying test suite additions cover portability across roots, symlinked roots, legacy unversioned/absolute entries, frontmatter/body hashing, semantic cache merge/prune/clear behavior, and namespace handling.
Worth a look
- id restore drops marker when id_restore is empty, silently changing id —
graphify/cache.py· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- _absolutize_ids_in re-anchors any string starting with marker, including legitimate user data —
graphify/cache.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 847 functions depend on the 192 node(s) this change touches.
Health — this change adds coupling hotspots:
- worse:
load_cached()— 39 callers, 5 callees - worse:
save_cached()— 24 callers, 5 callees - worse:
_flush_stat_index()— 6 callers, 2 callees
Verification — 847 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 562 function(s) in the blast radius were not formally verified this run
· 3 more finding(s) on lines outside this diff (see the check run).
Fixes #2257.
Problem
Extractors mint node ids from the path string they are handed (
_make_id(str(path)),_file_node_id(path)), so an AST cache entry written under root A embeds A's slug in every id and edge endpoint.save_cachedrelativized onlysource_file— never the ids.Because
extract()'s id-remap / final-canonicalization passes key their rewrites off the current run's paths, an A-derived id matches no key on a warm hit under root B (a clone, a moved checkout, a second mount), so the stale machine slug survives intograph.json.Reproduced before touching any code — the same corpus, warm vs cold under root B:
This is distinct from #2231/#2243 (which fix producers on a cold run) and from #2199 (stat-index portability) — it is the AST result cache replaying pre-canonicalization ids.
Fix
Store entries root-anchored, re-anchor on read — the same store-portable / re-anchor-on-load contract
source_file(#777) and the stat index (#2199) already use:_relativize_ids_in(insave_cached, after thesource_filepass) replaces the root's contribution with a$graphify-root$marker._absolutize_ids_in(inload_cached) restores what the current run's extractor would mint — the pre-remap form every downstream pass expects, so a replay reproduces a cold run exactly and no other pass changed.Details worth reviewing:
normalize_iddistributes over path joins), so a symlinked root (macOS/tmp→/private/tmp) or relative inputs decompose exactly.root("src") is a strip candidate but never a restore candidate, so admitting it would rewrite an already-canonicalsrc_utils_foointo<abs-root-slug>_utils_fooon thesave_semantic_cachepath (which forwardsrootunresolved). Covered by a regression test.raw_calls[].caller_nid,swift_extensions[].nid,edges[].target_file,bash_sources[].source_fileand*_type_table.path— the last three being resolution inputs that would otherwise still point at root A and diverge warm-vs-cold. Values only, never dict keys.save_cached's deepcopy is now unconditional. The old truthiness gate skipped it for a payload whose only content lives outsidenodes/edges(emptynodesbeside a populatedbash_sources), which would have let the transform mutate the caller's dict — andextract()'sid_remapis keyed on the absolute form, so that would break cold runs.A cache entry on disk now looks like:
{ "id": "$graphify-root$_run_sh__entry", "source_file": "run.sh", "target_file": "$graphify-root$/lib/common.sh" }Tests
tests/test_cache.py:test_warm_cache_from_another_root_does_not_leak_that_root— the issue's scenario. Extract a python/C/bash/markdown corpus under root A, copy the tree andgraphify-outto root B, extract under B on the warm cache. Asserts the run is genuinely warm (zero extractor invocations), no node id or edge endpoint carries A's slug, the on-disk entries hold neither A's slug (case-insensitive — ids are casefolded, paths are not) nor an absolute path, a cold run still yields canonical ids, and warm ≡ cold._JS_CACHE_BYPASS_SUFFIXESand never cached, which would make every warm assertion vacuous.test_cached_ids_round_trip_under_the_same_root— the stored form must restore to the exact absolute-derived id, or a same-root warm hit would breakextract()'s remap.test_relative_root_does_not_reanchor_an_already_canonical_id— the strip/restore asymmetry above.Full suite: 3787 passed. The 4
test_ollama_retry_cap.pyfailures are pre-existing in my environment (openainot installed) — confirmed by stashing this branch and re-running.Additionally verified out-of-band on graphify's own 334-file tree (216 cached files, 0 warm misses, warm graph identical to cold at 3067 nodes / 5865 edges), across the default layout,
--out(decoupledcache_root), a symlinked root, and the parallel process pool.Note for the maintainer
Pre-fix entries carry no marker and their content hash never changes, so they cannot self-heal. They are swept when the release bumps the version, since AST entries live under
cache/ast/v{version}/— so the version bump is part of this fix landing, not incidental to it. Per repo convention this PR does not touchCHANGELOG.mdor the version itself.