Skip to content

skillgen: code_only check hard-codes 31 of 94 CODE_EXTENSIONS, so code-only updates pay for a semantic pass #2227

Description

@tihh07

Summary

The code_only check in the --update runbook hard-codes a set of code extensions that has drifted far from graphify.detect.CODE_EXTENSIONS. On 0.9.22 the literal covers 31 of 94 extensions. The consequence is a silent token cost: an incremental update that touched only code is classified code_only: False and takes the full semantic path, dispatching LLM subagents over a changed-file set that has nothing semantic in it.

Where

Source of truth is the shared fragment:

  • tools/skillgen/fragments/references/shared/update.md

which propagates to every host variant (graphify/skills/{pi,copilot,opencode,kilo,...}/references/update.md) and to the tools/skillgen/expected/* fixtures — 36 code-search hits for the literal in total.

code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts','.lua','.toc','.f','.F','.f90','.F90','.f95','.F95','.f03','.F03','.f08','.F08'}
...
code_only = all(Path(f).suffix.lower() in code_exts for f in all_changed)

What is missing

The literal is a strict subset of CODE_EXTENSIONS — nothing in it is stale, it simply never grew. The 68 missing entries include extensions that do have an AST extractor shipped in graphify/extractors/:

.sql .ps1 .psm1 .psd1 .jl .zig .tf .tfvars .hcl .pas .dpr .ex .exs .dart .groovy .cls .trigger .sv .v .m .mm .tsx .jsx .mjs .cjs .mts .cts .vue .svelte .astro .razor .cshtml .sh .bash .json .sln

Reproduction

--update on a repo where the only changed files are three .sql files:

code_only: False        # expected True

.sql is in CODE_EXTENSIONS, is dispatched to extractors/sql.py, and extracted deterministically by Step 3A — so the semantic pass that this misclassification triggers has nothing to do. On my run the AST step produced 12 nodes / 9 edges for those files; the semantic dispatch would have added zero and cost a subagent batch.

Suggested fix

Read the list from the installed package instead of restating it:

try:
    from graphify.detect import CODE_EXTENSIONS as code_exts
except ImportError:
    code_exts = {...}   # existing literal as fallback

Then regenerate tools/skillgen/expected/*. I am running this patched locally against 0.9.22 and the three cases that flip (.sql-only, .ps1-only, .R-only) all behave correctly; mixed code+doc and doc-only are unchanged.

One caveat worth deciding on

CODE_EXTENSIONS is not exactly the right predicate — it is broader than "has an extractor". .r is the clearest case: it is in CODE_EXTENSIONS but has no AST dispatch (#1689), so after this fix an R-only update becomes code_only: True and correctly skips the semantic pass, but then produces zero nodes on the AST path. The net outcome for R is unchanged (it yields nothing either way, since code-typed files are never queued for semantic extraction), so this fix does not regress anything — but it does mean the truly correct predicate is "extensions with a dispatchable extractor", i.e. gated on _get_extractor(...) rather than on the extension table. That distinction is the same one #1689 is about, and it might be worth exposing as a single source of truth both places can import.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions