Extract base substrate from Claude Code module#2
Open
Meldrey wants to merge 4 commits into
Open
Conversation
Extract _ensure_base_tables() with the generic flex storage contract (chunks, sources, edges, meta, presets, FTS) from the CC-specific extension tables (tool_ops, message types, delegations, soft_ops, file bodies). _ensure_core_tables() remains as a backward-compatible wrapper that calls both. This separation lets non-coding-agent modules (e.g. Matrix, Telegram) bootstrap cells without CC-specific table baggage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New insert_base_chunk(conn, chunk, source_type) writes only to the generic _raw_chunks and _edges_source tables. Any flex module can call this without needing CC-specific extension tables. insert_chunk_atom() now delegates to insert_base_chunk() for the base write, then adds CC-specific tables (types_message, tool_ops, delegations, soft_ops) as before. No behavior change for existing callers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add substrate parameter ('claude_code' or 'base') that controls which
tables are created. Base substrate gets only the generic chunk-atom
tables + content store. CC substrate adds coding-agent extension tables
and SOMA, matching previous behavior.
bootstrap_claude_code_cell remains as a backward-compatible alias.
Default substrate='claude_code' means zero behavior change for existing
callers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Modules can now declare substrate='base' in their spec to get a clean generic cell without CC-specific tables or enrichment. Changes: - coding_agent_install.py: read substrate from spec, pass to bootstrap_cell(), conditionally apply enrichment stubs/pipeline and use appropriate contract validator - contract.py: add REQUIRED_BASE_TABLES and validate_base_cell() for non-coding-agent modules - __init__.py: extract BASE_ENRICHMENT_STUBS (_ops, _views) from ENRICHMENT_STUBS. Full CC stubs = base + CC-specific. Backward compatible -- ENRICHMENT_STUBS still exports everything. This enables non-coding-agent modules (Matrix, Telegram, etc.) to use the flex module system without inheriting CC-specific schema, enrichment, or validation requirements. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ffe1f8c to
d0173e0
Compare
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: Extract base substrate from Claude Code module
The problem
Every flex module — regardless of what it ingests — must declare
substrate: "claude_code"and inherit CC-specific tables, enrichment, and validation. A Matrix chat module gets_edges_tool_opsand_types_messagetables it will never write to. A Telegram module would get_edges_delegationsfor agent spawns that don't exist in chat history.The generic storage layer (chunks, sources, content, FTS) is tangled with Claude Code's domain-specific extensions (tool operations, message threading, agent delegation, file bodies). New module authors must either accept the baggage or work around it.
The fix
Separate the generic flex storage contract from CC-specific extensions. Four changes, zero breakage:
1. Split
_ensure_core_tables()→_ensure_base_tables()+_ensure_cc_tables()The base tables are the shared contract any module needs:
_raw_chunks,_raw_sources,_edges_source— core chunk storage_meta,_presets— cell metadata and query presetschunks_fts— full-text searchThe CC tables are domain-specific:
_edges_tool_ops— tool call metadata (tool_name, target_file, success)_types_message— message threading (role, sidechain, branch)_edges_delegations— agent spawn tracking_edges_soft_ops,_file_body_index,_types_file_body_ensure_core_tables()remains as a backward-compatible wrapper.2. Extract
insert_base_chunk()frominsert_chunk_atom()insert_base_chunk(conn, chunk, source_type)writes only to_raw_chunksand_edges_source. Any module can call this without CC extension tables.insert_chunk_atom()callsinsert_base_chunk()internally, then writes CC-specific tables. No behavior change.3. Rename
bootstrap_claude_code_cell()→bootstrap_cell(substrate=...)New
substrateparameter:substrate='claude_code'(default) — base + CC + content + SOMA. Existing behavior.substrate='base'— base + content only. Clean generic cell.bootstrap_claude_code_cellremains as an alias.4. Make
run_from_spec()substrate-awaresubstratefrom the module spec (defaults to'claude_code')New exports:
BASE_ENRICHMENT_STUBS,validate_base_cell(),REQUIRED_BASE_TABLES.What this enables
A non-coding-agent module can now declare:
And get a clean cell with only the tables it needs. Its worker calls
insert_base_chunk()instead ofinsert_chunk_atom()and creates its own domain-specific extension tables.Backward compatibility
from ... import bootstrap_claude_code_cellfrom ... import insert_chunk_atomsubstrate: "claude_code"in any module specENRICHMENT_STUBSimportvalidate_coding_agent_cell()Tested
claude_codecell (134K chunks, 2.4K sources) queries correctlyhermes_agentcell queries correctlysubstrate='base'cell creates with 7 base tables, zero CC tables