Skip to content

Extract base substrate from Claude Code module#2

Open
Meldrey wants to merge 4 commits into
damiandelmas:mainfrom
Meldrey:feat/base-substrate
Open

Extract base substrate from Claude Code module#2
Meldrey wants to merge 4 commits into
damiandelmas:mainfrom
Meldrey:feat/base-substrate

Conversation

@Meldrey

@Meldrey Meldrey commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

PR: Extract base substrate from Claude Code module

The problem

Every flex module — regardless of what it ingests — must declare substrate: "claude_code" and inherit CC-specific tables, enrichment, and validation. A Matrix chat module gets _edges_tool_ops and _types_message tables it will never write to. A Telegram module would get _edges_delegations for agent spawns that don't exist in chat history.

The generic storage layer (chunks, sources, content, FTS) is tangled with Claude Code's domain-specific extensions (tool operations, message threading, agent delegation, file bodies). New module authors must either accept the baggage or work around it.

The fix

Separate the generic flex storage contract from CC-specific extensions. Four changes, zero breakage:

1. Split _ensure_core_tables()_ensure_base_tables() + _ensure_cc_tables()

The base tables are the shared contract any module needs:

  • _raw_chunks, _raw_sources, _edges_source — core chunk storage
  • _meta, _presets — cell metadata and query presets
  • chunks_fts — full-text search

The CC tables are domain-specific:

  • _edges_tool_ops — tool call metadata (tool_name, target_file, success)
  • _types_message — message threading (role, sidechain, branch)
  • _edges_delegations — agent spawn tracking
  • _edges_soft_ops, _file_body_index, _types_file_body

_ensure_core_tables() remains as a backward-compatible wrapper.

2. Extract insert_base_chunk() from insert_chunk_atom()

insert_base_chunk(conn, chunk, source_type) writes only to _raw_chunks and _edges_source. Any module can call this without CC extension tables.

insert_chunk_atom() calls insert_base_chunk() internally, then writes CC-specific tables. No behavior change.

3. Rename bootstrap_claude_code_cell()bootstrap_cell(substrate=...)

New substrate parameter:

  • substrate='claude_code' (default) — base + CC + content + SOMA. Existing behavior.
  • substrate='base' — base + content only. Clean generic cell.

bootstrap_claude_code_cell remains as an alias.

4. Make run_from_spec() substrate-aware

  • Reads substrate from the module spec (defaults to 'claude_code')
  • Applies enrichment stubs conditionally (base stubs vs full CC stubs)
  • Runs CC enrichment pipeline only for CC substrate
  • Validates against the appropriate contract (base vs coding-agent)

New exports: BASE_ENRICHMENT_STUBS, validate_base_cell(), REQUIRED_BASE_TABLES.

What this enables

A non-coding-agent module can now declare:

MODULE = {
    "cell_type": "matrix",
    "substrate": "base",
    "views_from": ("matrix",),
    "presets_from": ("matrix",),
    ...
}

And get a clean cell with only the tables it needs. Its worker calls insert_base_chunk() instead of insert_chunk_atom() and creates its own domain-specific extension tables.

Backward compatibility

Existing code Status
from ... import bootstrap_claude_code_cell Works (alias)
from ... import insert_chunk_atom Works (calls insert_base_chunk internally)
substrate: "claude_code" in any module spec Works (default)
ENRICHMENT_STUBS import Works (base + CC combined)
validate_coding_agent_cell() Works (unchanged)

Tested

  • Existing claude_code cell (134K chunks, 2.4K sources) queries correctly
  • Existing hermes_agent cell queries correctly
  • New substrate='base' cell creates with 7 base tables, zero CC tables
  • Base contract validation passes
  • All imports resolve, alias identity confirmed

Meldrey and others added 4 commits June 29, 2026 12:47
Extract _ensure_base_tables() with the generic flex storage contract
(chunks, sources, edges, meta, presets, FTS) from the CC-specific
extension tables (tool_ops, message types, delegations, soft_ops,
file bodies).

_ensure_core_tables() remains as a backward-compatible wrapper that
calls both. This separation lets non-coding-agent modules (e.g. Matrix,
Telegram) bootstrap cells without CC-specific table baggage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New insert_base_chunk(conn, chunk, source_type) writes only to the
generic _raw_chunks and _edges_source tables. Any flex module can
call this without needing CC-specific extension tables.

insert_chunk_atom() now delegates to insert_base_chunk() for the
base write, then adds CC-specific tables (types_message, tool_ops,
delegations, soft_ops) as before. No behavior change for existing
callers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add substrate parameter ('claude_code' or 'base') that controls which
tables are created. Base substrate gets only the generic chunk-atom
tables + content store. CC substrate adds coding-agent extension tables
and SOMA, matching previous behavior.

bootstrap_claude_code_cell remains as a backward-compatible alias.
Default substrate='claude_code' means zero behavior change for existing
callers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Modules can now declare substrate='base' in their spec to get a clean
generic cell without CC-specific tables or enrichment.

Changes:
- coding_agent_install.py: read substrate from spec, pass to
  bootstrap_cell(), conditionally apply enrichment stubs/pipeline
  and use appropriate contract validator
- contract.py: add REQUIRED_BASE_TABLES and validate_base_cell()
  for non-coding-agent modules
- __init__.py: extract BASE_ENRICHMENT_STUBS (_ops, _views) from
  ENRICHMENT_STUBS. Full CC stubs = base + CC-specific. Backward
  compatible -- ENRICHMENT_STUBS still exports everything.

This enables non-coding-agent modules (Matrix, Telegram, etc.) to
use the flex module system without inheriting CC-specific schema,
enrichment, or validation requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant