Skip to content

128: Per-repo feature/test map: lazy, incremental bootstrap generation for customer repos#150

Merged
kjgbot merged 14 commits into
mainfrom
factory/128-agentworkforce-factory-08527a1d
Jul 21, 2026
Merged

128: Per-repo feature/test map: lazy, incremental bootstrap generation for customer repos#150
kjgbot merged 14 commits into
mainfrom
factory/128-agentworkforce-factory-08527a1d

Conversation

@agent-relay-code

@agent-relay-code agent-relay-code Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Factory dispatches agents against arbitrary customer repos with no equivalent of this repo's own .agentworkforce/features/manifest.yaml + critical-paths.md + verify/procedures.md. Those three files are how this repo tells an agent (human or LLM) what every feature is, where it lives, and what tier of prerequisite is required to verify it actually works. A dispatched implementer/reviewer/babysitter in a customer repo has no equivalent artifact, so "how do I test this" is answered ad hoc per task instead of from a maintained, versioned source of truth.

This issue is the foundation: generate that artifact for an arbitrary target repo, without requiring an expensive full-repo crawl up front.

Background

This repo's own feature map (.agentworkforce/features/manifest.yaml, critical-paths.md, verify/procedures.md) has three properties worth preserving in the generalized version:

  • a flat catalog of features (id, name, cli/api surface, description, location, verify_tier 1-6, category criticality)
  • a tiered verification model: tier 1 = nothing beyond a checkout, up to tier 6 = manual check against a live issue/PR, with a "verification by change area" table mapping code paths to minimum required tiers
  • narrative critical paths plus a hot paths table calling out sensitive code and what breaks if it regresses

That catalog was hand-authored (238 features across 19 categories) because it's this team's own product. We cannot hand-author this for every customer repo Factory touches — it needs to be machine-generated, and the only way to make that tractable is to generate it lazily: seed entries only for what's actually touched, rather than requiring a full-codebase indexing pass before Factory can be useful.

Proposed changes

A. New generation entry point

Add a new module (proposed: src/featuremap/generate.ts) that, given a target repo checkout path and a set of changed/touched file globs, produces or updates .agentworkforce/features/manifest.yaml inside that target repo's checkout — not this one. This runs as part of dispatch, most naturally as a step the implementer (or a dedicated pre-task agent) performs before or alongside implementation (see #131 for how it's consumed by task templates).

B. Bootstrap heuristics (v1, cheap)

For a repo with no .agentworkforce/features/ directory yet:

  • Do not crawl the whole repo. Scope discovery to the files the current issue's route/diff touches (TriageDecision routes already identify this — see src/triage/heuristic.ts).
  • For each touched file, infer a minimal feature entry: nearest exported CLI command / HTTP route / public function, a description synthesized from surrounding code/comments, location set to the actual file, and a verify_tier inferred from cheap signals (needs network/DB/auth to exercise → higher tier; pure/unit-testable → tier 1).
  • Reuse this repo's own manifest schema so downstream consumption and CI drift-checking don't need a second schema.

C. Incremental growth

On every subsequent dispatch against the same repo, extend rather than regenerate: read the existing manifest, add entries for newly-touched surfaces not already covered, and flag (but do not silently overwrite) entries whose location no longer exists — that's a drift signal for #132, not something this generation step should resolve unilaterally.

D. Cost/scope guardrails

  • Never block dispatch on generation completing; if the manifest can't be produced or updated in a bounded budget, proceed without it and log the miss (mirrors Factory's existing "exporter/telemetry failure is non-fatal" posture — see the observability section of README.md).
  • Cap manifest growth — this is meant to converge toward the touched-code footprint of real work, not attempt exhaustive coverage day one.

Acceptance criteria

  • Given a customer repo with no .agentworkforce/features/, dispatching an issue that touches N files produces a manifest with entries for those N files' relevant public surfaces, valid against the same schema this repo's own Tier-1 self-check parses (see .agentworkforce/features/verify/procedures.md lines 22-51 for the exact validation shape to satisfy: unique ids, required fields, valid verify_tier 1-6, location paths that exist).
  • A second dispatch against the same repo touching a mix of previously-seen and new files adds only the new entries and does not duplicate or destructively rewrite existing ones.
  • Generation failure or timeout never blocks or fails the underlying issue dispatch.
  • Unit tests cover: empty-repo bootstrap, incremental extension, and a stale-location entry being flagged rather than deleted.

Notes for the implementer

Related: #131 (consumes this manifest in task templates), #132 (CI drift-check on top of this manifest)

Fixes #128


Summary by cubic

Adds lazy, per-repo feature map generation that creates and incrementally updates .agentworkforce/features/manifest.yaml from touched files only, using the shared validator with a file lock. Supports v1.0 and v1.1 manifests, routing the generated category to existing verification procedures; exposes generateFeatureMap(...) and parsing helpers via dist and root exports. Fixes #128.

  • New Features

    • generateFeatureMap(...) and parseFeatureMapManifest(...); validator now returns categoryIds.
    • Touch-aware discovery with brace globs/negations and shallow bounds; ignores non-files/outside-repo and the manifest; preserves existing entries; appends to generated-touched-surfaces; flags stale locations.
    • Infers CLI commands, HTTP routes, and public APIs with adjacent comment descriptions; assigns verify_tier from lightweight signals.
    • v1.1 support: routes generated-touched-surfaces through an existing verification.categories procedure without changing hand-authored content.
    • Guardrails: non-fatal failures/timeouts; caps for files scanned, new entries, manifest size/bytes, and stale checks; serialized writes via a lock. Tests cover bootstrap, incremental/concurrent updates, malformed manifests, glob bounds, caps, stale-location handling, v1.1 routing, and category-order safety.
    • Added a self-catalog entry in this repo’s manifest; exported in src/featuremap/index.ts and src/index.ts.
  • Bug Fixes

    • Serialized concurrent manifest updates and reconciled generation with the shared validator.
    • Bounded shallow glob traversal and emit a warning when traversal is capped.
    • Isolated protected PID test fixture by providing a connectionPath in InternalFleetClient tests.

Written for commit fc12a3c. Summary will update on new commits.

Review in cubic

@kjgbot
kjgbot merged commit 23d625a into main Jul 21, 2026
2 checks passed
@kjgbot
kjgbot deleted the factory/128-agentworkforce-factory-08527a1d branch July 21, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Per-repo feature/test map: lazy, incremental bootstrap generation for customer repos

1 participant