[Schema Consistency] Schema Consistency Check — 2026-05-22

[github-actions[bot]]

Overview

Automated cross-check of the main workflow JSON schema (pkg/parser/schemas/main_workflow_schema.json) against the parser/compiler (pkg/parser/*.go, pkg/workflow/*.go), the curated documentation (docs/src/content/docs/reference/frontmatter.md), and the 200+ real workflow files in .github/workflows/. Findings re-confirm three high-signal patterns from prior runs and surface one resolved item — infer has been removed from the parser allowlist, so that previously-reported leak is now fixed.

Summary

• Inconsistencies confirmed: 4 categories, ~13 distinct issues
• Categories analyzed: Schema ↔ Parser, Schema ↔ Docs, Parser ↔ Docs, Schema ↔ real workflows
• Strategy bucket: proven (day-of-year mod 10 = 2)
• Strategies used: Weekly Research Report: AI Workflow Automation Landscape and Market Opportunities - August 2025 #7 (doc coverage), Add workflow: githubnext/agentics/weekly-research #8 (deprecation enforcement), Weekly Research Report: AI Workflow Automation Landscape and Strategic Opportunities - August 2025 #9 (validFields cross-check), Remove ai-inference, opencode, genaiscript agentic engines for now #10 (deprecation flag consistency)
• New finding: 0 (re-confirmations only)
• Resolved since last run: 1 (infer removed from include_processor.go)

Critical Issues

1. applyTo and inputs accepted by parser but absent from schema

pkg/parser/include_processor.go:182-204 defines a validFields allowlist that includes applyTo and inputs for shared/imported workflows, but neither field exists in pkg/parser/schemas/main_workflow_schema.json. IDEs configured against the schema will flag these as unknown fields even though the parser accepts them silently. Either add them to the schema (with a constraint scoping them to imported files) or rename them in the parser.

2. rate-limit legacy alias accepted with no user-facing warning

pkg/workflow/role_checks.go:219-222 accepts the legacy rate-limit key and falls back to it when user-rate-limit is absent. The only signal at line 300-302 is roleLog.Print("Extracted legacy rate-limit configuration") — a debug log, not a console warning. Users migrating to user-rate-limit get no compile-time nudge, and gh aw compile output stays silent. Recommend emitting console.FormatWarningMessage similar to the existing pattern used for unexpected fields in include_processor.go:217.

High-Priority Inconsistencies

3. Schema describes fields as deprecated but lacks structured flags

A jq scan of properties confirms zero fields in the main schema have "deprecated": true or "x-deprecation-message". Two fields self-describe as deprecated/legacy in prose only:

• inline-sub-agents — description: "Deprecated switch for inline sub-agent support... Setting this to false is not supported and causes a compilation error." (Note: the compiler hard-rejects false at pkg/workflow/compiler_validators.go:60, so the schema is doubly misleading — false is documented as a valid example but rejected at compile time.)
• rate-limit — description: "Legacy alias for 'user-rate-limit'. Prefer 'user-rate-limit' with 'max-runs-per-window'."

Without the structured flags, JSON-schema-aware IDEs (VS Code, IntelliJ) cannot render the strikethrough / hint UI that signals deprecation to users.

4. inline-sub-agents example value false violates compiler

From the schema:

"inline-sub-agents": {
  "type": "boolean",
  "description": "... Setting this to false is not supported and causes a compilation error.",
  "examples": [true]
}

Good — the example is true. But the type is boolean with no const: true or enum: [true], so the schema still validates inline-sub-agents: false even though the compiler will reject it at compiler_validators.go:60. Tighten to "const": true or "enum": [true] so schema validation catches the error before the compile step.

Documentation Gaps

5. Ten schema fields have zero mentions in curated frontmatter.md

The authoritative narrative doc docs/src/content/docs/reference/frontmatter.md has 0 mentions of each of these fields, despite them being in the schema and (for most) actively used in real workflows:

Field	Used in .github/workflows/*.md?	Notes
tracker-id	89 workflows	Heaviest gap — heavily used, completely undocumented in curated docs
experiments	yes (see field_gaps)
user-rate-limit	yes	replaces legacy rate-limit
rate-limit	no (in repo)	legacy alias
check-for-updates	(boolean)
disable-model-invocation	(boolean, agent file)
run-install-scripts	(boolean)
secret-masking	(object)	security-relevant
inline-sub-agents	(boolean)	deprecated but still documented in schema
inlined-imports	(boolean)

The auto-generated frontmatter-full.md is not a substitute for narrative documentation — users discovering features grep the curated page first.

Schema ↔ Real-World Workflows

6. Heavy tracker-id usage with no curated docs entry

$ grep -l "^tracker-id:" .github/workflows/*.md | wc -l
89

89 of the project's own workflow files set tracker-id — yet docs/src/content/docs/reference/frontmatter.md has zero mentions of the field. This is the single most impactful documentation gap in the audit.

Resolved Since Last Run

• infer field: Previously flagged (cache strategy-8) as silently accepted by include_processor.go. A grep at pkg/parser/include_processor.go for infer now returns no matches — it has been removed from validFields. ✅

Recommendations

1. Schema: Add "deprecated": true and "x-deprecation-message" to both inline-sub-agents and rate-limit. Change inline-sub-agents to "const": true (or "enum": [true]) since false is a compile-time error.
2. Schema: Either add applyTo and inputs as top-level properties (scoped to imported workflows via if/then or allOf) or rename them in pkg/parser/include_processor.go to match an existing schema field.
3. Compiler: In pkg/workflow/role_checks.go around line 300, replace roleLog.Print("Extracted legacy rate-limit configuration") with a user-visible console.FormatWarningMessage recommending migration to user-rate-limit.
4. Docs: Add narrative sections in docs/src/content/docs/reference/frontmatter.md for the 10 missing fields, prioritising tracker-id (89 workflows depend on it).
5. Tooling: Consider adding a CI check that fails when a schema field is added without a corresponding entry in the curated reference doc — the auto-generated frontmatter-full.md shouldn't be the only documentation.

Strategy Performance

Strategy	Findings	Effectiveness	Reuse
#7 Documentation coverage audit	10	high	yes
#8 Deprecation enforcement check	2 (one resolved)	high	yes
#9 Parser validFields vs schema	2	high	yes
#10 Deprecation flag consistency	2	high	yes

All four strategies produced confirmable, actionable findings. No new strategy attempted this run (day_mod=2 → proven bucket). Cache updated to reflect that infer is resolved and to bump success counts.

Next Steps

• Add deprecated: true / x-deprecation-message to inline-sub-agents and rate-limit in pkg/parser/schemas/main_workflow_schema.json
• Tighten inline-sub-agents schema constraint from boolean to const: true
• Reconcile applyTo / inputs between parser allowlist and schema
• Surface rate-limit legacy usage with a console warning in role_checks.go
• Document tracker-id (and the 9 other gap fields) in docs/src/content/docs/reference/frontmatter.md

References:

• §26272433865

Generated by ✅ Schema Consistency Checker · ● 5.3M · ◷

• expires on May 23, 2026, 6:40 AM UTC