Skip to content

feat(runner): add declarative JSON gate policy - #34

Draft
jesse-merhi wants to merge 2 commits into
mainfrom
jesse/openclaw-native-gate-policy
Draft

feat(runner): add declarative JSON gate policy#34
jesse-merhi wants to merge 2 commits into
mainfrom
jesse/openclaw-native-gate-policy

Conversation

@jesse-merhi

@jesse-merhi jesse-merhi commented Jul 27, 2026

Copy link
Copy Markdown
Member

New behavior

Trusted profiles can now attach declarative rules to the JSON already emitted by any built-in or command-defined scanner. Scanner implementations do not need ClawScan-specific policy support, wrappers, or changed exit codes.

  • equals matches string, boolean, or exact JSON-number values.
  • exists: true fires when a path resolves to non-empty evidence.
  • Ordered path aliases, array traversal, per-object field aliases, identifier normalization, and preferred-root fallback cover scanner schema variants.
  • JSON and exit-code rules compose; block wins over warn.
  • Failed or skipped scanners do not fire policy, while valid evidence from ordinary nonzero scanner exits remains gate-eligible.
  • Raw scanner JSON remains byte-identical, judge execution is unchanged, and gate actions remain record-only.
  • A published JSON Schema now gives .clawscan.yml authors completion and early validation without replacing ClawScan's runtime checks.

The embedded clawhub profiles now express their SkillSpector and static-scanner policy with the same public rule format.

Profile and artifact proof

An immutable third-party scanner can be gated directly. The schema directive is optional and understood by YAML language-server integrations:

# yaml-language-server: $schema=https://raw.githubusercontent.com/openclaw/clawscan/main/schemas/clawscan.schema.json

version: 1
profiles:
  review:
    scanners:
      - id: third-party
        command: third-party scan --json {{target}}
        gate:
          rules:
            - id: critical-risk
              path:
                - result.risk
                - result.risk_level
              equals: critical
              normalize: identifier
              action: block
            - id: any-policy-violation
              path: result.violations[]
              exists: true
              action: warn

A fired JSON rule is recorded alongside the preserved scanner evidence:

{
  "gate": "block",
  "gateRules": [
    {
      "scanner": "third-party",
      "rule": "critical-risk",
      "path": "result.risk",
      "value": "CRITICAL",
      "action": "block"
    }
  ]
}

The shipped ClawHub policy blocks DO_NOT_INSTALL recommendations and CRITICAL findings, warns on HIGH findings, and warns when clawscan-static reports any finding. Preferred filtered finding collections override legacy raw collections when present.

How to verify

  1. Add the schema directive to a .clawscan.yml file and confirm an editor flags an unknown profile field, invalid scanner ID, missing {{target}}, or malformed gate rule.
  2. Resolve the same file with ClawScan and confirm supported profile syntax still loads normally.
  3. Feed fixture JSON that matches an equals or exists rule and inspect gate, gateRules, and the unchanged scanner raw value.
  4. Exercise ordered aliases with absent, null, empty, nonmatching, and matching values; confirm only the authoritative path controls the verdict.
  5. Return valid JSON with an ordinary nonzero exit and combine a JSON rule with blockOnExitCode; confirm both rules fire and the strongest action is recorded.
  6. Return valid JSON from a timeout, signal, or reserved infrastructure exit; confirm the evidence is preserved but does not gate.

Checks

  • env TMPDIR=/private/tmp go test -count=1 ./... — passed.
  • go vet ./... — passed.
  • make docs-site — passed.
  • Schema contract tests compile the Draft 2020-12 schema, validate the embedded ClawHub profile and every registered built-in scanner, and reject representative invalid scanner and gate declarations.

Implementation notes

The path language is intentionally small: dotted object fields, [] array traversal, and | field aliases. It is not a general JSONPath evaluator. Runtime parsing rejects unknown fields, duplicate IDs or paths, ambiguous conditions, invalid actions, unsupported normalization, and non-finite numeric values.

YAML-specific decoding and compatibility logic now live in internal/profiles/yaml.go; resolver.go remains focused on profile selection and CLI option resolution. The schema provides authoring feedback, while runtime validation remains the source of truth for execution.

@jesse-merhi
jesse-merhi force-pushed the jesse/openclaw-native-gate-policy branch from 422c7a3 to b9605f9 Compare July 27, 2026 15:30
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 29, 2026, 6:08 PM ET / 22:08 UTC.

ClawSweeper review

What this changes

The branch adds declarative JSON gate rules to profile-configured scanners, evaluates those rules against preserved scanner JSON, records matched rules in run artifacts, migrates the embedded ClawHub profile, and publishes a .clawscan.yml schema.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

Keep this PR open for explicit maintainer review. It is a MEMBER-authored, draft feature that intentionally changes the embedded clawhub profile from scanner collection into policy-derived warn and block artifact outcomes; that compatibility contract needs a maintainer decision before merge.

Priority: P3
Reviewed head: 2954e46b3893289cd0e0a9ff0887f88586f6a29a
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) The patch is broad but purposefully tested and documented; merge readiness is limited by the maintainer compatibility decision rather than a concrete correctness finding.
Proof confidence 🌊 off-meta tidepool Not applicable: This is a repository-member PR, so the external-contributor real-behavior-proof gate does not apply; the PR body nevertheless describes profile/artifact verification and reports focused validation. A maintainer should still inspect a redacted real artifact before accepting the built-in profile compatibility change.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This is a repository-member PR, so the external-contributor real-behavior-proof gate does not apply; the PR body nevertheless describes profile/artifact verification and reports focused validation. A maintainer should still inspect a redacted real artifact before accepting the built-in profile compatibility change.
Evidence reviewed 5 items Declared compatibility change: The PR description says the embedded ClawHub policy will block DO_NOT_INSTALL and CRITICAL results and warn on HIGH or any clawscan-static finding, creating new gate signals while raw scanner JSON remains unchanged.
Artifact contract surface: The diff expands fired gate-rule artifacts from exit-code-only entries to include rule identifiers, matched JSON paths, and matched JSON values, and the CLI summary renders those values.
Validation coverage supplied by PR: The branch adds focused resolver, runner, command-summary, scanner-adapter, and schema contract tests; the PR body reports go test -count=1 ./..., go vet ./..., and docs-site generation as passing.
Findings None None.
Security None None.

How this fits together

ClawScan resolves profile YAML into scanner runs, preserves each scanner's raw JSON evidence, and writes run artifacts consumed by CLI users and optional judge harnesses. This PR inserts profile-defined JSON policy evaluation between scanner evidence collection and the final gate/artifact summary.

flowchart LR
  A[Profile YAML] --> B[Profile resolver]
  B --> C[Scanner execution]
  C --> D[Raw JSON evidence]
  D --> E[Declarative gate rules]
  E --> F[Run artifact gate]
  F --> G[CLI summary and judge harness]
Loading

Decision needed

Question Recommendation
Should the embedded clawhub profile begin emitting warn and block gate outcomes from existing scanner JSON, including warnings for any clawscan-static finding? Approve the profile contract: Accept the new ClawHub gate semantics and merge with the artifact fields documented as supported operator output.

Why: The branch is technically scoped around a new declarative policy format, but changing a built-in profile's artifact semantics affects operators and downstream consumers even when scanner evidence is unchanged.

Before merge

  • Resolve merge risk (P1) - Merging changes the existing embedded clawhub profile so unchanged scanner JSON can now produce warn or block gate results in artifacts; downstream consumers may have treated the previous scanner-only profile as informational.
  • Resolve merge risk (P1) - The profile schema publishes a static list of built-in scanner IDs, so future scanner-registry additions need a matching schema update to avoid profile-authoring drift.
  • Complete next step (P2) - A maintainer must choose whether the embedded clawhub profile may begin publishing policy-derived gate outcomes for previously unchanged scanner evidence.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch surface 27 files; 2,416 added, 269 removed The feature spans profile parsing, runner artifacts, scanner adapters, documentation, schema validation, and tests rather than a single isolated policy hook.
Compatibility-bearing profile rules 4 embedded ClawHub JSON rules These rules alter gate outcomes for the repository's built-in ClawHub scanning profile.
New direct dependency 1 added (github.com/santhosh-tekuri/jsonschema/v6) The schema contract test and published configuration schema add a maintained dependency surface.

Merge-risk options

Maintainer options:

  1. Confirm the new built-in profile contract (recommended)
    Approve the new clawhub gate values only after deciding that downstream artifact consumers may rely on the resulting warn/block outcomes.
  2. Decouple engine and rollout
    Merge or revise the declarative engine separately while keeping the embedded ClawHub profile unchanged until compatibility is explicitly released.
  3. Pause for consumer evidence
    Keep the draft open if no maintainer can confirm that current ClawHub artifact consumers safely handle the added gate signals.

Technical review

Best possible solution:

Land the feature only after a maintainer explicitly accepts the embedded ClawHub gate-outcome compatibility change and confirms that artifact consumers should treat the new warn and block signals as part of the supported profile contract.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a feature PR, not a report of broken established behavior. The relevant validation is after-fix profile resolution and real scan-artifact output rather than a failing current-main reproduction.

Is this the best way to solve the issue?

Unclear: the declarative engine is a coherent way to avoid scanner-specific wrappers, but enabling it in the embedded clawhub profile changes an existing operator-visible artifact contract and needs explicit approval.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ef760b176da7.

Labels

Label justifications:

  • P3: This is a non-urgent configuration and artifact-policy feature awaiting maintainer direction.
  • merge-risk: 🚨 compatibility: The embedded clawhub profile changes from collecting scanner output to emitting new warn/block gate outcomes for existing evidence.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a repository-member PR, so the external-contributor real-behavior-proof gate does not apply; the PR body nevertheless describes profile/artifact verification and reports focused validation. A maintainer should still inspect a redacted real artifact before accepting the built-in profile compatibility change.

Evidence

What I checked:

  • Declared compatibility change: The PR description says the embedded ClawHub policy will block DO_NOT_INSTALL and CRITICAL results and warn on HIGH or any clawscan-static finding, creating new gate signals while raw scanner JSON remains unchanged. (internal/profiles/clawhub/clawscan.yml, 2954e46b3893)
  • Artifact contract surface: The diff expands fired gate-rule artifacts from exit-code-only entries to include rule identifiers, matched JSON paths, and matched JSON values, and the CLI summary renders those values. (internal/runner/runner.go, 2954e46b3893)
  • Validation coverage supplied by PR: The branch adds focused resolver, runner, command-summary, scanner-adapter, and schema contract tests; the PR body reports go test -count=1 ./..., go vet ./..., and docs-site generation as passing. (internal/runner/runner_test.go, 2954e46b3893)
  • Repository policy applied: The checked-in review guidance requires scanner evidence to remain raw, keeps gate effects record-only, treats scanner IDs as public CLI surface, and requires focused plus full Go validation for behavior changes; the proposed design and stated checks address those constraints. (AGENTS.md:1, ef760b176da7)
  • No implementation-close provenance: The supplied GitHub state identifies this as an open, cleanly mergeable draft PR at head 2954e46; it does not provide a merged replacement or release provenance showing the feature is already on current main. (2954e46b3893)

Likely related people:

  • jesse-merhi: Repository member and author of both commits in this PR; no separate current-main feature-history owner was established from the provided review material. (role: current proposal owner; confidence: medium; commits: d8e57e37a403, 2954e46b3893; files: internal/profiles/clawhub/clawscan.yml, internal/profiles/yaml.go, internal/runner/runner.go)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Confirm a representative redacted ClawHub run artifact shows the intended new gate and gateRules contract.
  • Record maintainer approval for changing the embedded profile from scanner collection to policy-derived gate outcomes.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (7 earlier review cycles)
  • reviewed 2026-07-27T15:33:48.159Z sha b9605f9 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-28T15:12:23.508Z sha d8e57e3 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-29T12:55:59.010Z sha 2954e46 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-29T18:11:45.107Z sha 2954e46 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-29T19:09:31.982Z sha 2954e46 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-29T20:12:26.421Z sha 2954e46 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-29T21:08:52.288Z sha 2954e46 :: needs maintainer review before merge. :: none

@jesse-merhi
jesse-merhi force-pushed the jesse/openclaw-native-gate-policy branch from b9605f9 to 916d6df Compare July 28, 2026 14:36
@jesse-merhi jesse-merhi changed the title feat(runner): add native built-in gate policy feat(runner): add declarative JSON gate policy Jul 28, 2026
@jesse-merhi
jesse-merhi force-pushed the jesse/openclaw-native-gate-policy branch from 916d6df to d8e57e3 Compare July 28, 2026 14:54
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jul 28, 2026
@jesse-merhi

Copy link
Copy Markdown
Member Author

/clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 29, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant