Skip to content

fix(audit): recursive redaction for nested maps and inlined secrets (PILOT-304)#27

Closed
matthew-pilot wants to merge 1 commit into
mainfrom
openclaw/pilot-304-20260530-191046
Closed

fix(audit): recursive redaction for nested maps and inlined secrets (PILOT-304)#27
matthew-pilot wants to merge 1 commit into
mainfrom
openclaw/pilot-304-20260530-191046

Conversation

@matthew-pilot
Copy link
Copy Markdown
Collaborator

What

BuildEntry previously only redacted top-level attribute keys via redactKey(). If an attribute value was:

  • A map[string]interface{} with nested secret keys (e.g. config={token=sk-abc})
  • A string containing embedded secrets (e.g. stringified JSON, key=value pairs)

…the secrets landed verbatim in the audit log.

Fix

  • formatVal — dispatches to recursive map-walker or string-scanner based on value type
  • formatMap — recursively walks maps, redacting secret keys at any depth
  • scanSecrets — scans string values for known secret patterns (token=, "token":") and replaces values with <redacted>

Scope

 audit/audit.go             | 113 ++++++++++++++++++++++++++++++++++++++---
 audit/zz_redaction_test.go |  53 ++++++++++++++++++++
 2 files changed, 158 insertions(+), 8 deletions(-)

Verification

  • go build ./...
  • go vet ./...
  • Full test suite (18 packages): all pass ✓
  • New tests cover: nested map redaction, deeply nested map redaction, JSON string scanning, key=value string scanning, clean-string no-op

Ticket

Closes PILOT-304: audit redaction — only top-level keys checked, nested values bypass

@matthew-pilot matthew-pilot added the matthew-fix-larger Medium-scope autonomous fix (≤10 files, ≤200 LoC) label May 30, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@matthew-pilot
Copy link
Copy Markdown
Collaborator Author

📊 PR Status — #27 PILOT-304

Field Value
State OPEN
Mergeable ✅ MERGEABLE (CLEAN)
Draft No
Branch openclaw/pilot-304-20260530-191046main
Files 2 files, +158/−8 (audit/audit.go, audit/zz_redaction_test.go)
Labels matthew-fix-larger

CI Checks (2/2 passing)

Check Result
test ✅ pass
codecov/patch ✅ pass

Canary

🧪 Running — dispatched run 26692500596, pending completion.

Jira

PILOT-304 — QA/IN-REVIEW (unassigned). Updated: 2026-05-30T22:11 EEST.

Last operator activity

PR created by matthew-pilot at 19:11 UTC. No operator activity yet.

@matthew-pilot
Copy link
Copy Markdown
Collaborator Author

🔍 PR Explanation — #27 PILOT-304

What this does

Extends audit entry redaction from top-level keys only to recursive deep inspection: nested maps, deeply-nested maps, and inline secrets embedded in string values (e.g. JSON blobs or key=value pairs).

The problem

The existing redactKey() check only looked at the top-level attrs slice passed to BuildEntry. If an operator logged a config map like {"host":"x","token":"sk-secret"}, the token appeared unredacted in the audit log. Similarly, a raw JSON string like {"api_key":"sk-emb"} in an env value would leak its secret because the scanner didnt look inside string values.

Walkthrough: 2 files (+158/−8)

audit/audit.go (+105/−8)

  1. BuildEntry refactored (line 277): switches from string concatenation to strings.Builder. Calls new formatVal(k, v) instead of inline redaction — delegates all formatting and redaction to the new recursive pipeline.

  2. formatVal(k, v) — top-level dispatcher:

    • If key matches redactKey()<redacted> (unchanged behavior).
    • If value is map[string]interface{} → delegates to formatMap (recursive).
    • If value is string → runs through scanSecrets (new: inline secret scanning).
    • Otherwise → fmt.Sprintf("%v") as before.
  3. formatMap(prefix, m) — recursive map walker:

    • Formats as prefix={k1=v1, k2=v2, ...}.
    • For each key: redacts if secret, recurses into nested maps, scans string values.
  4. scanSecrets(s) — string value scanner:

    • Scans for secret-key prefixes embedded in string values (JSON "token":"...", key=value pairs, etc.).
    • Replaces the value portion with <redacted>.
    • Uses substring matching (no regex dependency).
  5. findValueEnd(s) — value boundary detector:

    • Stops at first whitespace, comma, quote, or newline after a secret key prefix.
  6. secretKeyPrefixes — pattern list:

    • admin_token, private_key, api_key, signature, token, password, passwd, secret, bearer, credential
    • Both JSON ("key":") and plain (key=) formats. Longer prefixes first to avoid partial matches.

audit/zz_redaction_test.go (+53/−0) — 5 new test cases:

  • Nested map with token → redacted
  • Deeply nested map (2 levels) → redacted
  • Stringified JSON with embedded token → scanned and redacted
  • key=value string with api_key=sk-kv → scanned and redacted
  • Clean string (host=ex.com) → untouched

Why this approach

  • Zero new dependencies — substring matching, no regex. Keeps the audit package dependency surface small.
  • Recursive by construction — any nesting depth works because formatMap calls formatVal which can call formatMap.
  • Backward-compatible — top-level redactKey() behavior is unchanged. The new code adds deeper scanning alongside the existing check.

@matthew-pilot
Copy link
Copy Markdown
Collaborator Author

📊 PR Status — #27 PILOT-304

Field Value
PR OPEN · MERGEABLE ✅
CI 2/2 passing (test ✅, codecov/patch ✅)
Labels matthew-fix-larger, canary-failed 🔴
Reviews 0 reviews
Created 2026-05-30 (4 days old)

⚠️ Canary Failed

The canary harness reported a failure for this PR. Do not merge until resolved. See the canary run for details.

🤖 Auto-check

Self-created PR by matthew-pilot. Awaiting operator review. No operator activity detected.


matthew-pr-worker tick 2026-06-03T20:53 UTC

…PILOT-304)

BuildEntry previously only redacted top-level attribute keys. Nested
maps (e.g. config={token=sk-abc}) and string values containing
embedded secrets (e.g. stringified JSON, key=value pairs) bypassed
redaction entirely.

Adds formatVal/formatMap to recursively walk map values, plus
scanSecrets to scan string values for known secret patterns
(token=, "token":") and replace them with <redacted>.

Scope: audit/audit.go (+113/-8), audit/zz_redaction_test.go (+53)
@TeoSlayer TeoSlayer force-pushed the openclaw/pilot-304-20260530-191046 branch from 9c98d5d to c245653 Compare June 6, 2026 08:32
TeoSlayer added a commit that referenced this pull request Jun 6, 2026
…PILOT-304)

Resupersedes #27 with a clean rebase. Replaces the BuildEntry details
loop with formatVal/formatMap/scanSecrets helpers that recurse into map
values and substring-scan string values for embedded secret patterns
(token=, api_key=, etc., plus JSON-encoded equivalents).
@TeoSlayer
Copy link
Copy Markdown
Contributor

Superseded by clean-rebase replacement.

@TeoSlayer TeoSlayer closed this Jun 6, 2026
@TeoSlayer TeoSlayer deleted the openclaw/pilot-304-20260530-191046 branch June 6, 2026 09:26
TeoSlayer added a commit that referenced this pull request Jun 6, 2026
…PILOT-304) (#40)

* fix(audit): recursive redaction for nested maps and inlined secrets (PILOT-304)

Resupersedes #27 with a clean rebase. Replaces the BuildEntry details
loop with formatVal/formatMap/scanSecrets helpers that recurse into map
values and substring-scan string values for embedded secret patterns
(token=, api_key=, etc., plus JSON-encoded equivalents).

* test(audit): cover recursive redaction paths (PILOT-304)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

canary-failed matthew-fix-larger Medium-scope autonomous fix (≤10 files, ≤200 LoC)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants