chore(engine): redundancy cleanup across 8 crates

[LKSNDRTMLKV]

Summary

Track A of the redundancy audit (docs/audit/dpp-engine/audit/redundancy-optimization-backlog-2026-07-20.md) — 10 of the 12 "top pick" items, each landed as its own commit gated by the full just check suite (fmt, lint, debug-check, subjects-check, mod-rs-check, test, audit). No behavior changes intended anywhere except the two perf: items, which are pure performance improvements with identical outputs (covered by existing + new tests).

Related issue

Addresses items #2–#11 of the redundancy backlog. Item #12 (unifying the three dpp-types outbox traits) was scoped in detail and intentionally not pursued — filed as #43 for future reference, since the apparent duplication turned out to be mostly load-bearing domain differences rather than incidental copy-paste.

Changes

• perf(resolver) 882be8c — cache the operator DID document (5-minute TTL) instead of an HTTP fetch per passport resolution.
• perf(plugin-host) 42fb39a — build the wasmtime Linker once per plugin instead of per invocation (confirmed Linker::instantiate takes &self, so one linker safely serves unlimited concurrent instantiations).
• fix(dal) 0b434fe — require_updated helper now applied to all outbox repos (registry-sync, webhook, snapshot) so every mark_* call fails closed on an unknown row id instead of silently no-op'ing.
• perf(integrator) 3a8dd89 — bound import concurrency with futures::stream::buffer_unordered instead of one tokio::spawn + semaphore per row.
• perf(node) 6a962e4 — drop unneeded transactions from PgJobStore's single-statement operations.
• perf(vault) 5620012 — avoid an avoidable SectorData clone in apply_compliance; reorder publish() so both view payloads come from one serialize instead of two.
• chore(node) e884721 — extract set_{registry,webhook,snapshot}_gauges helpers, removing 3× duplicated gauge-setting code across boot-time reconciliation and the post-drain re-check.
• refactor(cli) 70a21a0 — describe_error() parses the RFC 7807 problem body instead of dumping the raw JSON blob at ~35 call sites; load_client() collapses the Config::load() + OdalClient::new(...) glue duplicated across ~30 command functions (the interactive console already had this centralized — now both layers share one implementation).
• refactor(render) 523637d + perf(render) 05be1d5 — shared str_field/f64_field/u64_field/bool_field/array_len_field helpers replace the repeated get→as_T→format→fallback idiom across all 10 sector HTML sections; esc() is now single-pass instead of 5 chained .replace() allocations; build_qr_svg's per-module-rect loop uses write! into a pre-sized buffer instead of format! + push_str per dark module.
• refactor(vault) 096f4d9 + 612691a — shared require_admin/require_write guards replace 4 duplicate implementations (plus their duplicate test suites) across 12 handler files; shared not_found_error/conflict_error/validation_error helpers replace ~21 hand-rolled DppError→HTTP match arms across 17 files, with every handler-specific status code and message preserved exactly (including one real outlier: evidence.rs's generate_evidence_handler maps Validation to 409 rather than the usual 422).

Verification

• Every commit gated by the full just check suite before landing.
• cargo nextest run -p dpp-vault --features integration-tests: 198/198 passing (Docker-backed) after the vault changes — confirms no HTTP status code or response body regressions.
• cargo nextest run -p dpp-render, -p dpp-cli: all passing, including new unit tests for the extracted helpers (fields.rs, describe_error, not_found_error/conflict_error/validation_error).

Checklist

• Tests added or updated for new behaviour
• just lint passes locally
• just fmt applied
• just test passes; just test-integration run (persistence/auth touched by the vault items)
• No println!/eprintln!/dbg! introduced
• No secrets, credentials, or .env files in the diff
• Docs updated — N/A, no public API/endpoint/CLI surface changed (internal refactors + perf only)
• DB schema — N/A, no migration changes

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 5 complexity · -64 duplication

Metric	Results
Complexity	5
Duplication	-64

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

_{TIP This summary will be updated as you push new changes.}

[codacy-production]

Pull Request Overview

This pull request executes a significant cleanup of the engine, notably reducing duplication in rendering and repository patterns. Codacy analysis indicates the changes are up to standards. However, two primary issues should be addressed before merging: the loss of deterministic ordering in bulk import reports and a potential cache stampede vulnerability in the DID resolution path. Additionally, while the Wasm plugin Linker reuse and batch concurrency limits were implemented as requested, corresponding automated test scenarios for these features were not found.

About this PR

• The transition to a fail-closed pattern for status updates ensures that operations against missing or already-processed rows are explicitly surfaced as NotFound. Verify that all downstream service handlers are updated to catch this error rather than assuming a silent no-op.

Test suggestions

• Verify describe_error correctly parses RFC 7807 problem bodies and falls back to truncated raw bodies for other response shapes.
• Verify DAL mark_* methods fail with DppError::NotFound when targeting a non-existent row ID.
• Verify operator DID document caching respects the TTL and maintains isolation between different URLs.
• Verify render field helpers handle missing JSON fields and incorrect data types by returning the specified fallback values.
• Verify vault authentication guards correctly enforce Admin and Write scope requirements, returning 403 Forbidden on violations.
• Verify integrator batch processing correctly bounds concurrency using buffer_unordered.
• Verify Wasm plugin Linker is cached at load time and reused across multiple invocations.

Prompt proposal for missing tests

Consider implementing these tests if applicable:
1. Verify integrator batch processing correctly bounds concurrency using buffer_unordered.
2. Verify Wasm plugin Linker is cached at load time and reused across multiple invocations.

_{TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback}

[codacy-production]

_{🟡 MEDIUM RISK}

The new TTL-based cache for DID documents is a critical performance optimization, but the implementation is currently vulnerable to cache stampedes. Under heavy load, multiple concurrent requests for the same URL will trigger simultaneous network fetches before the first result is cached. Consider implementing a synchronization mechanism (e.g., using OnceCell or a library like moka) to ensure only one fetch occurs per unique URL.

[codacy-production]

_{🟡 MEDIUM RISK}

Suggestion: Using buffer_unordered causes the lists of created/updated items and errors to be returned in non-deterministic order. For bulk operations, maintaining the input row order in the report is standard practice to facilitate user debugging. Use buffer_ordered to restore row-order consistency while still benefiting from concurrent stream processing.