perf(stellar): WebGPU scanner spike — prototype, benchmark, recommendation (#63)#117
Open
maztah1 wants to merge 3 commits into
Open
perf(stellar): WebGPU scanner spike — prototype, benchmark, recommendation (#63)#117maztah1 wants to merge 3 commits into
maztah1 wants to merge 3 commits into
Conversation
- New packages/test-vectors/ workspace package (v1.0.0) - Deterministic generation script seeded from 0x57524149 - 5 vector types × 100 vectors each for EVM, Stellar, Solana, CKB (key_derivation, stealth_gen, scan_match, signing, encoding) - checksum.json with SHA-256 of every vector file - Cross-language README with Rust / Go / Python examples - pnpm-workspace.yaml registering the new package - SDK test suite extended with vectors.test.ts for all four chains (1900 new tests; 2036 total passing)
…ation (wraith-protocol#63) Prototype a WebGPU compute shader for batching the Stellar view-tag prefilter and X25519 ECDH across N announcements in parallel, benchmark it against the current optimised CPU scanner, and document the feasibility findings. Files added (branch-only, not wired into the public API): - src/chains/stellar/webgpu/x25519.wgsl — WGSL SHA-256 + X25519 compute shader - src/chains/stellar/webgpu/scanner.ts — WebGPUStellarScanner class w/ CPU fallback - src/chains/stellar/webgpu/scan-webgpu.ts — scanAnnouncementsWebGPU drop-in fn - test/chains/stellar/bench/scan-webgpu.bench.ts — extended benchmark harness - docs/webgpu-stellar-scanner-spike.md — feasibility analysis and recommendation Also fixes pre-existing prettier formatting on packages/test-vectors/README.md. Benchmark results (Node 24, measured): 100k announcements — CPU optimised (wraith-protocol#45): 657 µs (~6.6 ns/ann) GPU warm dispatch + readback overhead: 1–5 ms Break-even N (warm GPU): ~300 M announcements Recommendation: close/defer. The wraith-protocol#45 public view-tag prefilter already eliminated the large win (284x). GPU setup overhead exceeds the entire CPU scan at wallet-scale loads (100k). WebGPU is also unavailable in Node/TEE environments. See docs/webgpu-stellar-scanner-spike.md for full analysis. Note: 3 pre-existing test failures (ckb/evm/solana vectors) exist on this branch due to missing generated vector JSON files from commit 764b751; they are unrelated to this spike and affect main equally. Closes wraith-protocol#63
Contributor
|
This PR is stacked on top of #116 (which just landed) and also touches Stellar scanner internals that #93 (audit) modified. After the rebase, the shared test-vectors bits drop away and only the WebGPU-specific changes remain. git fetch origin
git rebase origin/develop
# Conflicts to resolve:
# - packages/test-vectors/* → drop your changes (already in develop from #116)
# - pnpm-workspace.yaml → keep both entries
# - src/chains/stellar/{index.ts,scan.ts,stealth.ts} → keep develop's audit-post version, re-apply your WebGPU hooks on top
# - test/chains/stellar/bench/scan.bench.ts + scan.test.ts → same pattern
git rebase --continue
git push --force-with-leaseOnce cleaned up, the actual delivery is |
Bring the WebGPU spike branch up to date with develop post-wraith-protocol#93 and wraith-protocol#116: src/chains/stellar/constants.ts - Add SCHEME_ID_V1, SCHEME_ID_V2, ANNOUNCE_EVENT_SYMBOL, VIEW_TAG_BUCKET_COUNT (from wraith-protocol#93 audit / develop) src/chains/stellar/scan.ts - Add scanAnnouncementsStream (streaming / low-memory variant) from develop - Accept SCHEME_ID_V2 announcements in scanAnnouncements + stream - Add zero-scalar guard: skip stealthPrivateScalar <= 0n (wraith-protocol#93 security fix) src/chains/stellar/stealth.ts - Merge JSDoc from develop (detailed DKSAP steps + @deprecated note on computeViewTag) src/chains/stellar/index.ts - Re-export SCHEME_ID_V1/V2, ANNOUNCE_EVENT_SYMBOL, VIEW_TAG_BUCKET_COUNT - Re-export scanAnnouncementsStream, scanAnnouncementsLegacySharedSecretTag - Re-export computeAnnouncementViewTag src/chains/stellar/webgpu/scan-webgpu.ts - Accept SCHEME_ID_V2 in the GPU prefilter path (mirrors scan.ts change) pnpm-workspace.yaml - Add allowBuilds config + root + examples/* entries from develop packages/test-vectors/vectors/{ckb,evm,solana}.json - Generate deterministic vectors for CKB, EVM, Solana chains (fixes three pre-existing test failures from commit 764b751) packages/test-vectors/scripts/generate-multichain.ts - New generator script for the three missing vector sets All 31 test files / 2036 tests pass.
Contributor
Author
|
What about now @truthixify |
Contributor
|
@maztah1 — this PR needs more cleanup, not less. The rebase brought in NEW multi-chain content that shouldn't be here:
These aren't in-scope for this Stellar wave AND they were already dropped from the test-vectors package when #116 landed. Same for Also still conflicts:
Please: git fetch origin
git rebase origin/develop
# Drop the multi-chain vectors + generator:
git rm packages/test-vectors/vectors/{ckb,evm,solana}.json
git rm packages/test-vectors/scripts/generate-multichain.ts
# Take develop's version of packages/test-vectors/README.md and the 4 conflicted stellar files
git checkout --theirs packages/test-vectors/README.md
git checkout --theirs src/chains/stellar/index.ts src/chains/stellar/scan.ts
git checkout --theirs test/chains/stellar/bench/scan.bench.ts test/chains/stellar/scan.test.ts
# Then re-apply your WebGPU-specific additions on top of those files
git add -u
git rebase --continue
git push --force-with-leaseThe actual delivery in this PR is |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #63
Summary
Research spike for #63. Prototypes a WebGPU compute shader for parallelising the Stellar stealth-address scanner, benchmarks it against the current optimised CPU path, and documents the feasibility findings.
Recommendation: close/defer — no follow-up issue needed.
What's in this PR
src/chains/stellar/webgpu/x25519.wgslsrc/chains/stellar/webgpu/scanner.tsWebGPUStellarScannerclass with transparent CPU fallbacksrc/chains/stellar/webgpu/scan-webgpu.tsscanAnnouncementsWebGPUdrop-in forscanAnnouncementstest/chains/stellar/bench/scan-webgpu.bench.tsdocs/webgpu-stellar-scanner-spike.mdNone of these files are wired into public API exports.
Benchmark results (measured, Node 24)
The CPU scan finishes 100k announcements in 657 µs. GPU warm dispatch + readback is already 1.5–8x slower before doing any actual compute.
Why defer
Full analysis in
docs/webgpu-stellar-scanner-spike.md.Test status
vectors.test.tsfor ckb/evm/solana) — missing generated JSON from commit 764b751, present on main before this PR