Skip to content

add mobench support to ProveKit main#450

Open
dcbuild3r wants to merge 4 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures
Open

add mobench support to ProveKit main#450
dcbuild3r wants to merge 4 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures

Conversation

@dcbuild3r

@dcbuild3r dcbuild3r commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Reopened from #429 using the new dcbuilder/* branch naming.

Old PR: #429
Replacement branch: dcbuilder/pr429-main-mobench-fixtures
Old head branch: dcbuild3r/main-mobench-fixtures
Head SHA: e14fa60beadc25bf1e7d238c9bdbf8f6237fa9f2

Previous PR CI / benchmark runs from #429:

Summary

  • add main-based mobench fixtures for passport, OPRF, and P-256 bigcurve
  • run BrowserStack triad profile across Vivo Y21, Pixel 7, S24 and iOS fallback triad
  • fix Android prove timing so fixture clone/setup is outside the measured prove span
  • use jemalloc as the Android fallback allocator under the native C FFI allocator
  • harden Android incomplete-fixture reporting with timeout/build/kill-evidence fields

Android mobench fix note

The previous failing run was https://github.com/worldfnd/provekit/actions/runs/26007468825. The missing Vivo Y21 monolithic cell did not recover a BrowserStack session payload or summary.json; available artifacts only show the BrowserStack fetch timeout after 7200s for build c943753d95fd0b34f5775aa0a3bc6ff58cbcc3ca.

Before/failure memory from that run:

  • Vivo Y21 OPRF: 328 MB process peak, 10.061s
  • Vivo Y21 fragmented age check: 1187 MB process peak, 65.342s
  • Vivo Y21 monolithic age check: no row; BrowserStack timeout/no summary
  • S24 OPRF: 426 MB process peak, 2.809s

What changed:

  • Android and iOS now use the same semantic boundary for prove benches: per-iteration fixture setup happens before the measured closure, and profile_phase("prove") wraps only the prover entry point. Previously prepared.clone() ran inside profile_phase("prove"), so clone cost and peak memory were charged to proving.
  • Android native C FFI fallback allocation now routes through jemalloc instead of Bionic malloc when no host callback/mmap allocator is active.
  • If BrowserStack returns no summary, failure.json now records attempts, fetch timeout seconds, build id, and any LMK/OOM/SIGKILL lines recovered from attempt/device logs.

Latest benchmark numbers were reposted as a PR comment from #429.

Validation

  • cargo fmt --all
  • cargo test -p bench-mobile --lib
  • cargo test -p bench-mobile --test examples_smoke
  • cargo test -p bench-mobile --test passport_smoke
  • cargo check -p provekit-ffi --target aarch64-linux-android with NDK 26.1 aarch64-linux-android34-clang
  • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/mobile-bench-reusable.yml")'
  • git diff --check

@dcbuild3r ChatGPT Codex Connector

dcbuild3r commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author

Latest mobile benchmark numbers

Reposted from #429 comment #429 (comment).

Source: Mobile Bench PR Auto run 26368378239 on dcbuild3r/main-mobench-fixtures at e14fa60b.
Mobench version: 0.1.42. Artifact summaries report 2 measured runs per function/device plus 1 warmup run.

iOS

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF iPhone 16 Pro Max 2 1 766.622ms 766.622ms 775.181ms 758.064ms 775.181ms 1.533s 3.364s 6.728s 438.8% 10.95 MB 10.95 MB 362.28 MB
OPRF iPhone 14 2 1 1.096s 1.096s 1.118s 1.075s 1.118s 2.192s 4.497s 8.994s 410.2% 32.55 MB 32.55 MB 206.95 MB
OPRF iPhone SE 2022 2 1 1.085s 1.085s 1.086s 1.085s 1.086s 2.171s 4.486s 8.972s 413.3% 33.48 MB 33.48 MB 209.44 MB
Passport fragmented iPhone 16 Pro Max 2 1 4.751s 4.751s 4.805s 4.696s 4.805s 9.501s 21.524s 43.049s 453.1% 150.95 MB 150.95 MB 1042.44 MB
Passport fragmented iPhone 14 2 1 6.598s 6.598s 6.625s 6.570s 6.625s 13.195s 28.438s 56.877s 431.0% 264.30 MB 264.30 MB 845.34 MB
Passport fragmented iPhone SE 2022 2 1 7.053s 7.053s 7.233s 6.874s 7.233s 14.106s 30.661s 61.323s 434.7% 265.72 MB 265.72 MB 851.73 MB
Passport complete iPhone 16 Pro Max 2 1 4.499s 4.499s 4.514s 4.484s 4.514s 8.997s 19.933s 39.866s 443.1% 376.08 MB 376.08 MB 1594.42 MB
Passport complete iPhone 14 2 1 7.772s 7.772s 8.889s 6.654s 8.889s 15.543s 28.034s 56.069s 360.7% 487.56 MB 487.56 MB 1220.12 MB
Passport complete iPhone SE 2022 2 1 7.443s 7.443s 7.505s 7.381s 7.505s 14.886s 30.253s 60.507s 406.5% 494.86 MB 494.86 MB 1104.30 MB

Android

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF Samsung Galaxy S24 2 1 2.725s 2.725s 2.762s 2.688s 2.762s 5.449s 9.657s 19.315s 354.4% 89.95 MB 89.95 MB 488.93 MB
OPRF Google Pixel 7 2 1 3.900s 3.900s 3.943s 3.858s 3.943s 7.800s 11.518s 23.036s 295.3% 18.23 MB 18.23 MB 237.57 MB
OPRF Samsung Galaxy M32 2 1 6.681s 6.681s 6.926s 6.436s 6.926s 13.362s 22.641s 45.282s 338.9% n/a n/a 352.76 MB
Passport fragmented Samsung Galaxy S24 2 1 14.516s 14.516s 14.697s 14.336s 14.697s 29.033s 54.556s 109.112s 375.8% 202.75 MB 202.75 MB 929.28 MB
Passport fragmented Google Pixel 7 2 1 26.237s 26.237s 28.870s 23.605s 28.870s 52.475s 74.610s 149.220s 284.4% 217.21 MB 217.21 MB 789.64 MB
Passport fragmented Samsung Galaxy M32 2 1 40.629s 40.629s 41.183s 40.075s 41.183s 81.257s 137.225s 274.451s 337.8% 284.89 MB 284.89 MB 859.60 MB
Passport complete Samsung Galaxy S24 2 1 11.250s 11.250s 11.671s 10.829s 11.671s 22.500s 46.655s 93.310s 414.7% 376.77 MB 376.77 MB 1157.10 MB
Passport complete Google Pixel 7 2 1 19.856s 19.856s 22.382s 17.330s 22.382s 39.712s 63.704s 127.408s 320.8% 304.69 MB 304.69 MB 980.12 MB
Passport complete Samsung Galaxy M32 2 1 28.921s 28.921s 30.002s 27.840s 30.002s 57.842s 102.707s 205.414s 355.1% 365.05 MB 365.05 MB 1220.69 MB

Memory growth is the benchmark-reported peak growth above baseline; process peak is the reported process RSS peak.

@dcbuild3r dcbuild3r mentioned this pull request Jun 3, 2026
Closed
@github-actions

github-actions Bot commented Jun 3, 2026 •
edited
Loading

Copy link
Copy Markdown

CSP benchmarks

Metric Value
Workflow status [PASS] success
Commit 95ec58517036
Run #27376286539
Circuits benchmarked 21
Iterations averaged per circuit 3

Prover time, peak RSS, peak heap, and verifier time are arithmetic means across the iterations. Peak heap comes from the largest peak memory entry in provekit-cli prove's tracing output; peak RSS is reported by /usr/bin/time -v (max-resident-set-size).

No baseline available yet — deltas will appear once this workflow has produced at least one successful main run.

Results
Circuit Constraints Witnesses Prover time Peak RSS Peak heap Verifier time Proof size PKP size
ecdsa_p256 143,282 258,158 2.98 s 258 MB 225 MB 340 ms 2.79 MB 810 KB
keccak_1024 822,870 1,543,366 6.32 s 986 MB 954 MB 843 ms 3.13 MB 6.07 MB
keccak_128 163,058 313,707 2.11 s 274 MB 242 MB 360 ms 2.80 MB 1.22 MB
keccak_2048 1,575,606 2,945,822 11.76 s 1.81 GB 1.80 GB 1.41 s 3.27 MB 12.36 MB
keccak_256 256,206 487,012 2.29 s 330 MB 291 MB 400 ms 2.85 MB 1.97 MB
keccak_512 445,094 839,130 3.62 s 594 MB 509 MB 547 ms 3.02 MB 3.40 MB
poseidon2_12 479 563 353 ms 23.85 MB 14.69 MB 100 ms 1.01 MB 436 KB
poseidon2_16 556 719 353 ms 24.22 MB 14.88 MB 103 ms 1.04 MB 530 KB
poseidon2_2 231 278 347 ms 23.04 MB 14.11 MB 100 ms 1.03 MB 108 KB
poseidon2_4 529 535 340 ms 23.32 MB 14.31 MB 100 ms 1.03 MB 31.67 KB
poseidon2_8 363 423 353 ms 24.08 MB 14.50 MB 100 ms 1.04 MB 365 KB
poseidon_12 504 524 353 ms 24.23 MB 14.69 MB 100 ms 1.04 MB 410 KB
poseidon_16 609 633 350 ms 24.26 MB 14.97 MB 100 ms 1.02 MB 536 KB
poseidon_2 240 249 340 ms 22.94 MB 14.02 MB 100 ms 1.02 MB 53.79 KB
poseidon_4 297 309 343 ms 23.33 MB 14.31 MB 100 ms 1.04 MB 210 KB
poseidon_8 402 418 350 ms 23.46 MB 14.50 MB 100 ms 1.03 MB 305 KB
sha256_1024 196,940 339,764 2.19 s 308 MB 273 MB 410 ms 2.83 MB 1.87 MB
sha256_128 46,398 80,974 1.08 s 101 MB 83.45 MB 260 ms 2.52 MB 510 KB
sha256_2048 345,399 612,724 3.57 s 548 MB 483 MB 587 ms 2.99 MB 3.02 MB
sha256_256 67,904 117,944 1.38 s 150 MB 130 MB 290 ms 2.64 MB 708 KB
sha256_512 110,916 191,884 1.52 s 181 MB 158 MB 320 ms 2.66 MB 1.12 MB

@dcbuild3r dcbuild3r added the bench Run mobile benchmarks on PRs label Jun 11, 2026
@dcbuild3r dcbuild3r deployed to Browserstack June 11, 2026 20:45 — with GitHub Actions Active
@github-actions

github-actions Bot commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Mobench Benchmark Results

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 1090.7 1067.7 1113.6 1090.7 1113.6 9.052s 33.11 MB 203.19 MB

2 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 871.4 862.3 880.5 871.4 880.5 6.725s 4.17 MB 357.72 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 1055.2 1051.3 1059.2 1055.2 1059.2 8.942s 32.84 MB 209.00 MB

2 iterations · 1 warmup · avg is primary metric

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 6530.1 6483.8 6576.4 6530.1 6576.4 54.939s 492.14 MB 1262.12 MB

2 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 4796.7 4778.0 4815.4 4796.7 4815.4 41.829s 178.97 MB 1429.42 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 7658.9 7648.3 7669.5 7658.9 7669.5 62.400s 490.27 MB 1163.73 MB

2 iterations · 1 warmup · avg is primary metric

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 6748.2 6699.1 6797.3 6748.2 6797.3 58.004s 265.25 MB 828.11 MB

2 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 4910.6 4866.6 4954.5 4910.6 4954.5 44.038s 188.11 MB 1047.95 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 6622.7 6593.1 6652.3 6622.7 6652.3 57.734s 264.36 MB 842.19 MB

2 iterations · 1 warmup · avg is primary metric

Ios Sina Plot

ios Sina plot

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 3822.4 3777.8 3867.0 3822.4 3867.0 21.422s 19.38 MB 239.08 MB

2 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 6951.1 6887.9 7014.3 6951.1 7014.3 45.176s — 330.16 MB

2 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy S24 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 2687.5 2623.8 2751.1 2687.5 2751.1 19.540s 102.14 MB 500.78 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 21222.1 20701.6 21742.5 21222.1 21742.5 123.123s 325.41 MB 981.42 MB

1 iterations · 0 warmup · avg is primary metric

ANDROID — Samsung Galaxy S24 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 10188.3 10000.8 10375.8 10188.3 10375.8 85.681s 221.45 MB 1143.84 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 24237.6 24233.9 24241.2 24237.6 24241.2 140.252s 219.62 MB 788.97 MB

2 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 38265.6 38203.9 38327.3 38265.6 38327.3 274.817s 272.46 MB 866.23 MB

1 iterations · 0 warmup · avg is primary metric

ANDROID — Samsung Galaxy S24 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 15102.7 15055.9 15149.4 15102.7 15149.4 113.217s 186.36 MB 919.43 MB

2 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 31177.1 31177.1 31177.1 31177.1 31177.1 110.126s 324.41 MB 1072.72 MB

1 iterations · 0 warmup · avg is primary metric

Android Sina Plot

android Sina plot

Posted by mobench at 2026-06-12 01:09 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bench Run mobile benchmarks on PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dcbuild3r