Skip to content

cascade a result or optional local's payload through .ok and .err reads#540

Merged
kacy merged 1 commit into
mainfrom
result-ok-read-cascade
Jul 23, 2026
Merged

cascade a result or optional local's payload through .ok and .err reads#540
kacy merged 1 commit into
mainfrom
result-ok-read-cascade

Conversation

@kacy

@kacy kacy commented Jul 23, 2026

Copy link
Copy Markdown
Owner

summary

the payload reclamation added in #537 only covered locals whose every use was a flag read (.is_ok, .is_err, == none). .ok/.err reads — by far the common shape — were held out, because whitelisting them made an http/2 valgrind case read freed memory. that turned out to be a separate ownership bug the leak had been covering up: a channel try_send handed its value to another task without taking a count, fixed in #538. with that gone, the wider whitelist is clean.

a .ok/.err read loads the payload slot as a borrow. where the value escapes (b := r.ok, return r.ok) the emitter retains right after the field load, so the consumer holds its own count; where it doesn't (f(r.ok), r.ok.len(), l.push(r.ok)) nothing is retained and nothing is released. either way the tuple keeps the count it was built with, so the cleanup path is still the payload's only releaser. i checked that per context with PITH_DUMP_IR rather than assuming it.

★ one shape did corrupt, and it is now guarded

fn build(i: Int) -> fn() -> Int:
    r := mk(i)
    return fn() => r.ok.len()

fails valgrind with an invalid read. the IR says why: capturing the local takes a count on the three-slot shell and never on the payload behind it, so a closure outliving the frame reads memory the cascade already dropped. any local mentioned inside a lambda is therefore ruled out entirely.

a second guard rules out a local whose name appears in an assignment target — on the left of =, .ok is a write, not a read, and would replace the payload the cascade expects to own.

both guards fail toward leaking, never toward corruption, which is the standing rule for this work.

everything else stays excluded exactly as before: params (a borrow the caller owns), !, ?, catch, unwrap_or, or_else, .value(), if-let/while-let/match binds, for-iter, call arguments, returns, and conflicting rebinds.

★ payoff

bench/http_server_mt under wrk -t2 -c8, both binaries from identical source, only the compiler differing:

requests rss growth per request
before 226,152 625,464 kb 2.77 kb
after 260,375 58,024 kb 0.223 kb
after, 30s 513,694 114,400 kb 0.223 kb

12.5x less per request. i reran it independently: 52,480 kb over 231,158 requests = 0.227 kb/req.

not zero, and the docs say so. the per-request figure is identical over a 30s run, so what remains grows linearly and is not warm-up — there is another leak in that path, with some other cause.

minimal repro, 1m iterations, ~260-byte payload:

pre-#537 main here
optional probed with == none 277 mb 2.5 mb 2.5 mb
result probed with .is_err 277 mb 2.5 mb 2.5 mb
result read through .ok 277 mb 277 mb 2.5 mb

what was tested

blast radius here is every result local in the codebase, so the bar was high:

  • make memcheck — all 47 cases clean, including test_http2_tls_roundtrip (run 3x standalone under valgrind, 0 errors each — the exact case that failed 3/3 before), test_channel_try_send_ownership, test_catch_heap_no_leak, test_optional_temp_release, test_tuple_ownership.
  • valgrind per extraction pattern, 11 probes: !; probe+.ok over String/Bytes/List/Map/struct; .err on a failure; catch/unwrap_or/or_else; if let/while let/.value()/?; index-propagate and for-iter; passed/returned/discarded/rebound-in-a-loop; .ok escaping into a list, map, struct field, argument, receiver, concat, return; .ok across send and try_send to a spawned task; a closure capturing the local (the one that failed, now guarded); .ok as a spawn argument.
  • make bootstrap-verify — ir fixed point verified.
  • run-regressions 237/237, run-examples 95/95, green-tests pass, fmt --check clean.
  • new tests/cases/test_result_ok_reclaim wired into MEMCHECK_CASES, covering probe-then-.ok over four payload kinds, .err, .ok escaping into containers and returns, a closure capture, a result parameter, and the refused extraction forms — so the lambda guard is pinned by valgrind rather than by a comment.

one flake seen: test_atomic_context failed twice inside the self-hosted loop on a loaded box, then passed 15/15 standalone and 237/237 on a quiet rerun. it's a known timing-sensitive concurrency test, unrelated.

docs

docs/performance.md: the reclamation entry now describes the whitelist as flag reads and payload reads, with the table remeasured end-to-end (labelled ~260-byte payload, since the old 295 mb column used a different payload); the "deliberately left alone" paragraph replaced with the try_send root cause; a new paragraph naming what still leaks, including closure capture and the param borrow. the bench/http_server entry drops to +0.22 kb/req with the honest caveat that rss still climbs. docs/ownership.md's known-gaps bullet gets the same correction. throughput numbers untouched.

https://claude.ai/code/session_01Uzg5VKBucCeHxan34VAsjv

… reads

a `T!`/`T?` local releases its payload at cleanup only when every use of it
is on a whitelist. that whitelist was flag reads — `.is_ok`, `.is_err`,
`== none` — because widening it to `.ok` made test_http2_tls_roundtrip fail
valgrind. that turned out to be a channel `try_send` handing its value to
another task without taking a count, which the leak had been covering up.
with that fixed the reads can come in.

a `.ok`/`.err` read is a borrow of slot 8 / slot 16. where the value escapes
the read (`b := r.ok`, `return r.ok`) the emitter retains right after the
field load, so the consumer holds its own count; where it does not
(`f(r.ok)`, `r.ok.len()`) nothing is retained and nothing is released. the
tuple keeps the count it was built with either way, so the cleanup is still
the only thing that drops the payload.

two things stay out, both found by valgrinding the widened whitelist:

  - a lambda that mentions the local. capture retains the three-slot shell
    and not the payload, so a closure outliving the frame reads memory the
    cascade already freed. a probe-only local inside a closure was fine
    before and is ruled out now too; that is a leak, not a dangle.
  - the local on the left of an assignment. `.ok` there is a write, not a
    read.

everything else is unchanged: params never cascade, and `!`, `catch`,
`unwrap_or`, `if let`, a bind, a call argument, a return and a match all
keep the shell-only release.

what it buys, on this 2-core machine:

  - bench/http_server_mt under `wrk -t2 -c8`: 2.8 kb/request of growth down
    to 0.22 kb. serve_connection reads each request into a `T!` and reads it
    back with `.ok`, so every request was leaking its whole
    HttpRequestBytes.
  - a million-iteration loop of `r := make(i); if r.is_err: break;
    b := r.ok` with a 260-byte payload: 277 mb peak rss down to 2.5 mb.

tested: bootstrap-verify to a fixed point, memcheck (all 47 cases, including
http2_tls_roundtrip three times over), regressions native and self-hosted,
examples, green-tests. plus valgrind on a dozen small programs covering each
extraction form on its own — `!`, probe-then-`.ok`, `.err`, catch, unwrap_or,
or_else, if-let, while-let, `?`, index propagation, a discarded call, passing
and returning a result local, a rebind in a loop, `.ok` into a list/map/struct
field/argument/return, `.ok` across a channel send and try_send, `.ok` as a
spawn argument, and a closure capturing the local.

Claude-Session: https://claude.ai/code/session_01Uzg5VKBucCeHxan34VAsjv
@kacy
kacy merged commit 5c48f2c into main Jul 23, 2026
2 checks passed
@kacy
kacy deleted the result-ok-read-cascade branch July 23, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant