fix(compile): stop leaking the object staging dir on --no-link, deliver its objects to -o (#7167)

benchmarks/repsel_census/README.md

@@ -74,20 +74,31 @@ of them is recorded at the site where the proof is dropped:
 reproducible without the report at all: compile the workload twice, once with
 `PERRY_PTR_SHAPE_LOCALS=0`, and compare the objects.
 
-`--no-link` does **not** honour `-o`: the object goes to a per-run temp
-directory and the path is printed. So capture the printed path in each arm and
-compare those — comparing the `-o` arguments compares two files that were never
-created.
+`--no-link` writes its objects to `-o` (#7167): verbatim for a single-module
+program, otherwise into `-o`'s directory under module-derived names. **Give each
+arm its own `-o`.** Two arms pointed at one path is not a comparison — the
+second compile overwrites the first and `cmp` then compares a file with itself,
+which is "identical" for every arm forever.
+
+Read the paths back off stdout rather than assuming `-o` named them, because a
+multi-module workload emits several and only one of them can be `-o`. An arm
+that reported no object is a harness error, not a silent pass — the same reason
+`_written_objects` in the census script raises.
 
 ```bash
-obj() {  # echo the object path this compile actually wrote
+objs() {  # echo every object path this compile actually wrote
   "$@" --no-link --no-cache 2>&1 | sed -n 's/^Wrote object file: //p'
 }
-a=$(obj perry compile <src> -o /tmp/ignored)
-b=$(PERRY_PTR_SHAPE_LOCALS=0 obj perry compile <src> -o /tmp/ignored)
+a=$(objs perry compile <src> -o "$PWD/ab/a.o")
+b=$(PERRY_PTR_SHAPE_LOCALS=0 objs perry compile <src> -o "$PWD/ab/b.o")
 cmp "$a" "$b" && echo "IDENTICAL — the promotion emitted nothing"
 ```
 
+Before #7167 the flag ignored `-o` entirely and left the objects in a
+`perry-objs-<pid>-<nanos>/` directory under `TMPDIR` that nothing ever deleted,
+which is why the older version of this recipe passed `-o /tmp/ignored` twice and
+still worked. It does not any more, and the version above is the one to copy.
+
 Byte-identical objects mean the promotions the report counted as wins changed
 nothing. `07_object_create` and `12_binary_trees` are byte-identical today.
 `09_method_calls` differs, but only by two `__pshape` clones with **zero call
@@ -319,10 +330,16 @@ changing it:
 * **The `TMPDIR` isolation is load-bearing**, not politeness. Counting entries
   in the shared system temp dir measures every other process on the box.
 
-It fails on the clang driver's own temp names (`perry_llvm_*`, `perry_cgu_*`,
-`perry_bc_*`) and merely *reports* anything else — today that is the compile
-driver's `perry-objs-<pid>-<nanos>/` staging directory, which `--no-link` never
-cleans up (#7167). Widen `OWNED_PREFIXES` to "everything" once that is closed.
+It fails on **anything** left behind, with no allowlist. It shipped with one —
+the clang driver's own names (`perry_llvm_*`, `perry_cgu_*`, `perry_bc_*`)
+failed and everything else was merely reported — because the compile driver was
+leaking a `perry-objs-<pid>-<nanos>/` staging directory on the `--no-link` path
+at the time (#7167), and a gate that goes red for another module's defect gets
+muted rather than fixed. #7167 closed that path and the carve-out went with it.
+
+The absence of an allowlist is the point. #7167 was *known*: this gate printed
+it on every run and could not turn one red. A gate that enumerates the leaks it
+is allowed to fail on cannot see the one nobody has written yet.
 
 No exemption for `PERRY_DEBUG_SYMBOLS`, and that is a change of belief rather
 than a change of policy. `-g` was documented as pulling the `.ll`'s **absolute**

changelog.d/7175-no-link-object-staging-dir.md

@@ -0,0 +1,111 @@
+Closes #7167: `perry compile --no-link` no longer leaks its object staging
+directory into the system temp directory, and now writes its objects where
+`-o` points.
+
+`run_pipeline.rs` created a per-invocation `perry-objs-<pid>-<nanos>/`
+directory for every compile and removed it on the paths that *link*.
+`--no-link` returns before those, so it removed nothing. Because the name
+carries the pid **and** a wall-clock nanosecond component, no two invocations
+ever reuse one: the leak is unbounded in **compiles**, not in distinct IR the
+way #7144's `.ll` leak was, and the staged objects are far larger than the
+`.ll`s. The machine this was written on had accumulated 3086 such directories
+(277 MB). Every `--no-link` user was affected — the flag itself, the
+separate-link workflow, and every harness in
+`scripts/compiler_output_harness/` (the census, knob-isolation and determinism
+gates all compile with `--no-link`), which is why running the representation
+census bled gigabytes a day.
+
+**The objects could not simply be deleted.** On `--no-link` they are the
+product: the flag is documented as "produce object file only", and the census
+and knob-isolation gates hash the paths it prints (`_written_objects` raises
+outright if a reported object does not exist on disk). What was wrong was
+*where* they went, not that they survived. The flag also did not honour `-o`
+at all — the census README carried a warning about it and a hand-written A/B
+recipe built around the wart.
+
+Two structural changes rather than a third `remove_dir`:
+
+* **`--no-link` no longer creates a staging directory**, so it cannot leak
+  one. Its objects are delivered to `-o`: verbatim when the program has one
+  native module (`cc -c foo.c -o foo.o`), otherwise into `-o`'s directory
+  under the module-derived names, because one `-o` cannot name N files. With
+  no `-o` they land in the current directory. The rule keys on the module
+  count — a property of the program — rather than on how many objects codegen
+  actually wrote, so `-o` does not mean two different things depending on
+  whether the object cache was warm. Bitcode-link mode emits `.ll`, never
+  takes `-o` verbatim.
+
+  Both object-cache paths had to be closed for that last property to be true.
+  A cold *store* and a warm *hit* each handed back the cache entry's path
+  (`Stored cached object:` / `Reused cached object:`) instead of the object the
+  user asked for, so `-o` went unwritten with the cache on. Harmless when
+  linking — the linker is the only reader — but not for `--no-link`. A hit now
+  copies the cached object out to the destination (copy, not hand back the
+  path: the cache entry is shared with every other build and must not become an
+  output the user may overwrite); a store keeps storing, so later builds still
+  hit, but falls through to write the object it just produced. Both report
+  `Wrote object file`, which also means the census harness's
+  `_written_objects` — which scrapes exactly those lines — sees a warm-cache
+  compile at all instead of finding no objects. Verified: cold and warm both
+  write `-o`, byte-identical.
+* **When linking, the staging directory is removed by `Drop`** (new
+  `crates/perry/src/commands/compile/object_staging.rs`), so both link exits,
+  the static-archive exit and every `?` in between clean up through one site.
+  Three call sites that must each remember is how the third came to be
+  missing. The default direction now matters: a future exit that does nothing
+  cleans up, where before it leaked.
+
+Removing the directory is unobservable to a concurrent compile, and that is a
+property of the name rather than a timing argument: pid + monotonic nanos means
+it belongs to exactly one invocation. This is the same conclusion #7144 reached
+for the `.ll` by a different route — there the fix had to *create* per-call
+ownership, because #7131 had made the `.ll` basename a pure function of the IR
+and two workers holding identical IR shared it. Nothing is shared here.
+
+Two further leaks the same guard closed, both broader than #7167 described:
+
+* The **executable** link — the default path — never removed the directory at
+  all. `cleanup_intermediates` only unlinks files, so every successful
+  `perry compile` left an empty `perry-objs-*` directory behind. Empty, but one
+  per compile.
+* The static-archive and shared-library exits used `remove_dir`, which is
+  non-recursive and silently no-ops when anything in the directory was not on
+  the cleanup list. `Drop` uses `remove_dir_all`.
+
+`--keep-intermediates` is still the single opt-in for retaining staged objects,
+disarmed once where the directory is created rather than re-checked at each
+exit. The codegen-failure paths now name the directory and say whether it
+survives; on `--no-link` a failed compile keeps every object it managed to
+write, at the path the user named.
+
+**Gate.** `census-temp-hygiene` (#7144) shipped with a carve-out: it failed on
+the clang driver's own temp names and merely *reported* anything else, because
+this leak was live at the time and a gate that goes red for another module's
+defect gets muted rather than fixed. The carve-out is gone — the gate now
+asserts the absolute property, that an isolated `TMPDIR` is empty after the
+corpus compiles, with **no allowlist**. #7167 is the argument for that: it was
+known, printed on every run, and could not turn a run red for a full release.
+The self-test asserts the flip directly (`perry-objs-*` inputs that returned 0
+now return 1) and asserts that a name from neither family fails too.
+
+**Evidence.** Two arms built sequentially from one target dir, distinct binary
+hashes, isolated `TMPDIR`, 27 census workloads × 2 compiles = 54:
+
+| arm | `perry-objs-*` entries left | harness exit |
+|---|---|---|
+| `main` | **108** (54 directories + 54 objects) | 1 |
+| this branch | **0** | 0 |
+
+The absolute property, not "no growth" — #7144's lesson, and here growth would
+have caught it, but on the next content-addressed leak it would not.
+
+No behavioural change: across all 27 census workloads the emitted objects are
+**byte-identical** to `main`'s, and a linked two-module executable is
+byte-identical and runs identically. That is structural rather than lucky —
+`compile_ll_to_object` returns the object *bytes* and `run_pipeline` writes
+them, so the staging path was never in the object. Verified directly: no
+`perry-objs`/`perry_llvm` string and no `__debug_*` section appears in a Perry
+Mach-O object, with or without `PERRY_DEBUG_SYMBOLS=1`.
+
+`census-determinism --repeat 3 --jobs 4` is 27/27 byte-identical on Darwin
+arm64 on both arms.

crates/perry/src/commands/compile.rs

@@ -36,6 +36,7 @@ mod link;
 mod lock_scan;
 mod lowering_report;
 mod object_cache;
+mod object_staging;
 mod optimized_libs;
 mod output_path;
 mod parse_cache;