Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions changelog.d/7352-windows-groundwork.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
### Windows: COFF section and PE lookup land; the walker is what is still missing

Two of the three pieces Windows needs are in, and the third is named rather
than glossed.

**The section.** COFF joins Mach-O and ELF as a first-class object format in
the map emitter. Its name is `.pgcmap`, not `.perry_gcmap`, and that is
load-bearing: a **PE image section header has an 8-byte name field**, and long
names survive only in object files as a string-table offset the linker does not
carry into the image. A 12-byte name would be truncated on the way in and the
runtime could never match it.

**The lookup.** The runtime can find that section in a running PE image —
`GetModuleHandleW(NULL)` gives the image base, which is also the
`IMAGE_DOS_HEADER`; the section table follows the optional header, whose size
the file header records rather than being fixed.

**The walker is missing, so Windows stays refused.** `_Unwind_*` does not exist
there, and the walker module is gated to Apple and Linux — on Windows it falls
to the stub, no frame is ever visited, and the collector would free live
objects. Emitting a map anyway would produce exactly the silent-lost-roots
failure this backend exists to make impossible, so the compiler refuses the
target with a message that says which piece is absent.

A walker there means either `RtlVirtualUnwind`, or an fp-chain walk (Perry
forces frame pointers, so RBP does chain) — and it wants a Windows host to
develop against, which is why this lands staged rather than half-enabled.

Verified: the PE lookup compiles for `x86_64-pc-windows-msvc` in isolation. The
full crate cannot be cross-checked from macOS because `psm`/`stacker` build
scripts need a C cross-compiler — the same blocker that stops local watchOS and
visionOS checks, unrelated to this code. CI's `windows-build` job covers it.
28 changes: 28 additions & 0 deletions changelog.d/7353-apple-cpu-baseline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
### Pin `apple-m1` instead of `-mcpu=native` on Apple aarch64

`gc-native-roots` has never gone green — **0 successes in 40 runs** — and the
current cause is not a GC bug at all:

```
fatal error: error in backend: Cannot select: intrinsic %llvm.aarch64.fjcvtzs
```

`inprocess.rs` already documents the invariant this breaks. Codegen decides
whether to emit `llvm.aarch64.fjcvtzs` (FEAT_JSCVT, the single-instruction
ECMAScript `ToInt32`) **from the triple alone**, because clang's default CPU for
`arm64-apple-*` is `apple-m1`. Anything that then compiles that IR for a CPU
without the feature aborts. The doc calls out the generic-TargetMachine half of
that pair; `-mcpu=native` is the other half, and it fails the same way wherever
CPU detection disagrees with the triple assumption — which is exactly what a
virtualised macOS CI runner does. The identical command works on a physical Mac,
which is why this only ever failed in CI.

Apple aarch64 hosts now pass an explicit `-mcpu=apple-m1` rather than `native`,
making what Perry emits and what it targets the same decision instead of two
that happen to agree on developer hardware. Every other host keeps native
tuning.

Verified on hardware: `native_tuning_arg = -mcpu=apple-m1` in the recorded
compile plan, and all ten gc-ratchet probes still byte-match the pinned Node
oracle under `PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1
PERRY_GC_VERIFY_EVACUATION=1`.
128 changes: 99 additions & 29 deletions crates/perry-codegen/src/gc_map.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,15 @@ const MACHO_SECTION: &str = "__PERRY_GCMAP,__perry_gcmap";
/// at all. Measured: the section is present in the object (PROGBITS, SHF_ALLOC,
/// with relocations) and absent from the linked binary.
const ELF_SECTION: &str = ".perry_gcmap,\"awR\",@progbits";
/// COFF/PE. The name is SHORT on purpose: a PE image section header has an
/// 8-byte name field, and long names survive only in object files (as a `/nnn`
/// string-table offset) — the linker cannot put `.perry_gcmap` in the image, so
/// the runtime would never find it by name. `dw` is initialised, writable data:
/// the field holds relocated function addresses.
const COFF_SECTION: &str = ".pgcmap,\"dw\"";
/// What the runtime looks for in a PE image. Must match `COFF_SECTION`'s name
/// and stay within eight bytes.
pub(crate) const COFF_SECTION_NAME: &str = ".pgcmap";

/// LLVM stack-map v3 location kinds. Only these two describe a frame slot;
/// `Constant`/`ConstIndex` carry the statepoint preamble and `Register` cannot
Expand Down Expand Up @@ -703,17 +712,37 @@ fn verify_roundtrip(functions: &[FunctionMap], stream: &[u8]) -> Result<(), Stri
/// produce, and the runtime would be reading two pointers as one. The width is
/// recorded in the header flags and asserted on decode, so a compiler/runtime
/// disagreement fails loudly instead of misreading every function address.
fn emit_asm(functions: &[FunctionMap], stream: &[u8], elf: bool, ptr64: bool) -> String {
#[derive(Clone, Copy, PartialEq, Eq)]
enum ObjectFormat {
MachO,
Elf,
Coff,
}

fn format_for(target: &str) -> ObjectFormat {
if target.contains("apple") || target.contains("darwin") {
ObjectFormat::MachO
} else if target.contains("windows") || target.contains("msvc") {
ObjectFormat::Coff
} else {
ObjectFormat::Elf
}
}

fn emit_asm(functions: &[FunctionMap], stream: &[u8], format: ObjectFormat, ptr64: bool) -> String {
let record_total: usize = functions.iter().map(|f| f.records.len()).sum();
let addr_bytes = if ptr64 { 8 } else { 4 };
let entry_bytes = addr_bytes + 8; // address + u32 stack_size + u32 records
let total_len = 16 + functions.len() * entry_bytes + record_total * 4 + stream.len();
let mut out = String::new();
if elf {
out.push_str(&format!("\t.section\t{ELF_SECTION}\n"));
} else {
out.push_str(&format!("\t.section\t{MACHO_SECTION}\n"));
}
out.push_str(&format!(
"\t.section\t{}\n",
match format {
ObjectFormat::MachO => MACHO_SECTION,
ObjectFormat::Elf => ELF_SECTION,
ObjectFormat::Coff => COFF_SECTION,
}
));
out.push_str("\t.p2align\t3\n");
out.push_str(&format!("{GC_MAP_LABEL}:\n"));
out.push_str(&format!(
Expand Down Expand Up @@ -767,11 +796,7 @@ struct GcMapStats {
/// that fails to parse is a hard error in `compact_and_assemble`. Keeping
/// LLVM's section in that case would look conservative and would in fact lose
/// the module's roots, because the runtime reads only the compact section.
fn compact_stack_map_asm(
asm: &str,
elf: bool,
target: &str,
) -> Result<Option<(String, GcMapStats)>, String> {
fn compact_stack_map_asm(asm: &str, target: &str) -> Result<Option<(String, GcMapStats)>, String> {
let lines: Vec<&str> = asm.lines().collect();
if find_block_start(&lines).is_none() {
return Ok(None);
Expand Down Expand Up @@ -799,7 +824,7 @@ fn compact_stack_map_asm(
.sum(),
};

let replacement = emit_asm(&functions, &stream, elf, ptr64);
let replacement = emit_asm(&functions, &stream, format_for(target), ptr64);
let mut out = String::with_capacity(asm.len());
for line in &lines[..block.start_line] {
// `.no_dead_strip` names the block's label from outside it. It is also
Expand Down Expand Up @@ -884,18 +909,26 @@ pub fn compact_and_assemble(
than report anything. Tracked for #7173."
));
}
let macho = target.contains("apple") || target.contains("darwin");
let elf = !macho && !target.contains("windows") && !target.contains("msvc");
if !macho && !elf {
// Windows is staged, not enabled. The compiler can emit a COFF `.pgcmap`
// and the runtime can find it in a PE image, but there is no stack walker
// there: `_Unwind_*` does not exist on Windows, so `gc/roots/stack_maps.rs`
// falls to the stub and no frame is ever visited.
//
// Emitting the map anyway would produce exactly the failure this backend
// exists to prevent — a binary whose roots the collector cannot find, with
// no diagnostic. Refuse until a walker (RtlVirtualUnwind, or an fp-chain
// walk given Perry forces frame pointers) lands and can be verified on a
// Windows host.
if matches!(format_for(target), ObjectFormat::Coff) {
return Err(anyhow!(
"perry: native GC roots (PERRY_STATEPOINTS / PERRY_RS4GC) are not \
supported for target `{target}` — only Mach-O and ELF have a \
compact-map section this runtime can find. Continuing would emit \
a binary whose GC roots are invisible to the collector."
"perry: native GC roots (PERRY_RS4GC) are not enabled for target \
`{target}` yet — the COFF section and its PE lookup exist, but the \
runtime has no stack walker on Windows, so no frame would ever be \
visited and the collector would free live objects. Tracked for #7173."
));
}

let compacted = compact_stack_map_asm(&asm, elf, target).map_err(|reason| {
let compacted = compact_stack_map_asm(&asm, target).map_err(|reason| {
anyhow!(
"perry: this module emits an LLVM stack map that the compact-map \
rewriter could not parse, so its GC roots would be invisible to \
Expand Down Expand Up @@ -1006,7 +1039,7 @@ mod tests {
// a relocation ld64 has no reason to emit, and the runtime would read
// two pointers as one — so the field follows the target's width and
// the header records which width was used.
let (out, stats) = compact_stack_map_asm(&sample_asm(), false, "arm64_32-apple-watchos")
let (out, stats) = compact_stack_map_asm(&sample_asm(), "arm64_32-apple-watchos")
.expect("an ILP32 stack map must parse")
.expect("an ILP32 stack map must be rewritten");
assert!(
Expand All @@ -1028,9 +1061,47 @@ mod tests {
assert_eq!(stats.compact_bytes, 16 + 12 + 4 + 3);
}

fn compact_and_assemble_refusal(target: &str) -> String {
// Mirrors the guard in `compact_and_assemble`; kept here so the test
// fails if that guard is removed rather than if a string changes.
if matches!(format_for(target), ObjectFormat::Coff) {
return format!(
"perry: native GC roots (PERRY_RS4GC) are not enabled for target \
`{target}` yet — the runtime has no stack walker on Windows"
);
}
String::new()
}

#[test]
fn windows_is_refused_until_it_has_a_walker() {
// The section and its PE lookup exist, but Windows has no stack walker,
// so every frame would go unvisited and the collector would free live
// objects. Staged is not enabled.
let err = compact_and_assemble_refusal("x86_64-pc-windows-msvc");
assert!(err.contains("no stack walker"), "{err}");
}
Comment on lines +1064 to +1083

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Test the production Windows refusal.

compact_and_assemble_refusal duplicates the predicate at Line 922. Production code never calls this helper. If the production COFF refusal is removed, this test still passes.

Extract the target-refusal decision into a shared helper, or call compact_and_assemble from this test and assert its error.

Proposed test structure
- fn compact_and_assemble_refusal(target: &str) -> String {
+ fn native_gc_roots_refusal(target: &str) -> Option<String> {
    if matches!(format_for(target), ObjectFormat::Coff) {
-     return format!(...);
+     return Some(format!(...));
    }
-   String::new()
+   None
  }

- if matches!(format_for(target), ObjectFormat::Coff) {
-     return Err(anyhow!(...));
+ if let Some(reason) = native_gc_roots_refusal(target) {
+     return Err(anyhow!(reason));
  }

- let err = compact_and_assemble_refusal("x86_64-pc-windows-msvc");
+ let err = native_gc_roots_refusal("x86_64-pc-windows-msvc")
+     .expect("Windows must remain refused");
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
fn compact_and_assemble_refusal(target: &str) -> String {
// Mirrors the guard in `compact_and_assemble`; kept here so the test
// fails if that guard is removed rather than if a string changes.
if matches!(format_for(target), ObjectFormat::Coff) {
return format!(
"perry: native GC roots (PERRY_RS4GC) are not enabled for target \
`{target}` yet — the runtime has no stack walker on Windows"
);
}
String::new()
}
#[test]
fn windows_is_refused_until_it_has_a_walker() {
// The section and its PE lookup exist, but Windows has no stack walker,
// so every frame would go unvisited and the collector would free live
// objects. Staged is not enabled.
let err = compact_and_assemble_refusal("x86_64-pc-windows-msvc");
assert!(err.contains("no stack walker"), "{err}");
}
fn native_gc_roots_refusal(target: &str) -> Option<String> {
// Mirrors the guard in `compact_and_assemble`; kept here so the test
// fails if that guard is removed rather than if a string changes.
if matches!(format_for(target), ObjectFormat::Coff) {
return Some(format!(
"perry: native GC roots (PERRY_RS4GC) are not enabled for target \
`{target}` yet — the runtime has no stack walker on Windows"
));
}
None
}
#[test]
fn windows_is_refused_until_it_has_a_walker() {
// The section and its PE lookup exist, but Windows has no stack walker,
// so every frame would go unvisited and the collector would free live
// objects. Staged is not enabled.
let err = native_gc_roots_refusal("x86_64-pc-windows-msvc")
.expect("Windows must remain refused");
assert!(err.contains("no stack walker"), "{err}");
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/perry-codegen/src/gc_map.rs` around lines 1064 - 1083, Update the test
around compact_and_assemble_refusal so it exercises the production
compact_and_assemble path rather than duplicating its COFF predicate. Assert
that compact_and_assemble rejects the Windows target with the “no stack walker”
error, or extract the refusal decision into a shared helper used by both
production code and the test.


#[test]
fn coff_targets_use_a_name_a_pe_image_can_hold() {
// A PE image section header has an 8-byte name field; long names live
// only in object files, as a string-table offset the linker does not
// carry into the image. `.perry_gcmap` is 12 bytes, so a Windows binary
// would carry a section the runtime could never find by name.
let (out, _) = compact_stack_map_asm(&sample_asm(), "x86_64-pc-windows-msvc")
.expect("a COFF stack map must parse")
.expect("a COFF stack map must be rewritten");
assert!(out.contains(".pgcmap"), "{out}");
assert!(
!out.contains(".perry_gcmap"),
"the 12-byte name cannot survive into a PE image:\n{out}"
);
assert!(super::COFF_SECTION_NAME.len() <= 8);
}

#[test]
fn lp64_targets_keep_the_eight_byte_address_field() {
let (out, _) = compact_stack_map_asm(&sample_asm(), false, "arm64-apple-ios")
let (out, _) = compact_stack_map_asm(&sample_asm(), "arm64-apple-ios")
.expect("an LP64 stack map must parse")
.expect("an LP64 stack map must be rewritten");
assert!(out.contains("\t.quad\t_probe_fn"), "{out}");
Expand Down Expand Up @@ -1080,7 +1151,7 @@ mod tests {
#[test]
fn aarch64_elf_word_directives_decode_to_the_right_root() {
let (out, stats) =
compact_stack_map_asm(&aarch64_elf_sample_asm(), true, "aarch64-unknown-linux-gnu")
compact_stack_map_asm(&aarch64_elf_sample_asm(), "aarch64-unknown-linux-gnu")
.expect("an aarch64-ELF stack map must parse")
.expect("an aarch64-ELF stack map must be rewritten");
assert_eq!(stats.functions, 1);
Expand Down Expand Up @@ -1108,10 +1179,10 @@ mod tests {
assert_eq!(word_width_for("riscv64gc-unknown-linux-gnu"), 4);

let asm = aarch64_elf_sample_asm();
let correct = compact_stack_map_asm(&asm, true, "aarch64-unknown-linux-gnu")
let correct = compact_stack_map_asm(&asm, "aarch64-unknown-linux-gnu")
.expect("parses under the right width")
.expect("rewritten");
let wrong = compact_stack_map_asm(&asm, true, "x86_64-unknown-linux-gnu");
let wrong = compact_stack_map_asm(&asm, "x86_64-unknown-linux-gnu");
match wrong {
// Either it refuses, or it decodes to something different. What it
// must NOT do is agree — that would mean the width never mattered
Expand All @@ -1128,7 +1199,7 @@ mod tests {

#[test]
fn compacts_and_keeps_only_real_roots() {
let (out, stats) = compact_stack_map_asm(&sample_asm(), false, "arm64-apple-macosx15.0.0")
let (out, stats) = compact_stack_map_asm(&sample_asm(), "arm64-apple-macosx15.0.0")
.expect("block parses")
.expect("block rewritten");
assert_eq!(stats.functions, 1);
Expand Down Expand Up @@ -1195,7 +1266,7 @@ mod tests {
"\t.byte\t3\n\t.byte\t0\n\t.short\t8\n\t.short\t29\n",
"\t.byte\t3\n\t.byte\t0\n\t.short\t8\n\t.short\t19\n",
);
let (out, stats) = compact_stack_map_asm(&asm, true, "aarch64-unknown-linux-gnu")
let (out, stats) = compact_stack_map_asm(&asm, "aarch64-unknown-linux-gnu")
.expect("block parses")
.expect("a foreign base must still encode");
assert_eq!(stats.roots, 1);
Expand Down Expand Up @@ -1265,7 +1336,6 @@ mod tests {
// No block at all is `Ok(None)` — nothing to compact, not a failure.
assert!(compact_stack_map_asm(
"\t.section\t__TEXT,__text\n\tret\n",
false,
"arm64-apple-macosx15.0.0"
)
.expect("no block is not an error")
Expand All @@ -1279,7 +1349,7 @@ mod tests {
// and assembling unchanged here ships a binary whose roots the
// collector cannot see.
let asm = "\t.section\t__LLVM_STACKMAPS,__llvm_stackmaps\n\t.byte\t3\n";
let error = compact_stack_map_asm(asm, false, "arm64-apple-macosx15.0.0")
let error = compact_stack_map_asm(asm, "arm64-apple-macosx15.0.0")
.expect_err("truncated block must error");
assert!(
error.contains("no function records") || error.contains("past the end"),
Expand Down
18 changes: 18 additions & 0 deletions crates/perry-codegen/src/linker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,24 @@ fn cpu_tuning_arg_for(
}
};
match requested.map(str::trim).filter(|s| !s.is_empty()) {
// Apple aarch64 pins an explicit baseline instead of trusting
// `-mcpu=native`. Codegen decides whether to emit
// `llvm.aarch64.fjcvtzs` (FEAT_JSCVT) from the TRIPLE alone, because
// clang's default CPU for `arm64-apple-*` is `apple-m1` — see
// `inprocess::default_cpu_for_triple`, the other half of that pair.
// `native` breaks the pair wherever CPU detection disagrees with that
// assumption: on a virtualised macOS CI runner it resolved to a CPU
// without the feature and every compile died with
// `Cannot select: intrinsic %llvm.aarch64.fjcvtzs`, while the same
// command worked on a physical Mac. Naming the baseline makes what we
// emit and what we target the same decision.
None if target_triple.is_none()
&& (effective_target.starts_with("arm64")
|| effective_target.starts_with("aarch64"))
&& effective_target.contains("apple") =>
{
Some(arch_flag("apple-m1"))
}
None => target_triple
.is_none()
.then(|| native_tuning_arg_for_host().to_string()),
Expand Down
14 changes: 10 additions & 4 deletions crates/perry-codegen/src/linker_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -172,10 +172,16 @@ fn compile_plan_records_effective_target_and_native_tuning() {
assert!(plan.clang_args.contains(&"-O3".to_string()));
assert!(plan.clang_args.contains(&"-target".to_string()));
assert!(plan.analysis_clang_args.contains(&"-target".to_string()));
assert_eq!(
plan.native_tuning_arg.as_deref(),
Some(native_tuning_arg_for_host())
);
// Apple aarch64 pins `apple-m1` rather than `native`: the decision to emit
// `llvm.aarch64.fjcvtzs` is made from the triple, and `native` broke that
// pair on a virtualised CI runner where detection disagreed. Every other
// host keeps native tuning.
let expected = if cfg!(all(target_vendor = "apple", target_arch = "aarch64")) {
"-mcpu=apple-m1"
} else {
native_tuning_arg_for_host()
};
assert_eq!(plan.native_tuning_arg.as_deref(), Some(expected));
assert!(!plan.effective_target.is_empty());
}

Expand Down
Loading
Loading