Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# Exactly 2 coverage uploads per commit (CI.yml: ubuntu × {lts, 1.x} — the
# two version trees). Wait for both before posting statuses/comments, so a
# partially-merged report can never surface as a phantom coverage drop.
codecov:
notify:
after_n_builds: 2

coverage:
status:
project:
Expand All @@ -16,3 +23,4 @@ comment:
layout: "reach,diff,flags,files"
behavior: default
require_changes: true
after_n_builds: 2
9 changes: 9 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,20 @@ jobs:

- uses: julia-actions/julia-runtest@v1

# Coverage: upload from exactly TWO jobs — ubuntu × {lts, 1.x}. The two
# Julia versions compile different version-gated branches of the shared
# sources (legacy vs modern tree), so both are needed for a complete
# merged report; the other 7 matrix legs add no unique lines (no
# OS-conditional source, and 1.11 exercises the same legacy tree as lts).
# .codecov.yml `after_n_builds: 2` holds notification until both arrive,
# so a partially-merged report can't surface as a phantom coverage drop.
- uses: julia-actions/julia-processcoverage@v1
if: matrix.os == 'ubuntu-latest' && matrix.version != '1.11'
with:
directories: src

- uses: codecov/codecov-action@v5
if: matrix.os == 'ubuntu-latest' && matrix.version != '1.11'
with:
files: lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,6 @@ LocalPreferences.toml

# Auto-generated docs (created by make.jl)
/docs/src/index.md

# Release changelog — maintained manually outside version control
/CHANGELOG.md
100 changes: 0 additions & 100 deletions CHANGELOG.md

This file was deleted.

58 changes: 45 additions & 13 deletions docs/src/safety/compile-time.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,43 @@ The `@with_pool` macro statically analyzes your code at macro expansion time —

## Escape Analysis (`PoolEscapeError`)

Rejects any expression that would return a pool-backed array. Tracks aliases, containers, and convenience wrappers:
Detects any tail expression that would deliver a pool-backed array out of the
scope. Tracks aliases, containers, and convenience wrappers. **Severity is
form-based**, because the two forms differ in what the macro can know:

- **Function form** and **explicit `return`** (either form): the value
definitively reaches the enclosing function's caller — a `begin` block is
not a function boundary, so `return` inside a block form returns from the
*enclosing function*. These always throw `PoolEscapeError` at expansion.
- **Block form, implicit last expression**: the macro cannot see its own call
site, so whether the block's value is used (`out = @with_pool ...`) or
discarded (a loop body, a bare statement) is undecidable. These emit a
`@warn` and the expansion **replaces the escaping value with an inert
`EscapedPoolArray` guard**: discarded → completely silent; used → the first
operation throws `EscapedPoolUseError` with the variable name, shape, and
scope location.

```julia
@with_pool pool begin
@with_pool pool function bad()
A = acquire!(pool, Float64, 3, 3)
return A # ← PoolEscapeError: function return, always an error
end

out = @with_pool pool begin
v = acquire!(pool, Float64, 100)
w = v # alias tracked
w # ← PoolEscapeError: w escapes
w # ← @warn + guard: `out` is an EscapedPoolArray
end

@with_pool pool function bad()
A = acquire!(pool, Float64, 3, 3)
return A # ← PoolEscapeError: explicit return
out[1] # ← EscapedPoolUseError: names `w`, its shape, and the scope

# The guard is inert when the value is never used — this common
# copy-out-and-discard shape runs silently and correctly:
Threads.@threads for k in 1:nmodels
@with_pool pool begin
y = acquire!(pool, Float64, m, n)
compute!(y, k)
results[:, :, k] = y # copy runs; the yielded `y` is guarded; @threads discards it
end
end
```

Expand Down Expand Up @@ -64,7 +89,8 @@ end
end
```

Fix: end the block with `nothing` if the value is meant to be discarded:
Fix: end the block with `nothing` if the value is meant to be discarded —
this also silences the warning:

```julia
@with_pool pool begin
Expand All @@ -74,14 +100,20 @@ Fix: end the block with `nothing` if the value is meant to be discarded:
end
```

These three patterns are gated by the `escape_lint` preference (default
`"error"`, matching the direct-return patterns above). Set `"warn"` to
downgrade to a `@warn` diagnostic (migration escape hatch), or `"off"` to
disable this stage of checking entirely:
In block form these follow the same warn + `EscapedPoolArray`-guard treatment
as all implicit tails; in function form or under an explicit `return`, they
throw. The `escape_lint` preference tunes the **block-form** severity:

- `"warn"` (default) — `@warn` diagnostic + the guard rewrite described above.
- `"error"` — block-form implicit tails also throw `PoolEscapeError`
(the strict pre-guard behavior; no guard rewrite needed).
- `"off"` — no warning and no guard rewrite (the block tail escapes the raw
pool array, as before this feature). Function-form and explicit-`return`
direct escapes still always error.

```julia
using Preferences
Preferences.set_preferences!("AdaptiveArrayPools", "escape_lint" => "warn")
Preferences.set_preferences!("AdaptiveArrayPools", "escape_lint" => "error")
```

## Mutation Analysis (`PoolMutationError`)
Expand Down
6 changes: 4 additions & 2 deletions docs/src/safety/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,10 @@ zero-GC hot path under `S=1` — use `S=0`.
exit — it cannot prove ownership through arrays captured by closures, stored in
globals that outlive the scope, or aliased behind opaque calls that never surface
in the return value. These are undecidable statically; `S=1` catches the
return-position cases, and the [compile-time lint](compile-time.md) catches the
common syntactic ones.
return-position cases, and the [compile-time analysis](compile-time.md) handles
the common syntactic ones (function-form/`return` escapes error at expansion;
block-tail escapes are replaced by an `EscapedPoolArray` guard that traps on
first use even at `S=0`).

**Test hygiene.** For tests that run under both `S=0` and `S=1`: keep asserts
S-adaptive (don't assert poison values unless `S≥1`), and end `@with_pool` blocks
Expand Down
10 changes: 8 additions & 2 deletions src/AdaptiveArrayPools.jl
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,20 @@ export AbstractTypedPool, AbstractArrayPool # For subtyping
export DisabledPool, DISABLED_CPU, pooling_enabled # Disabled pool support
# Note: Extensions add methods to _get_pool_for_backend(::Val{:backend}) directly

# Expansion-time incidental-tail escape severity: "error" (default) | "warn" | "off".
# Expansion-time block-form escape severity: "warn" (default) | "error" | "off".
# Under "warn", block-form implicit-tail escapes warn AND the escaping value is
# replaced by an EscapedPoolArray guard (see src/escape_guard.jl); function-form
# and explicit-return escapes always error regardless of this setting.
# Read once at package load; a compile-time constant like RUNTIME_CHECK.
const ESCAPE_LINT = let v = @load_preference("escape_lint", "error")
const ESCAPE_LINT = let v = @load_preference("escape_lint", "warn")
v in ("error", "warn", "off") ||
error("escape_lint preference must be \"error\", \"warn\", or \"off\" (got \"$v\")")
v
end

# Version-independent escape guard (used by macro expansion on both trees)
include("escape_guard.jl")

# All includes grouped under a single version branch
@static if VERSION >= v"1.12-"
include("types.jl")
Expand Down
122 changes: 122 additions & 0 deletions src/escape_guard.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# ==============================================================================
# EscapedPoolArray — inert guard replacing a pool-backed value that escapes a
# block-form `@with_pool` scope as its implicit tail value.
#
# The macro cannot see its own call site, so whether a block's value is used
# (`out = @with_pool ...`) or discarded (loop body, bare statement) is
# undecidable at expansion time. Instead of erroring on safe-but-flagged code
# or warning on buggy code, the expansion replaces the escaping value with this
# guard: discarded → completely inert; used → throws `EscapedPoolUseError`
# with full provenance at the first trapped operation.
#
# Deliberately metadata-only — the guard does NOT hold the array: it must not
# extend the rewound buffer's GC lifetime, and "unwrap and use anyway" must be
# impossible. `typeof`/`size` are recorded at the tail-capture point (before
# rewind), where the array is still valid.
#
# Version-independent (shared by the modern and legacy trees, like macros.jl).
# ==============================================================================

struct EscapedPoolArray
var::Symbol # source variable name, or :expression for anonymous tails
arraytype::Type # typeof(x) at escape, e.g. Matrix{Float64}
dims::Tuple{Vararg{Int}} # size(x) at escape; () when not an AbstractArray
file::Union{String, Nothing} # @with_pool scope location
line::Union{Int, Nothing}
end

# Expansion-emitted constructor: records provenance, discards the array.
function EscapedPoolArray(
x, var::Symbol, file::Union{String, Nothing}, line::Union{Int, Nothing},
)
return EscapedPoolArray(var, typeof(x), x isa AbstractArray ? size(x) : (), file, line)
end

"""
EscapedPoolUseError <: Exception

Thrown on the first use of an [`EscapedPoolArray`](@ref) guard — a pool-backed
array that escaped its `@with_pool` scope as the block's implicit tail value.
The array was recycled by the scope's rewind, so any use of the escaped value
is invalid; the guard defers the error to the moment of actual use (a merely
discarded escape stays silent).
"""
struct EscapedPoolUseError <: Exception
guard::EscapedPoolArray
op::Symbol # the trapped operation that was attempted
end

# "`x` (3×2 Matrix{Float64})" / "an anonymous expression (Vector{Float64})"
function _show_guard_identity(io::IO, g::EscapedPoolArray)
if g.var === :expression
print(io, "an anonymous expression")
else
print(io, "`", g.var, "`")
end
if isempty(g.dims)
print(io, " (", g.arraytype, ")")
else
print(io, " (", join(g.dims, "×"), " ", g.arraytype, ")")
end
return nothing
end

_guard_location(g::EscapedPoolArray) =
g.file === nothing ? nothing : (g.line === nothing ? g.file : "$(g.file):$(g.line)")

# Non-throwing, informative display: the REPL showing a discarded-but-displayed
# guard must explain itself, not crash.
function Base.show(io::IO, g::EscapedPoolArray)
print(io, "EscapedPoolArray: pool-backed array ")
_show_guard_identity(io, g)
print(io, " escaped its `@with_pool` scope")
loc = _guard_location(g)
loc !== nothing && print(io, " at ", loc)
print(io, " — invalid after the scope rewinds; any use throws EscapedPoolUseError")
return nothing
end

function Base.showerror(io::IO, e::EscapedPoolUseError)
g = e.guard
printstyled(io, "EscapedPoolUseError"; color = :red, bold = true)
print(io, ": `", e.op, "` on a pool-backed array that escaped its `@with_pool` scope\n\n")
printstyled(io, " Escaped value: "; color = :light_black)
_show_guard_identity(io, g)
print(io, "\n")
loc = _guard_location(g)
if loc !== nothing
printstyled(io, " Scope location: "; color = :light_black)
print(io, loc, "\n")
end
printstyled(
io,
"\n A pool-backed array is only valid inside its `@with_pool` scope — the\n" *
" scope's rewind recycles it, so the escaping value was replaced by this\n" *
" inert guard at scope exit.\n";
color = :light_black,
)
print(io, "\n Fix options:\n")
print(io, " • materialize an owned copy inside the scope: `collect(x)` / `copy(x)`\n")
print(io, " • return a scalar or other non-pool value instead\n")
print(io, " • end the block with `nothing` if the value is not meant to be used\n")
printstyled(
io,
"\n If this escape looks like a false positive, please file an issue\n" *
" with a minimal reproducer so we can improve the escape detector.\n";
color = :light_black,
)
return nothing
end

# Trap surface: the common array entry points all throw with provenance. Any
# operation not listed here falls through to a MethodError that names the
# guard type — self-documenting. Identity/no-dispatch operations (`===`,
# `isnothing`, `typeof`) intentionally keep working so harmless plumbing
# around a discarded value stays silent.
for f in (
:getindex, :setindex!, :size, :length, :axes, :iterate,
:firstindex, :lastindex, :similar, :copy, :view, :vec, :keys,
)
@eval Base.$f(g::EscapedPoolArray, args...) = throw(EscapedPoolUseError(g, $(QuoteNode(f))))
end
Base.broadcastable(g::EscapedPoolArray) = throw(EscapedPoolUseError(g, :broadcastable))
Loading
Loading