Skip to content

Add non-boxing forEach traversal to the Swiss map matrix#13

Open
alexander-yevsyukov wants to merge 4 commits into
masterfrom
claude/sad-goldberg-e00da2
Open

Add non-boxing forEach traversal to the Swiss map matrix#13
alexander-yevsyukov wants to merge 4 commits into
masterfrom
claude/sad-goldberg-e00da2

Conversation

@alexander-yevsyukov

Copy link
Copy Markdown
Contributor

What & why

The primitive-value maps (LongLongMap, IntIntMap, and the four other cells)
exposed no iteration API — no forEach, no cursor, not even the boxed
asMutableMap() view the boxed maps have. Every competitor primitive-map library
offers a non-boxing traversal (fastutil forEach, HPPC cursors, Eclipse
forEachKeyValue, Agrona longForEach), and their absence meant the comparative
iterate benchmark op could not include our maps.

This adds a non-boxing forEach plus a primitive-specialized callback interface,
generated from the codegen templates like the rest of the matrix, and wires the
iterate op into the benchmark harness.

API

  • Primitive mapsforEach(action: <K><V>Consumer), e.g.
    LongLongConsumer.accept(Long, Long): neither key nor value boxed.
  • Boxed mapsforEach(action: <K>ObjectConsumer<V>), e.g.
    LongObjectConsumer<V>.accept(Long, V): primitive key unboxed (value is already
    an object).

Each Consumer is a top-level public fun interface co-located in its map's file,
mirroring the existing LongHasher/IntHasher house style. A lambda
map.forEach { key, value -> … } SAM-converts to it.

Why a functional interface and not (K, V) -> Unit: a Kotlin function type
erases to Function2, whose invoke boxes each primitive — exactly the cost this
library exists to eliminate. A public inline fun was also rejected: it may not
touch the internal Swar or the private storage, so it can't be a published
entry point. forEach is therefore a member (it needs that private/internal
state) and reuses the table's existing full-lane control-word walk, so it stays
allocation-free and off the hot path.

Iteration order is unspecified; the contract is "do not structurally modify during
the walk" (matching java.util.Map.forEach). The primitive maps keep no
modCount, so there is deliberately no fail-fast guard — adding one would tax the
allocation-free hot path.

Benchmark: the iterate op

Wired across the boxing gradient so one column ranks all three storage strategies
on the same traversal:

Benchmark Boxing on traversal
LongLongMapBenchmark.iterate neither key nor value
SwissLongMapBenchmark.iterate value only (key unboxed)
StdlibHashMapBenchmark.iterate both — the boxed baseline
IntIntMapBenchmark.iterate / iterateBoxed primitive vs. boxed, self-contained

Each sums key xor value into a primitive sink consumed once through the
Blackhole; the boxed arms iterate via entry destructuring ({ (k, v) -> … }) so
the call resolves on Native, not just the JVM BiConsumer overload. Running the
matrix on pinned hardware and publishing numbers stays Phase 6.

Testing

  • New deterministic forEach exact-once test and a randomized forEach-vs-HashMap
    property, generated across every cell and mirrored by hand into the frozen
    LongLongMap oracle specs.
  • ./gradlew clean build dokkaGenerate green: all KMP targets, detekt, Kover,
    verifyGeneratedSwissMaps (zero drift), and Dokka (new KDoc links resolve).
  • Benchmarks compile on JVM and Native.

Reviews

Ran the repo pre-PR gate: spine-code-review and kotlin-engineer both
APPROVE with zero Must-fix / Should-fix. Version bumped
1.0.0-SNAPSHOT-010-011, dependency reports regenerated.

Note for maintainers: this branch was prepared in a git worktree, where
.agents/scripts/pre-pr-gate.sh cannot locate its sentinel — it uses
"$repo_root/.git/pre-pr.ok", but in a worktree $repo_root/.git is a file,
so the path is unwritable/unreadable. The gate was run and passed; the sentinel
was written to the real per-worktree git dir
(git rev-parse --git-path pre-pr.ok). Consider resolving the sentinel via
git rev-parse --git-path pre-pr.ok so the hook works under worktrees.

🤖 Generated with Claude Code

alexander-yevsyukov and others added 3 commits July 5, 2026 00:02
The primitive-value maps (LongLongMap, IntIntMap, and the four other cells)
exposed no iteration API, so the comparative `iterate` benchmark op could not
include them, though every competitor primitive-map library offers a non-boxing
traversal.

Add a `forEach` member plus a primitive-specialized callback interface,
generated from the codegen templates:

- Primitive maps: e.g. `LongLongConsumer.accept(Long, Long)` — neither key nor
  value boxed. A plain `(K, V) -> Unit` erases to `Function2` and boxes both; a
  public `inline fun` may not touch the internal `Swar` or private storage, so a
  functional interface is the only non-boxing path.
- Boxed maps: e.g. `LongObjectConsumer<V>.accept(Long, V)` — the primitive key
  is unboxed (the value is already an object).

`forEach` reuses the table's existing full-lane control-word walk, so it stays
allocation-free and off the hot path; the primitive maps keep no modCount, so
the contract is "do not structurally modify during the walk" rather than
fail-fast. Iteration order is unspecified.

Wire the comparative `iterate` op into the benchmark harness across the boxing
gradient — LongLongMap (boxes neither) and SwissLongMap (value only) against the
HashMap baseline (both) — plus a self-contained IntIntMap primitive-vs-boxed pair.

Regenerate the checked-in sources: verifyGeneratedSwissMaps is clean and
:elastic:check passes (all KMP targets, detekt, Kover).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 4, 2026 23:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a non-boxing forEach traversal API across the Swiss map “matrix” (primitive-value and boxed-value variants), generates the corresponding primitive-specialized consumer SAM interfaces, and wires a new iterate operation into the benchmark harness. It also adds deterministic + property tests to ensure forEach visits each entry exactly once, and bumps the published snapshot version.

Changes:

  • Add forEach member traversal to primitive maps (*Map) and boxed Swiss maps (Swiss*Map), plus generated *Consumer callback interfaces to avoid boxing.
  • Extend test suites (deterministic + property-based) to validate forEach correctness across the matrix.
  • Add iterate benchmarks for Swiss, primitive, and stdlib baselines; bump snapshot version and regenerate dependency reports/templates.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
version.gradle.kts Bumps publish version to 1.0.0-SNAPSHOT-011.
elastic/src/commonMain/kotlin/io/spine/elastic/SwissLongMap.kt Adds non-boxing-key forEach and LongObjectConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/SwissIntMap.kt Adds non-boxing-key forEach and IntObjectConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/LongLongMap.kt Adds fully non-boxing forEach and LongLongConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/LongIntMap.kt Adds fully non-boxing forEach and LongIntConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/LongDoubleMap.kt Adds fully non-boxing forEach and LongDoubleConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/IntLongMap.kt Adds fully non-boxing forEach and IntLongConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/IntIntMap.kt Adds fully non-boxing forEach and IntIntConsumer.
elastic/src/commonMain/kotlin/io/spine/elastic/IntDoubleMap.kt Adds fully non-boxing forEach and IntDoubleConsumer.
elastic/src/commonTest/kotlin/io/spine/elastic/SwissLongMapViewSpec.kt Adds deterministic forEach “visits every entry” test for boxed Swiss map view.
elastic/src/commonTest/kotlin/io/spine/elastic/SwissIntMapViewSpec.kt Adds deterministic forEach “visits every entry” test for boxed Swiss map view.
elastic/src/commonTest/kotlin/io/spine/elastic/LongLongMapSpec.kt Adds deterministic forEach exact-once test for LongLongMap.
elastic/src/commonTest/kotlin/io/spine/elastic/LongLongMapPropertiesSpec.kt Adds property test comparing forEach output to a HashMap model.
elastic/src/commonTest/kotlin/io/spine/elastic/LongIntMapSpec.kt Adds deterministic forEach exact-once test for LongIntMap.
elastic/src/commonTest/kotlin/io/spine/elastic/LongIntMapPropertiesSpec.kt Adds forEach vs model property test for LongIntMap.
elastic/src/commonTest/kotlin/io/spine/elastic/LongDoubleMapSpec.kt Adds deterministic forEach exact-once test for LongDoubleMap.
elastic/src/commonTest/kotlin/io/spine/elastic/LongDoubleMapPropertiesSpec.kt Adds forEach vs model property test for LongDoubleMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntLongMapSpec.kt Adds deterministic forEach exact-once test for IntLongMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntLongMapPropertiesSpec.kt Adds forEach vs model property test for IntLongMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntIntMapSpec.kt Adds deterministic forEach exact-once test for IntIntMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntIntMapPropertiesSpec.kt Adds forEach vs model property test for IntIntMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntDoubleMapSpec.kt Adds deterministic forEach exact-once test for IntDoubleMap.
elastic/src/commonTest/kotlin/io/spine/elastic/IntDoubleMapPropertiesSpec.kt Adds forEach vs model property test for IntDoubleMap.
codegen/src/main/resources/io/spine/elastic/codegen/template/SwissMap.kt.tpl Generates forEach + %K%ObjectConsumer into Swiss boxed maps.
codegen/src/main/resources/io/spine/elastic/codegen/template/PrimitiveMap.kt.tpl Generates forEach + %K%%V%Consumer into primitive maps.
codegen/src/main/resources/io/spine/elastic/codegen/template/SwissMapViewSpec.kt.tpl Generates deterministic boxed-Swiss forEach test.
codegen/src/main/resources/io/spine/elastic/codegen/template/PrimitiveMapSpec.kt.tpl Generates deterministic primitive forEach exact-once test.
codegen/src/main/resources/io/spine/elastic/codegen/template/PrimitiveMapPropertiesSpec.kt.tpl Generates primitive forEach vs model property test.
benchmarks/src/commonMain/kotlin/io/spine/elastic/benchmark/SwissLongMapBenchmark.kt Adds iterate benchmark using SwissLongMap.forEach.
benchmarks/src/commonMain/kotlin/io/spine/elastic/benchmark/LongLongMapBenchmark.kt Adds iterate benchmark using LongLongMap.forEach.
benchmarks/src/commonMain/kotlin/io/spine/elastic/benchmark/StdlibHashMapBenchmark.kt Adds iterate benchmark for boxed baseline (needs a small fix).
benchmarks/src/commonMain/kotlin/io/spine/elastic/benchmark/IntIntMapBenchmark.kt Adds iterate and iterateBoxed benchmarks (boxed variant needs a small fix).
docs/dependencies/pom.xml Updates docs dependency version to -011.
docs/dependencies/dependencies.md Regenerates dependency report for -011.
.agents/tasks/primitive-map-foreach.md Adds agent task note for the work (config-managed docs).
.agents/memory/MEMORY.md Updates agent memory index (config-managed docs).
.agents/memory/codegen-template-article-check.md Adds agent memory entry about template article guard (config-managed docs).

Replace `map.forEach { (k, v) -> }` with `for ((k, v) in map)` in the boxed
`iterate` baselines (StdlibHashMapBenchmark, IntIntMapBenchmark). Behavior is
identical — both walk the entry set and box each key/value — but the loop reads
unambiguously as entry iteration, distinct from the primitive maps' non-boxing
two-arg `forEach { k, v -> }`. Addresses PR review feedback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.10%. Comparing base (a306baa) to head (eba9cd3).

Additional details and impacted files
@@             Coverage Diff              @@
##             master      #13      +/-   ##
============================================
+ Coverage     96.98%   97.10%   +0.11%     
- Complexity      431      455      +24     
============================================
  Files            22       22              
  Lines          1857     1933      +76     
  Branches        285      301      +16     
============================================
+ Hits           1801     1877      +76     
  Misses           26       26              
  Partials         30       30              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants