Skip to content

common/test: reproduce BOLT 12 PR #1295 marker-jump spec contradiction#9170

Open
vincenzopalazzo wants to merge 3 commits into
ElementsProject:masterfrom
vincenzopalazzo:bolt12-payer-proof-marker-jump
Open

common/test: reproduce BOLT 12 PR #1295 marker-jump spec contradiction#9170
vincenzopalazzo wants to merge 3 commits into
ElementsProject:masterfrom
vincenzopalazzo:bolt12-payer-proof-marker-jump

Conversation

@vincenzopalazzo
Copy link
Copy Markdown
Collaborator

@vincenzopalazzo vincenzopalazzo commented May 27, 2026

Add a failing C unit test in common/test/run-bolt12_proof_marker_jump.c
that reproduces the BOLT 12 spec contradiction raised on
lightning/bolts#1295.

In lightning/bolts#1295 (head db3eff3a54, 12-offer-encoding.md),
the writer rule (lines 1041-1049) mandates that after a
proof_omitted_tlvs entry of 239, the next entry MUST be
1000000000, because the writer needs to skip past the
signature/payer-proof TLV range (240..999999999) which is not valid
for marker numbers.

However the reader rule (lines 1065-1067) only accepts the next entry
if it is prev + 1 or <included TLV> + 1. There is no carve-out for
the 239 -> 1000000000 jump, so a reader applying the spec literally
will reject a marker sequence the writer rule mandates. The issue is
still open on the spec PR.

CLN's reader in common/bolt12_proof.c lines 339-388 is a literal
transcription of the bogus reader rule. The buggy branch is at line
379, which errors out when omitted != prev_omitted + 1 and
find_tlv_num(pptlv, omitted - 1) is false:

if (omitted != prev_omitted + 1) {
    /* O(n^2) but doesn't matter */
    if (!find_tlv_num(pptlv, omitted - 1)) {
        return tal_fmt(ctx, "proof_omitted_tlvs[%zi] is"
                       " not one greater than the previous %"PRIu64
                       " nor an included tlv entry",
                       i, prev_omitted);
    }
}

Fed the writer-rule-mandated sequence [1, 2, ..., 239, 1000000000],
this returns an error on the 1000000000 entry, because it is not
prev_omitted + 1 = 240 and there is no included TLV at 999999999
for the fallback to find. That is exactly the contradiction.

The new test constructs the minimum tlv_payer_proof needed to reach
the marker-validation loop: required field pointers populated,
preimage matching invoice_payment_hash, pptlv->fields deliberately
left empty so find_tlv_num returns false in every branch.

Then it sets proof_omitted_tlvs to [1, 2, ..., 239, 1000000000]
and asserts that check_payer_proof returns NULL. It does not, and
the assertion blows up:

$ ./common/test/run-bolt12_proof_marker_jump
check_payer_proof rejected the writer-rule-mandated 239 -> 1000000000 jump: proof_omitted_tlvs[239] is not one greater than the previous 239 nor an included tlv entry
Assertion failed: (err == NULL), function main, file run-bolt12_proof_marker_jump.c, line 127.

The failure naming [239] (the slot holding 1000000000) and
previous 239 is exactly the spec contradiction surfaced in CLN.

The test is expected to fail on current master, the failure IS the
reproduction. I am not fixing the underlying bug in this PR, the fix
belongs in the spec PR first (amend reader rule 1065 to allow the
239 -> 1000000000 jump) and only then mirrored into CLN.

Landing the failing test now means the eventual spec fix has a
concrete CLN-side regression target instead of a vague note about
remembering to update the validator.

Link: lightning/bolts#1295 (comment)
Link: lightningdevkit/rust-lightning#4297

Signed-off-by: Vincenzo Palazzo vincenzopalazzodev@gmail.com


  • The changelog has been updated in the relevant commit(s) according to the guidelines.
  • Tests have been added or modified to reflect the changes.
  • Documentation has been reviewed and updated as needed.
  • Related issues have been listed and linked, including any that this PR closes.

…ec contradiction

Add a failing C unit test that reproduces the BOLT 12 spec
contradiction raised on lightning/bolts#1295.

The writer rule in `12-offer-encoding.md` (lines 1041-1049, head
db3eff3a54) mandates that after a `proof_omitted_tlvs` entry of
`239` the next entry MUST be `1000000000`, because the writer skips
past the signature/payer-proof TLV range (240..999999999).

The reader rule (lines 1065-1067) only accepts an entry that is
`prev + 1` or `<included TLV> + 1`, with no carve-out for the
239 -> 1000000000 jump.

A reader applying the spec literally rejects a marker sequence the
writer rule mandates.

CLN's reader in common/bolt12_proof.c lines 339-388 is a literal
transcription of the bogus reader rule (specifically the
`omitted != prev_omitted + 1` branch at line 379).

Fed the writer-rule-mandated sequence [1, 2, ..., 239, 1000000000]
it returns an error on the 1000000000 entry.

The new test in common/test/run-bolt12_proof_marker_jump.c builds
the minimum tlv_payer_proof needed to reach the marker-validation
loop and asserts check_payer_proof returns NULL.

It is expected to fail on current master, the failure IS the
reproduction. The fix belongs in the spec PR first (amend reader
rule 1065) and only then mirrored into CLN.

The same contradiction is reproduced on the Rust-Lightning side in
lightningdevkit/rust-lightning#4297, test
spec_writer_reader_rules_contradict_on_gap_jump in
lightning/src/offers/merkle.rs (commit 0034c79ae).

Changelog-None

Link: lightning/bolts#1295 (comment)
Link: lightningdevkit/rust-lightning#4297
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
In `lightning/bolts#1295` (head db3eff3a54, 12-offer-encoding.md), the
writer rule (lines 1041-1049) mandates that after a
`proof_omitted_tlvs` entry of `239` the next entry is `1000000000`
(jumping past the signature/payer-proof TLV range 240..999999999),
but the reader rule (lines 1065-1067) only accepts `prev + 1` or
`<included TLV> + 1`. A reader applying the spec literally rejects
a marker sequence the writer rule mandates. The contradiction was
raised on the spec PR:

  lightning/bolts#1295 (comment)

CLN's reader at `common/bolt12_proof.c` line 379 was a literal
transcription of the bogus reader rule, and CLN's writer at line
115 used `++last_type` which would have emitted `240` (invalid)
after `239` if a real invoice ever reached the boundary.

Fix this by collapsing both writer and reader onto a single helper
`next_marker(prev)` that returns the smallest valid marker number
strictly greater than `prev`. Both sides now reference the same
symbol, so the writer/reader asymmetry that caused this bug is
structurally impossible.

The unit test `run-bolt12_proof_marker_jump` is rewritten as a real
end-to-end test: it builds an invoice with the standard required
fields plus 64 dummy unknown TLVs at odd extension-range types
(1000000001, 1000000003, ..., 1000000127), makes a payer_proof
including only the three required fields, and asserts that
`check_payer_proof` returns NULL. The marker sequence produced ends
with [..., 237, 238, 239, 1000000000], exercising the gap-jump
end-to-end. The test also sanity-checks that 239 appears in the
sequence and that 1000000000 follows it immediately, so the test
cannot go green for the wrong reason.

Existing `run-bolt12_proof` and `run-bolt12_proof_vectors` still
pass with no changes to their behaviour.

Parallel LDK reproduction:
  lightningdevkit/rust-lightning#4297
(test `spec_writer_reader_rules_contradict_on_gap_jump` in
 lightning/src/offers/merkle.rs, commit 0034c79ae).

Changelog-Fixed: Reader of BOLT 12 payer_proof now accepts the writer-rule-mandated 239 -> 1000000000 marker jump.

Link: lightning/bolts#1295 (comment)
Link: lightningdevkit/rust-lightning#4297
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Add a new valid_vector `marker_jump_239_to_1000000000` to the JSON
output produced by `run-bolt12_proof_vectors`. The vector encodes a
real signed payer_proof whose `proof_omitted_tlvs` ends with
[..., 237, 238, 239, 1000000000], i.e. it actually contains the
spec gap-jump.

The point is cross-implementation coverage: with this vector in the
JSON corpus that CLN already emits, other BOLT 12 implementations
(LDK PR ElementsProject#4297, future eclair, ...) can consume the same bech32
string through their own decoder and verifier. A reader that still
has the old "must be prev + 1 or <included TLV> + 1" rule will
reject the proof with an error naming `1000000000` after `239`;
that surfaces the same latent bug across implementations rather
than having each project rediscover it independently.

The marker-jump invoice is built by `build_marker_jump_invoice`,
which is a minimal invoice (same keys A-F as the existing
vectors) plus 64 dummy unknown TLVs at odd extension-range types
(1000000001, 1000000003, ..., 1000000127). The new
`include_required_only` callback includes only the three required
fields (invreq_payer_id, invoice_payment_hash, invoice_node_id),
so the 64 trailing dummies and the optional invoice fields all
get omitted. The writer-side `next_marker` walks the marker
counter through 1..239 and then emits 1000000000 on the 64th
dummy, which is the case under test.

The existing 5 valid_vectors and 23 invalid_vectors are unchanged;
the new entry is appended at the end of `valid_vectors[]` so the
existing positions are preserved.

Link: lightning/bolts#1295 (comment)
Link: lightningdevkit/rust-lightning#4297
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant