perf(codec): fast-path versioned decode by peeking at the first byte#149
Open
Kewe63 wants to merge 1 commit into
Open
perf(codec): fast-path versioned decode by peeking at the first byte#149Kewe63 wants to merge 1 commit into
Kewe63 wants to merge 1 commit into
Conversation
NetCodec and WalCodec are used by the consensus node on every incoming message, and every message they ship is now prefixed with the V1 version byte (0x01). The decoder still attempted a full protobuf parse of the un-versioned payload first before reading the version byte, on every single message — work that is guaranteed to fail for a V1 message because 0x01 is an invalid leading byte for any real Arc protobuf payload (the smallest field tag with a non-zero field number is 0x08). Add a pre-check on the first byte: if it matches the configured version, strip the byte and decode the rest as protobuf in a single pass. The legacy 'try-raw-protobuf-first' branch is preserved for backward compatibility with messages from pre-versioning nodes — that branch is the only one exercised by the existing test_previous_codec_compatibility tests in wal.rs and network.rs. Note this is complementary to the Phase 3 cleanup in circlefin#132: that PR removes the legacy branch entirely once all nodes are upgraded; this one only adds the fast path on top of the existing code, so it is safe to merge before Phase 3. Closes circlefin#131
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NetCodecandWalCodecare used on every incoming message in the consensus node. Every message is now prefixed with the V1 version byte (0x01), yet the decoder still attempted a full protobuf parse of the un-versioned payload first — before reading the version byte — on every single message. This work is guaranteed to fail for a V1 message because0x01cannot start any real Arc protobuf payload (the smallest protobuf field tag with a non-zero field number is0x08).Fix
Added a pre-check on the first byte inside
impl_versioned_codec!: if it matches the configured version byte, strip it and decode the remainder as protobuf in a single pass. The legacy "try-raw-protobuf-first" branch is preserved for backward compatibility with messages from pre-versioning nodes.This is complementary to the Phase 3 cleanup in #132, which removes the legacy branch entirely once all nodes are upgraded. This PR only adds the fast path on top of the existing code — safe to merge before Phase 3, and #132 can rebase cleanly on top.
Changes
File:
crates/types/src/codec/(impl_versioned_codec!macro)File:
crates/types/src/codec/wal.rsfast_path_decodes_v1_prefixed_message_in_one_passtest locking in two invariants:How to Test
The legacy branch is exercised by the existing
test_previous_codec_compatibilitytests inwal.rsandnetwork.rs— any regression there surfaces in those tests, not in the new one.Risk & Impact
Low. The fast path is additive — the legacy branch is fully preserved and exercised by existing tests. No behavioral change for pre-versioning messages. V1 messages now skip a guaranteed-to-fail protobuf parse on every decode.
Type: ⚡ Performance improvement
Closes: #131