feat(js): add VSR framing to the Node SDK - #3763
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3763 +/- ##
============================================
+ Coverage 75.12% 75.51% +0.39%
Complexity 969 969
============================================
Files 1307 1317 +10
Lines 152751 154895 +2144
Branches 128184 128707 +523
============================================
+ Hits 114756 116971 +2215
+ Misses 34465 34295 -170
- Partials 3530 3629 +99
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Adds runtime-selectable VSR framing support to the Node SDK so it can connect to VSR servers while keeping the existing sendBinaryRequest(code, payload) API.
Changes:
- Introduces VSR wire support (headers, operations, namespace routing, register handshake, session state, and reply decoding) plus protocol parity checks.
- Extends the raw client transport to support both classic and VSR framing with bounded frame extraction, retries, and leader redirection.
- Adds consumer-group support in VSR polling (
SyncGroup) and expands unit/e2e/CI coverage (including a VSR e2e task).
Reviewed changes
Copilot reviewed 50 out of 51 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| foreign/node/src/wire/vsr/vsr.test.ts | Adds unit coverage for VSR framing behavior from the session encoder and command preparation. |
| foreign/node/src/wire/vsr/session.ts | Implements consensus session state for clientId/session binding and request id sequencing. |
| foreign/node/src/wire/vsr/session.test.ts | Tests session binding, monotonic request ids, exhaustion behavior, and re-arming registration. |
| foreign/node/src/wire/vsr/reply.ts | Implements VSR reply/eviction decoding and result-section stripping with typed errors. |
| foreign/node/src/wire/vsr/reply.test.ts | Tests reply decoding, truncation handling, and eviction mapping. |
| foreign/node/src/wire/vsr/register.ts | Implements VSR register/login request payloads and response deserialization. |
| foreign/node/src/wire/vsr/register.test.ts | Tests register payload encoding/decoding and truncation paths. |
| foreign/node/src/wire/vsr/operation.ts | Adds command-code → VSR operation mapping plus operation classification helpers. |
| foreign/node/src/wire/vsr/operation.test.ts | Tests operation mapping/classification and known-operation filtering. |
| foreign/node/src/wire/vsr/namespace.ts | Implements routing namespace derivation (peeking payloads for partition-plane ops). |
| foreign/node/src/wire/vsr/namespace.test.ts | Tests namespace routing rules and payload peek validation/error paths. |
| foreign/node/src/wire/vsr/index.ts | Provides VSR session encoder, response decode adapter, and legacy-login preparation for VSR. |
| foreign/node/src/wire/vsr/header.ts | Defines VSR header layout/constants and request header encoding helpers. |
| foreign/node/src/wire/vsr/header.test.ts | Tests header encoding at canonical offsets and maximum-width fields. |
| foreign/node/src/wire/topic/topic.utils.ts | Fixes base topic field offsets to match updated wire layout. |
| foreign/node/src/wire/topic/topic.utils.test.ts | Adds coverage for the updated topic wire layout offsets. |
| foreign/node/src/wire/message/poll-messages.command.ts | Adds VSR consumer-group polling flow using SyncGroup and local cursoring. |
| foreign/node/src/wire/message/poll-messages.command.test.ts | Tests VSR consumer-group polling: assignment reuse, resync, and error paths. |
| foreign/node/src/wire/error.utils.ts | Introduces ResponseError with structured commandCode/errorCode. |
| foreign/node/src/wire/consumer-group/sync-group.command.ts | Adds SyncGroup wire command and assignment deserialization. |
| foreign/node/src/wire/consumer-group/sync-group.command.test.ts | Tests SyncGroup serialization/deserialization and truncation/trailing validation. |
| foreign/node/src/wire/consumer-group/index.ts | Exposes SyncGroup from the consumer-group API surface. |
| foreign/node/src/wire/consumer-group/group.utils.ts | Adds TargetGroup type to share stream/topic/group identifiers. |
| foreign/node/src/wire/command.code.ts | Extends command registry with SyncGroup and offset-v2 command codes. |
| foreign/node/src/wire/command-set.ts | Wires SyncGroup into API and keeps raw sendBinaryRequest guard behavior. |
| foreign/node/src/wire/command-set.test.ts | Updates tests for updated sendCommand signature and custom code forwarding. |
| foreign/node/src/e2e/test-client.utils.ts | Allows selecting protocol via env var for e2e runs. |
| foreign/node/src/e2e/tcp.raw.e2e.ts | Expands raw e2e tests for VSR vendor codes and request-id gap behavior. |
| foreign/node/src/e2e/tcp.consumer-stream.e2e.ts | Adjusts partitioning strategy for VSR vs classic to ensure routable partitions. |
| foreign/node/src/e2e/tcp.consumer-group.e2e.ts | Updates message sending strategy for VSR-compatible partition selection. |
| foreign/node/src/e2e/tcp.cluster.e2e.ts | Adds expected metadata variant for VSR path. |
| foreign/node/src/client/index.ts | Exposes client config normalization in the public client exports. |
| foreign/node/src/client/client.type.ts | Adds protocol/options types and config knobs for response frame size and send options. |
| foreign/node/src/client/client.ts | Normalizes client config and refactors pooled/single client providers. |
| foreign/node/src/client/client.socket.ts | Implements classic + VSR send pipeline, retries, leader redirection, and session lifecycle. |
| foreign/node/src/client/client.socket.test.ts | Adds extensive VSR client socket tests (transients, eviction, redirect, concurrency). |
| foreign/node/src/client/client.frame.ts | Adds protocol-aware frame extraction with max-size enforcement. |
| foreign/node/src/client/client.frame.test.ts | Tests frame extraction for classic/VSR fragmentation/coalescing and size limits. |
| foreign/node/src/client/client.connection.ts | Refactors connection lifecycle, adds redirect/abort, and uses new frame extraction. |
| foreign/node/src/client/client.connection.test.ts | Tests connection attempt sharing, frame buffering, and shutdown error suppression. |
| foreign/node/src/client/client.config.ts | Adds config normalization/validation for protocol and response frame limits. |
| foreign/node/src/client/client.config.test.ts | Tests config normalization (protocol defaults, VSR constraints, size validation). |
| foreign/node/src/bdd/raw.ts | Keeps BDD raw steps compatible with unchanged sendBinaryRequest signature. |
| foreign/node/scripts/check-vsr-protocol.mjs | Adds build-time parity check against Rust protocol sources (codes/ops/layout/version). |
| foreign/node/package.json | Bumps package version and adds protocol check + VSR e2e script + lint quoting fix. |
| foreign/node/README.md | Documents selecting VSR protocol, constraints, retries, and max frame size config knob. |
| codecov.yml | Splits Node coverage flagging by task (unit/e2e/e2e-vsr). |
| .github/workflows/_test.yml | Updates Codecov upload logic and flags to include new e2e-vsr task. |
| .github/config/components.yml | Adds e2e-vsr to sdk-node CI task list. |
| .github/actions/node-npm/pre-merge/action.yml | Adds VSR server start/stop and VSR e2e run into the pre-merge action. |
Files not reviewed (1)
- foreign/node/package-lock.json: Generated file
Comments suppressed due to low confidence (2)
foreign/node/src/wire/vsr/namespace.ts:1
validateFieldaccepts negative values and non-integers (e.g.,-1,1.5) and would allow them through topackNamespace, producing invalid namespaces (including negativebigintvalues). Since these fields are wire-level u32 identifiers, validation should reject values< 0and non-integers (and ideally also rejectNaN/Infinity) in addition to the upper bound check.
foreign/node/src/wire/vsr/namespace.ts:1namespaceFromConsumerOffsetalways throws errors tagged withCOMMAND_CODE.StoreOffset, even when called forDeleteConsumerOffset,StoreOffset2, orDeleteConsumerOffset2. This makes thrownResponseError.commandCodemisleading and complicates debugging. Consider passing the originating command code intonamespaceFromConsumerOffset(similar to other helpers here) and using it consistently inresponseError(...).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/ready |
|
/ready |
The Node SDK only supported classic binary framing, preventing it
from connecting to VSR servers.
Add runtime-selectable VSR framing with registration, session
recovery, request sequencing, bounded response decoding, transient
retries, leader redirection, and consumer-group support.
Keep the existing sendBinaryRequest(code, payload) API. Known commands
derive their VSR operation from the protocol registry. Codes unknown
to the SDK are sent as non-replicated requests with the command code
stored in RequestHeader.reserved, allowing the server to handle an
extension or return the authoritative error. Session-control commands
remain blocked before I/O.