Data tracks#975
Draft
pblazej wants to merge 29 commits into
Draft
Conversation
Replace remote livekit-uniffi-xcframework dependency with a local path to the Rust SDK's UniFFI package output, enabling iteration on data track bindings without publishing releases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire livekit-datatrack Rust managers into the Swift SDK's Room, SignalClient, and transport infrastructure. Data tracks provide frame-oriented, real-time data delivery with built-in DTP packetization and optional E2EE. Scaffolding (Phase 1): - Forward raw WebSocket bytes to Rust managers for signal routing - Create _data_track publisher/subscriber WebRTC data channels - Delegate bridges for signal requests, DTP packets, and track events - Manager lifecycle tied to Room connect/disconnect/reconnect - DataTrackDelegate protocol for track published/unpublished events Public API (Phase 2): - LocalParticipant.publishDataTrack(name:) and withDataTrack(name:body:) - LocalDataTrack.send(contentsOf:) for piping AsyncSequence to a track - AsyncPolling protocol with .values for DataTrackStream iteration - DataTrackFrame convenience extensions (.now, .latency) E2E Tests (Phase 3): - 8 tests mirroring Rust data_track_test.rs (publish/receive, large frames, duplicate name, unauthorized, state, timestamp, resubscribe, many tracks) - Test helper Room.waitForDataTrack(name:) for async track discovery Tests require livekit-server with enable_data_tracks and a tokio runtime fix in livekit-uniffi (pending). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Forward serialized protobuf bytes after parsing (not raw WebSocket bytes) so JSON-encoded messages from the server are also forwarded to Rust data track managers - Register DataTrackWatcher before publishing to avoid missing the initial ParticipantUpdate event - Use AsyncStream-based watcher for reliable async track discovery - Simplify resubscribe test to 2 iterations with delay Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Upstream livekit-uniffi replaced the generic `handleSignalResponse` with specific per-message handlers: - handleSfuRequestResponse (for RequestResponse) - handleSfuPublishResponse (for PublishDataTrackResponse) - handleSfuParticipantUpdate (for ParticipantUpdate) - handleSubscriberHandles (for DataTrackSubscriberHandles) Each handler returns UnsupportedType for messages it doesn't handle, so the simplest integration is to call all four with the raw bytes and let them filter internally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 0.31.2 bindings give PushFrameErrorReason cases an associated `message: String` and conform it to Swift.Error, so `catch PushFrameErrorReason.QueueFull` no longer matches. Catch the error and pattern-match the case, rethrowing other variants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pblazej
force-pushed
the
blaze/datatracks-integration
branch
from
cb7f186 to
6c21122
Compare
|
|
- Collapse the two delegate bridges into a single DataTrackBridge that conforms to both manager delegate protocols (shared onSignalRequest, one weak-room instance for both). - Fold the data track channel/manager teardown into a single cleanUpDataTrack(). - Move the reconnect republish/resubscribe calls into the quick/full reconnect sequences so each sequence is self-contained and the retry loop stays clean. - Nest FrameDropPolicy under LocalDataTrack and drop the redundant AsyncPolling Element typealias (inferred from next()). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dd send backpressure - E2EE: bridge the UniFFI encryption/decryption providers to the existing E2EEManager via internal DataTrackEncryptionProvider/DataTrackDecryptionProvider adapters (the public E2EEManager can't adopt an internally-imported protocol directly). They reuse E2EEManager's AES-GCM data path (LKRTCDataPacketCryptor over the shared BaseKeyProvider); providers are passed only when E2EE is configured so plaintext tracks stay unmarked. Exercised end-to-end by the existing DataTrackTests, which run with E2EE on by default. - Attach remote data tracks to their RemoteParticipant (keyed by SID) so they can be enumerated, mirroring media tracks and the JS SDK. - Drop a whole frame when the publisher data track channel is congested instead of sending unconditionally, bounding the channel buffer (parity with the lossy data channel threshold). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…op latency - Merge the separate encryption/decryption providers into one DataTrackCryptor that conforms to both UniFFI protocols (mirrors the JS DataCryptor); one instance serves both managers. - Encapsulate reconnect restoration on the managers via handleReconnect(fullReconnect:) so the reconnect sequences just notify each manager with the mode. - Drop the test-only DataTrackFrame.latency helper (not exposed by Rust or JS); inline the recency check in the test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sState Group the data track managers and publisher/subscriber channels into a DataTracksState struct behind a StateSync. They were plain vars on the @unchecked Sendable Room, read from the Rust callback threads (packet send/receive) while connect/cleanup mutated them — a data race. The StateSync synchronizes access and lets cleanUpDataTrack reset everything atomically. Read sites keep working through get-only computed accessors; only the writes move to mutate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wrap the UniFFI data track types in native, documented, Objective-C-capable LiveKit types (LocalDataTrack, RemoteDataTrack, DataTrackStream, DataTrackFrame, DataTrackInfo, and public error enums), mirroring the JS SDK's public-class-over-internal-core design. This keeps LiveKitUniFFI internal (no token/access-token symbols leak) while giving the data track API a clean public surface. - LocalParticipant.publishDataTrack/withDataTrack/queryDataTracks and the track/stream/frame operations are now public, with doc comments kept close to the JS/Rust wording. - Fold the data track callbacks into the public RoomDelegate as @objc optional methods (the wrappers are NSObject-based, so this is now possible) and drop the separate internal DataTrackDelegate and dataTrackDelegates multicast. - RemoteParticipant.dataTracks is public; the bridge wraps UniFFI tracks before delivering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add DataTrackObjCTests exercising the public API end-to-end from Objective-C: publish, push frames, query, and unpublish on the local side, plus subscribe and receive frames via the callback-based reader on the remote side. Proves the wrappers bridge to Objective-C through the auto-generated completionHandler entry points. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an internal FFIBridged marker protocol (associated FFIType + init(_:)) in Support and conform each data track wrapper to it in its own file. Documents the FFI bridge boundary; not part of the public API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ck thread - Route outbound signal requests through an AsyncSerialDelegate so they reach the SFU in the order the manager emits them, instead of a bare Task per callback that could reorder a publish/unpublish pair (matches how SignalClient delivers its own callbacks). - Send data track packets with DispatchQueue.liveKitWebRTC.async instead of sync, so the Rust callback thread isn't blocked on the WebRTC queue. The queue is serial, so frame order is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eError Match the JS SDK's error name. Also leave a TODO on RemoteDataTrack.subscribe() to expose subscription options once the UniFFI layer supports subscribe_with_options. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…he SID Match the media track delegate convention: room(_:participant:didPublishDataTrack:) and room(_:participant:didUnpublishDataTrack:) now carry the RemoteParticipant, and the SID is a typed DataTrack.Sid (a namespaced String alias, mirroring Track.Sid). RemoteParticipant.dataTracks is keyed by DataTrack.Sid. Feeding the remote data track manager moves from didReceiveRawResponse to didUpdateParticipants (after the participant is added), so onTrackPublished can always resolve the publisher — the raw path ran before participants existed, which would drop the event. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Merge publishAndReceive and publishLargeFrames into one parameterized test over payload size / frame count. - Replace the loose catch blocks with #expect(throws: DataTrackPublishError.self) for the duplicate-name and unauthorized cases. - Use try #require over guard + Issue.record for stream.next() results. - Assert the received track is E2EE-encrypted (withRooms enables it by default). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…npublish delegate Add tests for the public API the suite didn't exercise: LocalDataTrack.send(contentsOf:), RemoteParticipant.dataTracks attachment, and the room(_:participant:didUnpublishDataTrack:) delegate (DataTrackWatcher gains unpublish observation). queryDataTracks is left to the ObjC test — queryTracks() returns nothing in a plain publish-then-query Swift scenario (despite isPublished being true), so a reliable Swift test isn't possible without the longer round-trip the ObjC test happens to perform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ss test Publish several tracks, then push every frame on every track at once, exercising the Rust manager's per-track send queues and the bridge's packet dispatch under contention. Parameterized over a many-small-frames and a large-multi-packet-frames scenario. Each frame is tagged with (trackIndex, sequence); the unreliable channel may drop frames, but whatever arrives must reach the right track with no misrouting, duplicates, or corruption. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Asserts queryDataTracks returns the local participant's confirmed publications. Holds the returned LocalDataTrack across the query — the caller owns the publication lifetime, so dropping it unpublishes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dinator Collapse the Room's data-track surface — four computed accessors, a DataTracksState struct, and a lazy channel delegate — to a single `DataTracks` reference. It owns the local/remote managers, the data channels, and the manager-delegate shim, and routes Room/participant calls to the right manager, keeping the subsystem off the god-object's surface. Created in configureTransports so its lifecycle brackets the transport lifecycle (cleanUpRTC tears it down) across connects and reconnects, and so it takes ownership of the publisher data track channel as that channel is created. The subscriber channel is retained too — its Swift wrapper must outlive the call for native delegate callbacks to reach us. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously DataTracks was created in configureTransports and torn down in cleanUpRTC, so a full reconnect rebuilt the managers from scratch; republishTracks() then ran on an empty manager and locally-published tracks were lost. The reference SDKs (Rust/JS) keep the manager for the whole session and only re-establish transports. Create DataTracks once at connect and tear it down only on a real disconnect — cleanUpDataTracks(isFullReconnect:) skips the cleanup during a full reconnect — so the managers persist and republish their publications. configureTransports now just hands the new publisher channel to the existing subsystem. Adds republishesTrackAfterFullReconnect, which forces a full reconnect and asserts the publication survives. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integrates the finalized UniFFI methods from rust-sdks PR #1034: - handleSfuJoinResponse: fed from the join handler so a participant sees data tracks already published by others when it joins. Verified e2e — the earlier "join carries no data tracks" finding was a test discarding the returned track (which unpublishes it), not a server gap. - publishResponsesForSyncState: wired into SyncState.publishDataTracks so a quick reconnect preserves local publications without a full republish. query_tracks() is now internal in the FFI, so the public queryDataTracks() is removed along with its Swift/ObjC coverage. Tests: bake roomName/identity into RoomTestingOptions to stage a late joiner into an existing room; add receivesTrackPublishedBeforeJoin, and rework the reconnect test to verify republish end-to-end (subscriber re-sees the track) now that queryDataTracks is gone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ndle The Rust LocalDataTrack unpublishes on drop (RAII), so discarding the returned handle silently tore the publication down. JS instead keeps publications in its manager until an explicit unpublish. Match JS: after publishing, spawn a task that awaits the track's unpublish, keeping it alive until an explicit/SFU unpublish or teardown. The wait does not fire during a reconnect's republish, so session-scoped republish still works. Adds retainsPublicationWhenHandleDropped and drops the now-unnecessary keep-alive workarounds from the join and reconnect tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the TODO with dataTrackSurvivesQuickReconnect: publish, subscribe, force a quick reconnect (nextReconnectMode: .quick, which runs sendSyncState with publishDataTracks), and assert frames keep flowing on the same stream. The earlier flakiness was an unbounded stream read that hung on any hiccup; this uses a bounded read (15s) plus a background pusher, so a broken publication fails cleanly instead of hanging. Reliable across repeated runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire RemoteDataTrack.subscribe(bufferSize:) to the new UniFFI subscribe_with_options, letting callers tune the internal receive buffer (default 16). ObjC gets subscribeWithBufferSize:completionHandler:. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ant delegates Bring data track publish/unpublish notifications to parity with media tracks: dual-notify (participant then room) for both local and remote, on publish and unpublish. Adds the local variants to RoomDelegate and all four data-track methods to ParticipantDelegate (which had none). Local unpublish rides the existing retention task, so it fires once when the publication really ends. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add DataTrackDelegateRecorder (RoomDelegate + ParticipantDelegate) and a test asserting publish and unpublish each fire on both delegates for the local publisher and the remote subscriber. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ects On disconnect the participant is removed from the room before the async manager callback arrives, so onTrackUnpublished can't route the event and no unpublish fires. Clean up data tracks in RemoteParticipant.unpublishAll — where media tracks are already handled — so didUnpublishDataTrack fires on both delegates, using the still-alive participant. Add an e2e test covering disconnect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.