feat(mediaplayer): add RIST live streaming with loss recovery + encryption#853
Open
towneh wants to merge 4 commits into
Open
feat(mediaplayer): add RIST live streaming with loss recovery + encryption#853towneh wants to merge 4 commits into
towneh wants to merge 4 commits into
Conversation
Add RIST (Reliable Internet Stream Transport) as a receive-only live transport, behind a BASIS_WITH_RIST CMake option that is OFF by default so existing builds are unaffected and link no new dependencies. librist (Main Profile) owns the UDP sockets, ARQ recovery, jitter buffer and optional PSK-AES; basis_rist.c buffers the recovered MPEG-TS through a lock-guarded ring and exposes it as a basis_read_fn, so basis_ts_run demuxes RIST exactly like the existing TCP/HTTP byte sources and rides the same reconnect loop. basis_url gains the rist:// scheme and captures the query (secret/aes-type/buffer) that librist parses. The librist static is built separately (see the build commit); third_party/ holds only .gitkeep scaffolding and a build README.
Admit rist:// through the existing BasisMediaPlayerSecurity allowlist (boundary validation, no new code path), and fold an optional Options["buffer"] (milliseconds) into the URL query so librist sees the receive-buffer depth. Non-RIST schemes are untouched.
librist is built from source rather than committed: build-librist.{ps1,sh} clone the pinned tag (v0.2.11) and build a single static (its bundled mbedTLS linked in) into third_party/<rid>/, run by CI and locally alike. media-native.yml builds the static per platform (Windows x64, Android arm64) and uploads it as an artifact. The statics stay gitignored.
Note the optional RIST build and librist's BSD-2-Clause licensing (and its vendored mbedTLS) in the package README and third-party notices.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds RIST (Reliable Internet Stream Transport) as a receive-only live-ingest transport for the media player — broadcast-grade loss recovery (ARQ) plus optional content encryption (Main-Profile PSK-AES) for live shows. This lands the prototype discussed in #848.
It's additive and low-blast-radius:
BASIS_WITH_RISTCMake option that is off by default, so existing builds are byte-for-byte unchanged and link no new dependencies.basis_ts_rundemuxer, the same OS decode / zero-copy texture path, and the same exponential-backoff reconnect loop. No new framework surface, no new events.rist://is admitted through the existingBasisMediaPlayerSecurityallowlist (validated at the boundary), and an optionalOptions["buffer"](ms) is folded into the URL query that librist parses.librist itself is a third-party native dependency built from source in CI (
v0.2.11, its bundled mbedTLS linked into a single static), not committed — the shipping answer to the "how should the first native dep be shipped" question from #848.Commits (logical, reviewable):
feat(mediaplayer): RIST live-ingest transport (native + CMake)— the receiver module (basis_rist.c/.h),rist://scheme + query handling inbasis_url, therun_ristdispatch, and the CMake wiring.feat(mediaplayer): accept rist:// URLs in the player— the security allowlist entry and theOptions["buffer"]→ query fold.build(mediaplayer): build librist from source for RIST—build-librist.{ps1,sh}+ themedia-native.ymlworkflow.docs(mediaplayer): record librist in README and THIRD_PARTY_NOTICES— BSD-2-Clause licensing + the optional-build note.Design detail (technical + business specs) is linked from #848.
basis_rist_gh.mp4
Demo: AES-128
rist://stream decoding live in the editor — clip attached on #848Required checks
All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.
TransformAccessArrayor are otherwise batched. I have not added per-frametransform.position/transform.rotation/transform.localPositioncalls inside loops. Whenever I need both position and rotation, I use the combined APIs —SetPositionAndRotation/SetLocalPositionAndRotationfor writes,GetPositionAndRotation/GetLocalPositionAndRotationfor reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.Resources.Load, no direct asset references that pull large content into memory on scene load.GetComponent/AddComponentwhere avoidable — Where unavoidable, the result is cached on a field, and anyGetComponent<T>is replaced withTryGetComponent<T>(out var x)— bareGetComponentwill be denied.TryGetComponentis the modern API (Unity 2019.2+) and skips the Editor-only GC allocationGetComponentcauses when a component is missing: Unity wraps thenullreturn in a managed "fake null" object so its overloaded==operator can still detect destroyed C++ objects, and constructing that wrapper allocates;TryGetComponentreturns aboolplusoutparameter and never builds the wrapper. None of these calls run insideUpdate,LateUpdate,FixedUpdate, jobs, or other per-frame code paths.BasisEventDriver— Any new per-frame work hooks intoBasisEventDriverrather than adding standaloneUpdate/LateUpdate/FixedUpdatecallbacks on a MonoBehaviour.BasisEventDriveris bulletproof, or guarded bytry/catch—BasisEventDriverruns the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in atry/catchthat contains the failure and surfaces it throughBasisDebug— logged once / rate-limited, never every frame (see the existingHVRBasisBuiltInAddresses.Simulate()guard for the pattern). Expect this to be scrutinized closely in review.{ get; set; }properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things offprivate/internalwithout a real reason. Don't wrap a field in{ get; set; }when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For.Instancesingletons, callers reassigningType.Instanceis allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.BasisLocalCameraDriver— Code that needs the local camera (transform, projection, rig data, etc.) pulls it fromBasisLocalCameraDriverrather than looking one up itself. Don't roll a separate camera discovery path.BasisDebug— All new logging calls go throughBasisDebug.Log/BasisDebug.LogWarning/BasisDebug.LogError(with an appropriateLogTag) instead ofUnityEngine.Debug.Log/Debug.LogWarning/Debug.LogError.BasisDebugroutes through Basis's tagged, color-coded logger and respects the project-wideLoggingDisabledtoggle so logging can be killed at runtime; bareDebug.Logcalls bypass that and will be denied.FindObjectOfType/FindObjectsOfType/GameObject.Find/FindGameObjectsWithTagto locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.newon reference types, no LINQ, nostringconcatenation/interpolation, no boxing, noforeachover interface-typed collections. Allocate once at init and reuse the buffer.BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind#if UNITY_EDITORand remove (or leave gated) before merge..Count(lists) /.Length(arrays) into a localintbefore the loop instead of re-reading the property each iteration. PreferT[](with a separate length int when the array is over-sized) overList<T>where the data is hot — Unity's mono BCL doesn't exposeCollectionsMarshal.AsSpan(List<T>), so a list can't be fed intoSpan<T>/ unsafe paths cleanly. Where the perf justifies it, drop intoSpan<T>/reflocals /Unsafe.As/unsafepointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.Testing details
Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.
Input / control mode coverage:
Where applicable, confirm these flows still work after your changes:
Notes
What was tested. Validated end-to-end in the Unity editor on Windows against a live librist broadcaster over the public internet, on an optimized Release build of the plugin (the shape that ships, no debug info):
rist://host:portstream and a PSK-AES-128rist://host:port?secret=…&aes-type=128stream play the full pipeline — RIST receive → (decrypt) → MPEG-TS demux → Media Foundation H.264 decode → zero-copy texture, with audio.Android (Quest) is the natural next platform but is not device-tested yet, so it's left unticked. The librist Android static already builds in CI and the demux/decode path is shared, so it should be mostly an on-device build + run; I'd rather leave it honest than fake-green it.
Why most required checks are N/A. This change is a native-C transport plus a small init-path C# change — it adds no per-frame/Unity-runtime work, so most of the perf checklist doesn't apply: no transforms, no Addressables/asset loads, no
GetComponent/AddComponent, nothing added toBasisEventDriver, no camera access, no scene-wide discovery, no new C# logging (native failures surface through the existingsink->on_error→ player error path). The two that genuinely bite, and hold:basis_rist_openand reuses it;basis_rist_readdoes no per-read allocation. The C#Options["buffer"]fold runs once perLoadUrl, not per frame.Default builds are unaffected. With
BASIS_WITH_RISToff (the default),basis_rist.ccompiles only a graceful "not built" stub and the plugin links exactly as today — no librist, no new system libs.