perf(stream): submit fresh frames on arrival; repeat only when stale - #114
Closed
szdziedzic wants to merge 1 commit into
Closed
szdziedzic wants to merge 1 commit into
szdziedzic wants to merge 1 commit into
Conversation
The WebRTC publisher held every fresh capture for the next slot of its 60 Hz pump. That wait is 0–16.7 ms per frame and measured 10–15 ms at p50 tap-to-frame with the latency probe: p50 32–37 ms on 0.1.51 versus 22 ms with frames sent on arrival (M5 Pro, loopback, production flags). The pump was introduced in #92 so the encoder would not starve on an idle screen; it does not need to gate fresh pixels to do that. `ContinuousFramePacer` now treats an arrival as a submission: the frame goes out immediately and only the repeat chain is scheduled around it. The chain re-sends the retained frame whenever nothing fresh went out for one interval, so an idle or slowly changing screen still streams at the configured `--video-fps` and the encoder, bandwidth estimate and receiver jitter buffer keep a steady cadence. The chain keeps every hardening from #92: strict zero-leeway timers, grid-anchored slots so late wakes cost phase not rate, stall re-anchoring without a catch-up burst, and the lost-pump watchdog that lets an arrival restart a starved chain under a fresh generation. `/webrtc/stats` gains `arrivalFrames` and `repeatFrames` so the split is visible in production; the stats panel shows both. The pacer unit tests are rewritten for the new contract (arrivals never held, sustained 60 Hz arrivals produce zero repeats, repeats fill the cadence once arrivals stop, late ticks hold the rate, stalls do not burst, lost pumps are reclaimed). The cadence e2e test still passes: forwarded stays ~60/s on an idle simulator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The WebRTC publisher now submits every fresh capture to libwebrtc the moment it arrives. The timer chain becomes a repeat-only fallback: it re-sends the retained frame when nothing fresh went out for one interval.
/webrtc/statsgainsarrivalFramesandrepeatFrames, and the stats panel shows both.Why
The pump from #92 held each fresh frame for its next 60 Hz slot. That wait is 0–16.7 ms per frame and measured 10–15 ms at p50 tap-to-frame with the probe: p50 32–37 ms on 0.1.51 versus 22–25 ms with frames sent on arrival. The pump exists so the encoder does not starve on an idle screen; it does not need to gate fresh pixels to do that.
How
ContinuousFramePacertreats an arrival as a submission.latestFrameArrivednow returns.send,.sendAndSchedule(delay)or.sendAndRestart(delay); the frame always goes out, the decision only says what to do about the repeat chain.tickrepeats the retained frame when the grid slot is due and at least one interval has passed since the last submission; otherwise it returns.waitto the moment a repeat becomes due. Every hardening from fix(stream): hold the WebRTC frame-pump cadence on virtualized hosts #92 stays: strict zero-leeway timers, grid-anchored slots (late wakes cost phase, not rate), stall re-anchoring without a catch-up burst, and the lost-pump watchdog that lets an arrival restart a starved chain under a fresh generation.WebRTCPublisher.sendFramesubmits on the publisher queue, so repeats and arrivals serialize and a repeat can never overtake the fresher frame it would duplicate.Part of the tap-to-photon stack (bottom → top): latency probe → send frames on arrival → 240 Hz seed poll → bitrate floor → playout window → data-channel input → lossy moves lane. Everything is on by default, so a package built from any layer works with the unchanged EAS launch flags (
--transport webrtc --webrtc-codec vp8 --max-dimension 960 --video-bitrate 6000000 --video-fps 60).Measured with
scripts/latency-probe(M5 Pro, loopback, iPhone 17 / iOS 26.5, 25 taps): 0.1.51 p50 32–37 ms / p95 ≈ 43 ms → top of stack p50 27 ms / p95 38 ms, no outliers above 40 ms.🤖 Generated with Claude Code