feat(VideoDecoderIssueDetector): measure local decode demand, not fps volatility - #49
Open
jenspalmqvist wants to merge 4 commits into
Open
Conversation
… volatility decoder-cpu-throttling fired on every receiver of a sender that lowered its frame rate, because the detector only looked at inbound fps volatility. It now requires, per peer connection, a local frame shortfall (received frames not decoded) on more than affectedStreamsPercentThreshold of the inbound streams over a window of at least minWindowMs, a summed decode demand (totalDecodeTime per decoded frame times received frames per second) above decodeDemandThreshold, and at least one affected stream whose own demand is above affectedStreamDemandThreshold. Streams with packet loss above maxPacketLossPct, with mean RTP jitter above maxJitterMs, decoded in hardware (powerEfficientDecoder), without the required counters, with no decoded frames, or with a counter reset inside the window are excluded. framesDropped is no longer read. Browsers drop a frame before decode only when a later frame is decodable without it, so the detector covers streams with temporal layers (simulcast, SVC) and is silent on single-layer streams, which decode every frame late instead of dropping. statsSample adds decodeDemand, frameShortfallPct, windowMs, evaluatedStreams and throttledStreams. affectedStreamsPercent keeps its meaning (share of inbound streams that show the problem) and is now rounded to three decimals. throtthedStreams stays, deprecated, and still lists only the affected streams; its entries keep allFps and volatility, deprecated, next to the new numbers; allFps holds one value per stored sample (4 by default) instead of exactly 5. The earliest emission moves from the sixth to the fifth poll. affectedStreamsPercentThreshold keeps working. volatilityThreshold is accepted and ignored; a high value no longer silences the detector. New params: decodeDemandThreshold, affectedStreamDemandThreshold, frameShortfallPctThreshold, minFramesReceived, minWindowMs, maxPacketLossPct, maxJitterMs. The detector now needs id, timestamp, framesReceived, framesDecoded, totalDecodeTime, packetsReceived, packetsLost and jitter in the inbound video stats and is silent without them. ParsedInboundVideoStreamStats gains an optional powerEfficientDecoder field. The ExecPlan in docs/plans/EXECPLAN_DECODER_CPU_THROTTLING_SIGNAL.md records the design, six review rounds, and the test evidence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1KrAS941bzfk9EuBPG33k
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1KrAS941bzfk9EuBPG33k
… stack Seven runs of this branch's build inside a consumer web app's development container, against its signalling backend and a host mediasoup SFU with simulcast. The sender-cap run reproduces the old false positive's input (a 3 to 9 fps wobble on every receiver) and no receiver reports. A stats shim on one receiver reports alone, on Chrome and on Firefox. Leave and rejoin resets the history. Machine-wide CPU overload fires the other detectors and not this one. Two follow-ups recorded: mediasoup's probe stream (ssrc 1234) is counted in the affectedStreamsPercent denominator, and Firefox reports no powerEfficientDecoder, so the hardware-decode skip never applies there. No code changed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1KrAS941bzfk9EuBPG33k
…irefox Firefox exposes every inbound-rtp field VideoDecoderIssueDetector reads but not powerEfficientDecoder, so a Firefox receiver that decodes in hardware is never excluded and can reach the demand thresholds on pipeline latency alone. Observed with Firefox 146 in a local call on 2026-09-04. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1KrAS941bzfk9EuBPG33k
jenspalmqvist
force-pushed
the
execplan-decoder-cpu-throttling-signal
branch
from
September 5, 2026 09:06
af6c962 to
90c00a0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #48
Purpose
VideoDecoderIssueDetectoremitsdecoder-cpu-throttlingwhen the device that runs the library cannot decode the incoming video fast enough. Today it measures only how much the frame rate of each inbound stream wobbles over the last five samples. A remote participant whose encoder gives up frame rate because of a bad uplink or a slow CPU produces exactly that wobble on every receiver, so every receiver reportsdecoder-cpu-throttlingabout its own device. With three publishers, one wobbling sender is 33 percent of the inbound streams, above the default 30 percent threshold.After this change the issue fires only when, per peer connection, frames that arrived on this device were not decoded, enough inbound streams show that shortfall, the incoming video together demands a large share of decoder time, and at least one affected stream is itself costly to decode. Streams with packet loss, high RTP jitter, hardware decoding, missing counters, no decoded frames, or a counter reset in the window are excluded. A sender that sends fewer frames no longer triggers the issue.
What changed
src/detectors/VideoDecoderIssueDetector.ts: volatility signal replaced by decode demand (totalDecodeTimeper decoded frame times received frames per second), frame shortfall (framesReceived - framesDecoded), an affected-streams gate, and a per-stream corroboration. New paramsdecodeDemandThreshold,affectedStreamDemandThreshold,frameShortfallPctThreshold,minFramesReceived,minWindowMs,maxPacketLossPct,maxJitterMs.affectedStreamsPercentThresholdkeeps working.volatilityThresholdis accepted and ignored (deprecated).decodeDemand,frameShortfallPct,windowMs,evaluatedStreams,throttledStreams. KeepsaffectedStreamsPercentand the misspelledthrotthedStreams(deprecated, same array asthrottledStreams, still only the affected streams), withallFpsandvolatilitykept per entry as deprecated informational fields.src/types.ts: optionalpowerEfficientDecoderonParsedInboundVideoStreamStats.test/detectors/VideoDecoderIssueDetector.spec.ts: first spec for this detector, 22 tests with a cumulative-counter fixture builder. The testdoes not report a sender whose frame rate wobblesfails on the current code and passes here.README.md: section rewritten; the old example showed a field the code never emitted.docs/plans/EXECPLAN_DECODER_CPU_THROTTLING_SIGNAL.md: the design record, six review rounds, decision log, and test evidence.Known limitation
Browsers drop a frame before decode only when a later frame is decodable without it, which needs temporal layers (simulcast or SVC). On a single-layer stream (peer-to-peer VP8 without simulcast, H.264) an overloaded receiver decodes every frame late and shows no shortfall, so this detector stays silent there. The README says so.
Compatibility notes (minor release)
Earliest emission moves from the sixth to the fifth poll.
affectedStreamsPercentis rounded to three decimals.allFpsholds one value per stored sample (4 by default) instead of exactly 5. A very highvolatilityThresholdno longer silences the detector. The detector needsid,timestamp,framesReceived,framesDecoded,totalDecodeTime,packetsReceived,packetsLost, andjitterin inbound video stats and is silent without them.Verification
npm teston Node 24: 58 passing, 0 failing (36 existing plus 22 new), run on this exact commit.npm run lintandnpm run lint:tests: clean on this exact commit.[30, 30, 15, 30, 15], volatility 30, 33.3 percent affected.🤖 Generated with Claude Code
https://claude.ai/code/session_01V1KrAS941bzfk9EuBPG33k