6.44.0 - A constant rate is not always a whole number of ticks

Drop-in from 6.43.0. No API change. One behaviour change: the #409 composition-offset repair now also covers sources whose frame duration is not a whole number of ticks, which is the class its own reporter's asset turned out to belong to.

A constant rate that no step describes

6.41.0 repaired MP4s that dropped the composition-offset table while the H.264 bitstream still reorders pictures. @orut34iop retested it on the original reporting asset and it still juddered from the first frame, with the measurement to say why: the file's sample table alternates between 40040 and 40041 ticks in a five-picture cycle, and the classifier demanded one identical step. It read that as variable frame timing and fell closed, so nothing was repaired.

The diagnosis is right, and it points at an assumption worth naming: a constant frame rate does not always produce a ladder that advances by one constant. At a 1200000 timescale that asset's pictures are 200202/5 ticks apart. No integer describes it, so the sample table has no choice but to alternate between the two neighbouring counts, and a repair built on one step has nothing to stand on.

Reproduced here before anything was changed, on a twin pair built to that shape (one encode at 1000000/33367 fps in a 1200000 timescale, muxed twice, the composition offsets stream-copied away from the second): the shipped implementation passes all 33 packets through with PTS == DTS, every picture still in decode order.

A two-valued ladder is now read as the quantization it is:

• The cycle it repeats names the fraction. The smallest period the sampled deltas repeat under gives the cadence exactly, and a cycle only counts once it has been seen through twice, so a short run of jitter cannot pose as one.
• The pattern it rounds to names the phase of the lattice it was quantized from. This is the part a uniform ladder cannot carry, and it is not implied by anything else: the reporting asset sits on phase 2 while its reorder delay is 1. Exactly one phase of the period may reproduce the sampled window picture for picture; if more than one does, or none, nothing is repaired.
• Ranks are then placed on that lattice instead of on anchor + rank * step, so the repair reproduces what the muxer would have written rather than landing a tick beside it. A whole-tick ladder takes the path it always took.

Reading the phase has a second effect. Measured on the same file, a session that sampled after a seek instead of at the head classified it as unrepairable, because the ladder starts on a different phase there. That verdict is now independent of where the sample was taken, which matters for any consumer that opens at a resume position.

Three things deliberately did not change:

• How far the ladder runs ahead of presentation still comes from the container header. Every alignment fits the ladder equally well, so the ladder cannot answer that question; presentationShift reads it from start_time against the first index entry and clamps it to one reorder head, now resolved to a whole number of pictures.
• The container index is still folded by one constant, the same one the packets move by, so an index entry cannot disagree with the packet it points at and none can go missing from a segment plan.
• A sequence the lattice cannot place falls back to the rounded step, anchored on the container's own timestamp, so it cannot drift and nothing is ever handed on in decode order because an arithmetic path did not close.

Verified against the fractional twin pair (33 packets, three coded video sequences, both writer shapes, from the head and after a seek): every repaired packet carries the healthy twin's PTS and DTS exactly. The whole-tick pair from the first round still matches exactly, unchanged. Genuine variable frame timing, a ladder with a dropped picture, and a wobble that never repeats are all still left as the container delivered them.

The verdict line names what it measured, so a report can be read off the log:

#409 missing H.264 composition offsets confirmed on stream 0: repair step=40040 lead=80081
shift=80081 pocStep=2 cadence=200202/5 phase=3 ladderAhead=2 samples=12

Reported, retested and diagnosed by @orut34iop.

---

Commits: 69daa3d3 (repair), 14a079ce (verdict line). Full CHANGELOG, diff 6.43.0...6.44.0.

6.43.0 - A clock has to describe the picture it is over

Drop-in from 6.42.0. No API change. Four behaviour changes, all on the native loopback-HLS path, and the first of them repairs something 6.40.0 introduced.

A clock has to describe the picture it is over

@rrgomes reported captions running about four seconds ahead of the dialogue after a resume, further out after later seeks, on an asset whose Matroska Cues sit on non-random-access points. Lip sync was fine, because audio and video ride in the same segment; the clock every consumer reads was not.

6.40.0 gave the producer a gate that re-aims BELOW a plan boundary that turns out to carry no sync sample, so the segment covers its own advertised start. It shipped with the assumption that a segment keeping its own timestamps leaves the item axis where the plan puts it, and therefore published no shift for that case. The assumption was not testable at the time: every axis observable in the engine describes what it WROTE (MuxedVideoFrameTime, prodShift / hostShift), and none of them says where AVPlayer then PUT it.

aetherctl play --picture-probe now reads that out. It attaches an AVPlayerItemVideoOutput to the running item and decodes the source time from the frame itself, against a fixture whose picture states its own frame number in binary (Scripts/timecode-fixture.sh). The reading:

AVPlayer presents a segment at the position the playlist gives it, not at the tfdt the segment carries, and then plays continuously from there.

So a gate that opened below its boundary shifts the whole run by the re-aim. Two consequences, and the fix needs both. The offset a consumer folds is measured against the segment's ADVERTISED start, never against where the gate actually opened; a pinned (late) gate makes those identical, which is why publishing the muxer's shift held until a gate that opens early existed. And that offset belongs to the decode RUN rather than to the timeline: a seek that leaves the loaded region without provoking a restart begins a fresh run on an axis-true segment, where the previous offset must stop applying. Publishing only the first half mirrors the defect instead of fixing it.

Measured on a fixture carrying the reporting shape (Cues at non-sync positions, 12 s drought at 43 s), resuming at 53 s so the gate re-aims to 38.417, source time decoded from the picture:

arm	published shift	picture vs item axis	picture vs sourceTime
6.39.0 (pinned, late gate)	+3.000	+3.000	+0.075
6.40.0 to 6.42.0	0.000	-13.583	-13.550
6.43.0	-13.583	-13.583	-0.009

and +0.892 to +0.017 for a seek that leaves the loaded region. The control fixture, identical except that its Cues ARE its sync samples, is untouched: no re-aim, no seam, axisErr 0 on every tick. The muxer's own shift is unchanged, so no landing moves.

One thing to expect rather than to report: after a resume or seek into a drought the clock now reads LOWER than the target, by the re-aim. That is not a new landing error. The picture was already there; it is being described honestly for the first time.

A re-aim must not step over the sample that covers its boundary

Found while measuring the above, and only visible once the clock stopped hiding it. Each re-aim attempt opens on the first sync sample at or above where it aimed, and everything above the previous aim is already proven empty, so the DISTANCE between two attempts is the worst case by which the gate can overshoot the last covering sample. The backoff doubled, which spends that error where it is largest.

	aims	gate opens	published shift
before	48.0, 44.0, 36.0	38.417	-13.583
after	48.0, 44.0, 40.0	43.000	-9.000

Three attempts either way; the 8 to 16 jump flew over the 43.0 between it and the boundary at 52.0. seektest settles the same improvement from the seek side: settleError 3.80 s against 8.38 s, same burst and same throttle. Even steps cost no more to walk, because gateProvenEmptyFromPts stops each scan at the previous aim rather than at the boundary. Reach is unchanged at 32 s.

A wedge on a stored segment is the consumer's

Also from @rrgomes: landings of 11.94 s (Apple TV) and 15.75 s (Mac) after a restart, spent as 5 s of park detection plus 6 s of re-engage grace, while the producer had already served its first segment about two seconds in.

The wedge is an AVPlayer state (#65 / #93: zero GETs while the item never fails) and does not reproduce on the loopback path. What both logs settle is that the first repair could not have worked: the pump had marched to segment 15 and was sent back to segment 3, the consumer fetched nothing for the entire grace window, and the nudge that followed landed the seek in 240 ms.

A producer re-anchor is the repair for a consumer STARVED of content nobody is producing. A consumer silent about a segment that is already on disk is not starved, and re-anchoring throws the pump's forward work away to rebuild what it already has. The repair is now chosen on that question, with the other kept as the fallback inside the same grace window. WEDGE BROKEN carries consumerTargetStored= and highStored= so a report can say which repair a wedge called for. The 5 s park detection is deliberately unchanged.

Recovery paths must not read AVPlayer on the main actor

@rrgomes measured AVPlayerItem.currentTime() from a host's main actor not returning for 13.3 s during that wedge, with the app blocked throughout; it came back 30 ms after the re-engage watchdog fired.

These getters are synchronous XPC round trips to mediaserverd. AVFoundationOffMain has carried that warning since #134, including what it costs past the watchdog threshold, but only the 30 s memory probe had adopted it. The paths that run precisely BECAUSE the server is not answering had not: the seek-deadline loop took four round trips per pass (one island, three bufferedEnd), the stall nudge and item reload read currentTime() twice per call, the VOD shift-publish line read a buffer figure for a log, and the #287 premature-end recovery read three witnesses separately. All four now read off-main or from the rendered-position mirror.

For the recovery anchors the mirror is also the correct VALUE rather than merely the cheap one: recoveryAnchorPosition(currentRendered:) exists to keep the anchor off a frame the viewer has already passed (#115), and currentTime() is the clock, which diverges from the rendered frame during exactly the landing those paths run in (#123). The wedge path was already passing the mirror. Reads inside load() and the seek completion stay synchronous: both run where AVPlayer has just answered.

Harness

play --picture-probe and Scripts/timecode-fixture.sh are the instrument the first item needed: what SOURCE frame is AVPlayer presenting at a given item time. Per tick it prints pic, picItem, axisErr and capErr. docs/cli.md has the usage.

2082 tests green, including the fixture-backed restart-continuity suite.

Full diff: 6.42.0...6.43.0

6.42.0 - A landing line has to stand on ground somebody read

Drop-in from 6.41.0. No API change. One behaviour change: a PGS set can no longer claim a seek landing over source nobody read.

A landing line has to stand on ground somebody read

@rrgomes reported a caption that survives a re-aimed restart: on the 6.40.0 path, a coalesced far seek lands on target and the overlay comes up carrying a set from half a minute earlier, closed at the far side of the authored silence instead of at its own clear. Two devices, same asset, same day: an Apple TV 4K and an M4 Pro under a controlled repro (drain target 199.1, boundary re-aimed twice, gate open at 191.983, the 187.729 set standing over the scene at 199.4).

A PGS display set has no end of its own, so the reconstruction pass at a landing decides which set is still on screen by the store holding nothing between that set and the playhead. Over a stretch nobody read, that silence is not evidence. A run re-aimed just after it harvested a set leaves the set's own clear inside the ground the re-aim skipped: it decodes at the landing looking unclosed, becomes the landing's active line, and alignCueEnds then closes it at the next stored packet, which is the far side of the silence rather than its successor.

The report's premise about the store is worth correcting because it changes the fix: the packet store does not prune behind the producer. Since #125 its retention is byte-bounded only (32 MiB per stream, evict-oldest); the time-based prune was removed precisely because a backward seek into cache-resident content could land on packets it had evicted. The clear was not dropped, it was never read.

#362 round 2 measured that the packets alone cannot show this. A reader restarted BEHIND leaves a descending harvest sequence at the boundary, which is what harvestGapCut reads; one re-anchored FORWARD hangs its packets in ascending order behind the stretch it skipped, so the pair is indistinguishable from an authored silence. The coverage ledger was named there as the precise signal and left unbuilt for want of a reproducible defect. This is that defect, so it is built now.

SubtitleHarvestCoverage keeps one span per harvest run inside the packet store. The forward prefetcher reports its anchor, every in-place re-anchor and its read position as it goes; the pump's run begins where the producer opens or restarts (native and software paths) and reaches at least the playhead, since playback is rendering there, plus every packet it harvests. A set whose ground up to the playhead is not covered can no longer be the landing's active line, and the same rule closes the #100 stale hold's door onto the identical claim. A store nobody reports coverage to answers every span with yes, so a harvest path without notes behaves exactly as it did before.

The cost is the landing line in the case where a set really is still up and the proof is missing, which takes an authored dwell long enough to span the whole unread stretch. The alternative was paying it for every normally authored set that ends inside one.

Neither of the two shapes the report offered was taken. A PGS dwell cap is an end nobody authored, and it would have to be short enough to cut a two-second caption off before the landing while cutting a long sign short in the same file; three rounds of #362 went into removing laundered ends rather than adding one. A one-shot backfill of the skipped ground is the better of the two and the ledger makes it possible, but it spends a read on a link that has just paid for a seek, and it buys nothing the refusal does not in the reported case: the authored state at the landing is an empty screen either way.

Measured

500 s H.264 + PGS fixture with a scheduled sup (set at 184, authored clear at 190, then silence to 240), served over a loopback origin, seeking from 100 to 197 so the initial run's park edge falls between the set and its clear while the restarted pump opens at 192, above it. Counted over 10 runs per arm:

arm	landing tick met the hole	stale CUE 184.00-240.00	landingWithheld=	authored set at 240
6.41.0	4/10	4/10 (every one of them)	0/10	10/10
6.42.0	6/10	0/10	8/10	10/10

Whether the landing tick meets the hole is a race with the re-anchored side reader, which
repositions to target - 2 on the subtitle axis and so re-reads the clear it lands on; that is the
correction the report saw arriving after about seven seconds on the Mac and after tens of seconds on
the Apple TVs. The rows above are the same race on both arms. The refusal count is higher than the
hole count because the ground between the set and the playhead is unread in every run, whether or
not the clear has arrived by the time the tick decodes it; where it has, the outcome was already
correct and the refusal changes nothing.

Controls, unchanged in both arms: a far seek into fresh ground landing inside an authored dwell still publishes its line (CUE 292.00-295.50), and a plain playthrough delivers the same cues with no withholding.

A refusal names itself as landingWithheld=N on the #357 subtitle-delivery line, beside endsWithheld=.

2055 tests green (16 new), tvOS, iOS and visionOS Simulator builds green.

6.41.0 - The bitstream knows what the container dropped, and a stalled source outranks a seek

Drop-in from 6.40.0. No API change, one behaviour change worth reading: playbackPhase now ranks a stalled source above an in-flight seek.

A container that lost the presentation axis

@orut34iop reported an MP4 that judders from the first picture, with no seek needed to provoke it. The writer omitted the composition-offset table while the H.264 bitstream still reorders pictures, so every sample reports PTS == DTS, the container hands decode order out as presentation order, and the native stream-copy carries it into fMP4 unchanged.

Measured through AVFoundation's own decoder on a twin pair (one encode, muxed twice, ctts removed from one), matching decoded content against the healthy twin: 45 of 66 pictures presented at a time belonging to a different picture, and the content order stepped backwards 30 times, in a repeating +67 ms / -100 ms shuffle per B-group.

The reporter's proposed remedy (read best_effort_timestamp and route the file to software decode) was measured and does not hold: best_effort_timestamp is not the presentation axis, it is pkt_dts of the packet just fed in, which is the decode ladder displaced by the reorder delay. On the same pair that is +2002 ticks (two frames) on 63 of 66 pictures and a backwards step at the EOF flush. It trades judder for a permanent lip-sync error, and it would have cost hardware decode.

The container lost the information; the bitstream did not. libavcodec's H.264 parser reads each access unit's picture order count without decoding a pixel and takes MP4's length-prefixed payload directly, so the demuxer rewrites the packet to what the muxer should have written: presentation by display rank, decode pulled back by the reorder delay, which is also what keeps PTS >= DTS. The shift between the sampled ladder and the presentation timeline is read from the container (start_time against the first index entry) and clamped to one reorder delay, so both edit-list shapes land on the same axis and a malformed header cannot drag the picture off its audio.

Repairing at the demuxer boundary keeps one axis for every consumer (fMP4 producer, segment plan, software decoder, still extractor) and keeps hardware decode: a container defect no longer costs the native path.

Verified: 432 packets across three fixture pairs (both edit-list shapes, seven IDR boundaries) match their healthy twin exactly, the served output presents all 301 frames at the same times as the healthy twin's, segments stay independently decodable, and a 13-seek burst lands identically on both arms.

Detection is fail-closed and nearly free on healthy files: the first real PTS-DTS offset ends the sample. A source is repaired only when every sampled pair is equal, the decode ladder is uniform, the picture order regresses, and the ranks it produces are distinct and fill the sampled window. Live, still-extraction and video-discarded demuxers never sample at all.

A dead source is not "seeking"

@rrgomes reported that PlaybackPhase folds isSeeking ahead of the reader's reconnect axis, so a source outage disappears from the phase for as long as any seek is alive.

Over a source that stopped delivering, a seek cannot land, so that is the entire outage. From the report, an origin killed mid-playback and restored later:

03:41:57.861  playbackPhase → .stalled(reconnecting: true)          1.9 s after the cut
03:43:10.298  seek#1 nativeScrub began target=140.93                producer restart, not a host seek
03:43:10.299  playbackPhase → .seeking
03:43:31.789  [AVIOReader] pump reconnect exhausted at offset 0 status=0 (5 unproductive)
03:43:31.789  [HLSVideoEngine] restart at idx=35: old producer wedged; reopen failed
03:43:39.683  [AVIOReader] pump gen=42 first data after 37ms        origin back

29 s of a dead origin, two exhausted reconnect ladders and a failed reopen, all reported as .seeking, and the seek doing it was the engine's own: the producer's restart coalescer issues nativeScrub seeks while recovering, so the engine hid the outage it was recovering from with no host seek involved at all. A second run with host seeks on top reached 41 s.

Precedence is now:

error > ended > idle > loading > stalled > seeking > rebuffering > playing/paused

A seek remains fully observable through isSeeking and seekEvents, which carries the outcome a level signal structurally cannot (6.x, #38); the reader axis is observable nowhere else, which is what playbackPhase exists for. Over a delivering source nothing changes: the reader is flowing, a seek reads .seeking exactly as before, and a seek that lands from cache over a reconnecting reader still clears itself in milliseconds.

.stalled(reconnecting: false) now has a meaning. The value was reserved for "stalled, retries paused" and is emitted when a reconnect ladder is spent and recovery has passed to the producer's reopen. Until now the dying reader claimed delivery on its way out, so the whole reopen window read as a healthy source.

Only bytes that crossed the network move the axis back to healthy. This is the same definition of progress the reconnect ladders have used since #380: a serve out of the resident window, the retained head/tail spans or a resident detour block hands back read-ahead the origin paid for before it died, so it can no longer erase a stall. Without this the honest give-up would have been wiped by the next read out of the window.

A metered detour fetch now reports the stall it was already charging its ladder for. 429 / 503 / 509 on the arm built for throttling origins moved the ladder and told the phase nothing; measured against a local 509 origin, streak=3 with the phase still reading healthy.

One case is deliberately unchanged: the faulted-runway refill replaces a dead connection while megabytes are still resident and stays silent, because playback there is uninterrupted. The empty-window ladder reports it when playback genuinely starves.

The reconnect loop is also under test for the first time (it was left as device-verify pending when the phase was introduced): a real reader against a local refusing origin, pinning the phase sequence across runway, refusal and give-up, a transient refusal clearing on the origin's own delivery, and sixteen retained-head serves that must not move the axis.

Notes

• Suite 2039 tests in 291 suites.
• CHANGELOG - full diff

6.40.0 - A restart may not open past the boundary it was given

Drop-in from 6.39.0. No API change. One fix on the native VOD restart path, the second half of #408, and it is about a promise the container index makes and does not keep.

The index is not a keyframe index

@rrgomes reported that after a deadline re-anchor the seek does not land at its target: it lands at the next keyframe the restarted producer accepts, 3 to 14 s later, sixteen times across two devices, and the first picture it shows is visibly broken. One instance end to end:

seek#11 programmatic began target=246.57
...
[HLSSegmentProducer] still waiting for video keyframe: dropped=200 lastDts=252335
              isKey=false target=244119 baseIndex=61
[HLSSegmentProducer] video gate open: actual=255255 anchorPts=255297 target=244119
              desired=244119 shift=11136
seek#11 programmatic landed rendered=256.54 target=246.57

The plan boundary at 244119 comes from Demuxer.indexedKeyframes, which reads libavformat's index filtered on AVINDEX_KEYFRAME. For Matroska that index is the Cues table, entered by matroska_add_index_entries:

if (track && track->stream)
    av_add_index_entry(track->stream,
                       pos[j].pos + matroska->segment_start,
                       index[i].time / index_scale, 0, 0,
                       AVINDEX_KEYFRAME);        /* unconditional */

The block's own keyframe flag is never consulted. Cues mark seek points; nothing in the format says a cue point is a sync sample, and the reporting asset writes them at ordinary frames. So a plan line reading keyframe-aligned, 6004 IRAPs was never an IRAP count, it was a cue count, and three quarters of the boundaries it cut on cannot start a decode. The asset's own reading is one line: 244.119000,243.994000,___, a frame with no keyframe flag exactly where the index promised random access, and no K at all until 255.297.

That reading settles the open-GOP question too. ffprobe's K is not the container flag: with parsing on, which is the default, it comes from the elementary-stream parser, and the H.264 parser marks a slice carrying a recovery point SEI, not only an IDR, while the HEVC parser marks every IRAP NAL. So the gap holds no random-access point of any kind and the producer's forward scan was doing the only thing available to it.

Two rules changed

A restart may not open past a boundary that claimed random access. The gate goes back for a sync sample covering it, in widening steps (4, 8, 16, 32 s) and never more than four attempts. The common case costs one extra seek and finds the covering sample one segment back. Each attempt escalates the moment its scan reaches ground already proven empty, so a deep drought is read once rather than once per attempt, and the case is detected on the packet sitting on the boundary rather than after reading the whole gap: the repaired path reads less than the old one, not more.

A gate that opened EARLY keeps its own position. Pinning the epoch's first frame to the segment's advertised start is what turns a gate mismatch into a shift of the whole item axis, and it exists for the case that cannot be helped, since opening late leaves a hole at the advertised time and AVPlayer waits on holes forever. Opening early has no such problem: publishing the frame at its own position costs an overlap with the previous segment, which AVPlayer absorbs, and the item axis then stays where the plan put it, so the seek lands where it was aimed rather than short of it. Both operands of that comparison are on the item axis; mixing in a source timestamp publishes every restarted epoch a plan anchor early, which the fixture-backed restart-continuity tests caught.

Scope is deliberate. Only the keyframe-aligned plan is affected: the uniform grid never claimed its boundaries were random-access points, and a source-declared plan aims below its IRAP by design (#268). The tolerance for opening slightly past a boundary now covers the stream's own reorder depth, because a container index entry is a decode timestamp while the gate judges presentation time (#169 round 3), so a well-formed B-pyramid index legitimately places the keyframe's presentation above the boundary and must not pay a second seek for it.

Measured

Scripts/mkv-cue-fixture.py is new and builds the shape: it rewrites a Matroska's Cues table to mark positions that are not sync samples, which no muxer will write for you. Over a fixture with a deliberate 12 s keyframe drought at 43 s it reproduces the report line for line, down to the drop count.

Resuming at 45 s, two seconds into each arm:

	gate	clock at t=02
before	actual=55000 target=44000 shift=11000	57.80 s
after	actual=43000 reaimedTo=40000 shift=0	46.80 s

Backward seek to 46 s:

before:  seek#1 programmatic landed rendered=47.08 target=46.00
after:   seek#1 programmatic landed rendered=46.00 target=46.00

The control fixture, identical except that its cues ARE its sync samples, is untouched on both arms: no re-aim, no extra seek, and the same shift it always had.

Reported by @rrgomes.

---

Full notes: CHANGELOG.md · 6.39.0...6.40.0

6.39.0 - The island is measured at the target, and the producer follows it there

Drop-in from 6.38.0. No API change. Two fixes on the native VOD seek path, both from #408, and both about a measurement that proved less than the code believed it did.

The 12 s wait was a fallback, not a budget for that seek

@rrgomes reported that a seek whose target is already cache-resident waits 12 s, or 24 s at the top of the extension ladder, before the engine re-anchors the producer, and that the figure lands exactly on nativeSeekReconcileBudgetSeconds + nativeSeekExtensionBudgetSeconds rather than on anything the engine observed. Measured landings clustered on 12.07, 12.10, 12.11, 12.13 and 12.29 s against a median of 166 ms for the same asset and device when the path is not hit.

The arithmetic is right, and the constants are not what to change. The median seek lands in 166 ms because VideoSegmentProvider re-anchors the producer when the consumer's target moves. The reconcile budget is what runs when that did not happen, so the question is why it did not.

The island was not measured at the target

NativeAVPlayerHost.bufferedSecondsAtTarget summed every loaded range intersecting [target - 1 s, target + 30 s]. The window reaches thirty seconds PAST the target and every range inside it counted at full weight, so media loaded well downstream read as "the producer is serving the target". That is the only reading consistent with the reported trace:

seek slow but producer serving target
  (island=7.30s at target, rendered=456.13s buffered=456.13s); extending budget 1/4

rendered == bufferedEnd is AVPlayer parked with nothing ahead of it, and a seek that never lands. Had 7.3 s of media actually covered the target, the seek would have landed on it instead. The figure came from somewhere else in the window.

The first extension is granted on presence alone, because there is no earlier sample to compare growth against, so a phantom island bought 4 s on top of the 8 s budget on every instance, deterministically. That is the whole of the reported 12 s.

Coverage of the target is now a gate on the reading. The window keeps its width, because measuring how deep the served region runs is what separates a producer still filling from one that served a little and stopped; only the licence to read the window at all now requires the target to be inside it. In the reported shape the island reads 0, below nativeSeekProgressIslandFloorSeconds, so no extension is granted and the deadline goes straight to the re-anchor.

The producer was never told to move

VideoSegmentProvider.handleTargetChange skips the proactive re-anchor on a backward target jump when the target segment is still resident:

[HLSVideoEngine] declareTarget backward jump 102 -> 27: resident in cache, no restart

That gate exists for the Continuous-Audio handover refetch, roughly seven to ten segments backward into content the ACTIVE pump is still writing, where an unconditional restart re-arms the FLAC bridge and glitches the audio. Residency of the target segment alone does not carry that, because a scrub band left by an EARLIER pump is resident too, and it ends.

Reproduced headless with aetherctl play on a 600 s H.264 Matroska fixture, seeking back into a three-segment band while the pump was anchored at seg99:

[AetherEngine] seek#2 programmatic began target=155.00
[HLSVideoEngine] declareTarget backward jump 103 -> 38: resident in cache, no restart
...
[HLSVideoEngine] seg41: out-of-range fetch (cache.range=25..113 highWater=113 attempt=1/3), restarting producer

Nothing moved the pump. The consumer walked the band in 4 s and asked for the first index above it with 5 s of buffer left, so the whole re-anchor was paid at the one moment its buffer was empty. On a longer band the pump instead sat parked for 24 s, until the #65 backpressure wedge breaker moved it onto the AVPlayer position.

The gate now holds only in the two shapes where it is true: the resident run reaches the active march front, so the band carries the consumer straight back into the pump's own output and there is no gap at all, or the run is at least a prefetch window deep, so the gap gets asked for with a full cushion and re-anchoring early would only re-produce content already on disk. Anything shallower re-anchors when the target changes.

Same run, after:

[HLSVideoEngine] declareTarget backward jump 103 -> 38: resident only through seg40, which is
                 below the march front 113, so the band ends in a gap nothing is producing
[HLSVideoEngine] producer restarted at idx=38 (seek=152.00s, restart took 1ms)

On making the budget configurable

The report offered exposing the three constants as a fallback, on the precedent of AVIOReader.connStallTimeout. Holding off, and the precedent does not hold: connStallTimeout is an init parameter taken so tests can shorten it, with no host-facing knob and one shipped value. The complaint is that the wait was not derived from anything observed about the seek, and the answer to that is for it to be derived.

Still open in #408

The second half of the report, that a re-anchor lands systematically 3 to 14 s past its target with a visibly broken first picture, is not addressed here and needs evidence from the reporting asset. The leading mechanism is that a plan boundary is an entry in the container's keyframe index (AVINDEX_KEYFRAME, Matroska Cues or the MP4 sync sample table) while the producer's gate opens on AV_PKT_FLAG_KEY at the demuxed packet, two authorities on the same question with no path back between them: the restart seek may not land before the boundary and the gate may not open before it either, so a disagreement is scanned past rather than repaired. Tracking in #408.

Verification

1989 tests green, build clean, plus the aetherctl play before and after above. Issue408ReconcileEvidenceTests pins both decisions.

Reported by @rrgomes.

Full changelog: 6.38.0...6.39.0

6.38.0 - An untimed frame is repaired before it is refused

Drop-in from 6.37.0. No API change. One hardening from #407, which is worth reading for what it is not: the fix does not address the judder that was reported, because the cause the report proposed is not reachable in this engine. The hole it did expose is real on its own, and that is what closes here.

The report, and what it actually measured

@classicjazz reported VC-1 judder on Apple TV, with an analysis pointing at SoftwareVideoDecoder reading AVFrame.pts where every other FFmpeg-based player reads best_effort_timestamp, and an ffprobe capture showing raw pts_time=N/A on every I and P picture of the source.

The capture reproduces. It is also one layer lower than the analysis placed it, at the packet, before any decoder runs:

$ ffprobe -show_entries packet=pts_time,dts_time,flags vc1.mkv
pts_time=N/A     |dts_time=0.000000|flags=K__
pts_time=N/A     |dts_time=0.040000|flags=___
pts_time=0.080000|dts_time=0.080000|flags=___
pts_time=N/A     |dts_time=0.120000|flags=___
pts_time=0.160000|dts_time=0.160000|flags=___

libavformat/matroskadec.c:4103 in n8.1.2, the FFmpeg this engine links:

if (track->ms_compat)
    pkt->dts = timecode;
else
    pkt->pts = timecode;

ms_compat is set for V_MS/VFW/FOURCC tracks (matroskadec.c:2897), which is how WVC1 and the legacy Microsoft codecs are stored in Matroska. The block time becomes DTS, pkt->pts is never written, and libavformat's generic reorder then recovers a PTS for the non-reference pictures and not for the rest. So the split is a container rule that holds for every VFW-tagged Matroska track, independent of codec.

The engine does not see it. Demuxer.applyDemuxerOptions opens every source with fflags=+genpts, whose entire job is filling exactly these gaps, and both avformat_open_input sites go through it. Measured with the engine's own demuxer under the playback open profile:

$ aetherctl pktdump --count 300 --profile playback vc1.mkv
[PktDump] stream[0] tb=1/1000 avg_frame_rate=25/1 codecpar.video_delay=1
[PktDump] video packets=300 keyframes=300 NOPTS_dts=0 NOPTS_pts=0 nonMonotonicDts=0
[PktDump] dts-delta histogram (delta x count): 40x299

The same ffprobe run with -fflags +genpts reports a complete, evenly spaced series for the same file.

An untimed frame is refused, not misplaced

This is what rules the hypothesis out by shape rather than by argument. With +genpts suppressed in a local build, the state the analysis describes is forced, and the same VC-1 file plays like this:

[Renderer] dropped 1 frame(s) with no usable timestamp (unschedulable)
[Renderer] dropped 250 frame(s) with no usable timestamp (unschedulable)
[Renderer] dropped 1250 frame(s) with no usable timestamp (unschedulable)
[SWDiag] clk=2.53 eof=y enq=+214 status=unknown r4d=n
VERDICT: clock did not advance (t=2.53s); transport stalled after load

AVSampleBufferRenderSynchronizer cannot pace a sample whose PTS reads back as NaN, and a NaN also breaks the reorder buffer's ordering, since every comparison against NaN is false. So the gate refuses the frame outright rather than let it reorder its neighbours (#298). The result is no picture at all, and a 58 s file drained in 2.5 seconds because nothing is left to pace the demux loop. That is a different failure from judder on a picture that is present.

What changed

The software path had that drop at two layers and no repair between them: the deinterlacer discards its own untimestamped output, SampleBufferRenderer.enqueue refuses the sample, and nothing in between tried to recover a timestamp first. The only thing standing between an untimed picture and a dropped one was one demuxer flag on one open.

SoftwareVideoDecoder now reads best_effort_timestamp when the decoder set no PTS. That is libavcodec's own guess_correct_pts(pts, pkt_dts), the reconstruction every other FFmpeg-based player consumes. Three properties of the placement matter:

It sits directly after avcodec_receive_frame, so the A53 caption tap, the deinterlace graph and the emit path all read one repaired timestamp instead of each reading the raw field separately. Pushing an untimed frame into the filter graph would have produced untimed output that the graph's own drop then discards, so repairing only at emit would have fixed one of three consumers.

A decoder-set PTS is never overwritten. best_effort_timestamp is derived from that same PTS plus the packet DTS, so preferring it could only ever move a correctly timed frame.

A frame with neither value still falls through to the gate. Substituting zero, or the previous frame's timestamp, puts a wrongly placed picture into the reorder buffer, which is worse than one missing frame.

Measured on a VC-1 Matroska fixture with +genpts suppressed: the session above now runs at 25 enqueues per second on a 25 fps source with an advancing clock, and logs [SWDecoder] repaired N frame timestamp(s) from best_effort_timestamp. With +genpts on, the repair never fires and nothing changes.

Two shapes reach a decoder untimed on their own and are why this belongs at the frame rather than at the container: V_MS/VFW/FOURCC Matroska tracks as above, and live MPEG-TS, which delivers untimed pictures outright (the deinterlace path's own drop already names that case).

Not in this fix

The judder in #407 is still open, and two candidate causes are now measured and dead rather than merely unbuilt. The timestamp path is one of them, for the reason above. The other is decode starvation, which is worth writing down because the VC-1 decoder invites the guess: libavcodec grants it no decode parallelism at all, on any thread count.

[SWDecoder] Opened: 1440x576, codec=vc1,        threads=1, 8-bit
[SWDecoder] Opened: 1440x576, codec=mpeg2video, threads=8, 8-bit

Same machine, same 8-core request, one codec apart. There is no frame threading in that decoder and no slice threading on a single-slice stream. It still has ample headroom: the full software path (aetherctl swdecode in a release build, so decode plus the YUV to NV12 conversion plus the pixel-buffer allocation) runs a 1280x720 VC-1 Advanced stream at 13.4 Mb/s at 2999 frames in 4.54 s, 660 fps, 608 Mpixel/s on one core. 1080p at 23.976 needs 49.7 Mpixel/s, about 8 percent of that. So a single-threaded VC-1 decoder is a fact about the decoder, not an explanation for a dropped frame.

What is left is on the presentation side, and it is not measurable off tvOS, which is what the next round has to supply.

An explicit CMSampleTimingInfo.duration was the third recommendation in the report and is not taken. It may well be right, but nothing measured here changes with it, and an unmeasured second change does not belong in an open judder report.

Reported by @classicjazz.

6.37.0 - The no-cut watchdog stops waiting on the read it is timing out

Drop-in from 6.36.0. No API change. One fix, the fourth finding of the #405 trace, which was measured instead of built at the time because the measurement pointed somewhere neither the reporter nor I was looking.

The watchdog was inline in the loop it watches

HLSSegmentProducer evaluated its no-cut stall decision at the top of the pump's read loop, so it could only run between av_read_frame calls. av_read_frame does not return before a whole packet is assembled, and there is no interrupt_callback on the format context, so that call has no upper bound at all. An origin too slow to complete one packet inside the watchdog window did not make the watchdog late. It made it unable to run.

Reproduced on a loopback origin that serves a 300 s MPEG-TS at 200 KB/s, then drops to 100 bytes once per second for 45 s on the connection it already holds, then recovers. One connection for the whole run, Range: bytes=0-, no reconnects, which is the shape the field trace had (gen=1->1, reconnects=0, every wait signalled with nothing to show for it).

6.36.0:

[AVIOReader] slow read: 46642ms at offset=3670016 stallWaits=57(46642ms,57signaled) reconnects=0 iters=115 gen=1->1
[HLSSegmentProducer] no-cut stall: no segment finalized for 46s (packetsRead=1386, sinceFinalize=24, rate=0.5pkt/s, source starvation); exiting for host retune

Those two lines are adjacent, and in that order. The starvation window is 35 s and it classified at 46 s: the eleven seconds of overrun are exactly the time the read was blocked.

6.37.0, same origin, same run:

[HLSSegmentProducer] no-cut stall: no segment finalized for 35s (packetsRead=1493, sinceFinalize=131, rate=3.7pkt/s, source starvation); aborting the source read and exiting for host retune
[AVIOReader] slow read: 34586ms at offset=3932160 stallWaits=36(34585ms,36signaled) reconnects=0 iters=72 gen=1->2

The order is reversed, which is the whole point: the verdict is reached while the read is still parked, and the read line that follows is the abort ending it. Both arms exit reason=segmentStall into the host retune; this one reaches it 11.4 s earlier (53.2 s of pump against 64.6 s).

What changed

The window state moves out of the read loop's locals into NoCutStallWatchdog, which the read thread reports into and a 1 s timer evaluates. The classifier, both timeouts, the #177 slow-delivery hold and the log vocabulary are unchanged; only the position of the decision moved.

Three things that had to come with it:

The abort already existed. Demuxer.markClosed() makes the AVIO read callback return -1 and av_read_frame return at once, freeing nothing. It is what the #79 reopen path uses on a wedged read, and HLSVideoEngine's session teardown already pairs it with the producer's own stop(), because a cancel flag cannot reach a thread parked in a read. A .segmentStall exit tears the demuxer down anyway, so it has nothing to preserve across the abort.

That abort surfaces in the loop as a read error, and .readError sends a reopenable source into a URL reopen of the very origin that just starved. The latched verdict is what the exit means, so it overrides on the way out. A teardown in flight still wins.

A deliberately parked pump is not judged. The live headroom park sleeps the read thread on purpose, for a consumer that stopped polling. Inline, the watchdog could not run there at all; on a timer it must not count that time, or a dead consumer would be reported as a dead source. The park brackets itself and the window re-anchors on release.

Not in this fix

Two things were measured and rejected rather than shipped, both worth stating so nobody re-attempts them:

• Returning short reads while bytes trickle in. Patched, measured, worth one second (the watchdog fired at 47 s instead of 48 s in the original pairing). It moves the block up one layer; the packet still has to be assembled.
• A throughput floor in the reader. Both of the reader's stall detectors are zero-byte detectors rather than throughput detectors, which is why a trickling origin holds them at "flowing" indefinitely. But the floor proposed for it, less than one AVIO read in 20 s, is a 102 kbit/s lower bound and would hang up on healthy low-bitrate live audio. Withdrawn by the reporter after the measurement.

Who this changes anything for

A host with its own playhead-progress watchdog shorter than 35 s will see no difference; that watchdog still fires first. Hosts without one now reach the retune about eleven seconds earlier on a starved live source, and stop waiting on a read that has no bound of its own.

Traced and reported by @tschuegy in #405, split into #406.

6.36.0 - Live recovery asks the producer, types a dead lease, and catches a renumbered clock

Drop-in from 6.35.0. No API change. Three live-recovery readings corrected, all from one device trace (tvOS 26.6, an MPEG-TS channel behind an Xtream aggregator that answers with a 302 to a one-slot media host).

The origin caused the freezes in that trace. What is fixed here is what the engine concluded from them: recovery ran about 12 s longer than it needed to and replayed eleven seconds the viewer had already seen.

The stall ladder now asks the producer before replacing the consumer's item

Stage 2 of the #65 ladder loads a fresh AVPlayerItem, which is the right tool for a consumer that died under a healthy producer and the wrong one for a producer starved by its origin. It gated on consumer fetch activity and on the position budget, and both are silent in either case.

In the trace it fired while the source was still re-resolving and no bytes were arriving. AVPlayer rejoined an unchanged local playlist at edge-minus-holdback, five seconds behind the frozen position, refetched the tail it had already shown and parked again, and only two grace windows later did the final rung ask the host to retune.

The count of segments the producer has finalized is the one fact that separates the two, and it was already the currency of two other budgets (the reopen cycle and the muxer rebuild). Stage 2 now consults it and skips straight to liveSourceReset when nothing has been finalized since the stall.

A session with no local producer at all, a remote HLS route AVPlayer fetches itself, reports nil rather than zero and keeps its old behaviour: the absence of a producer to ask is not an answer from one. The producer-driven wedge re-anchor and the item-death chain are deliberately unchanged; neither describes a starved producer.

407 from a pinned redirect target is a dead lease

[AVIOReader] pump gen=2 rejected response status=407 at offset 34545940 from the pinned target <media host>
[AVIOReader] pump conn ended at offset 34545940 status=407, reconnecting (streak=1 retryAfter=0.0s)
[AVIOReader] pump gen=3 rejected response status=407 at offset 34545940 from the pinned target <media host>
[AVIOReader] Dropped resolved URL cache (unproductive reconnect streak)
[AVIOReader] pump conn start gen=4 offset=34545940 open-ended re-resolving through the source

407 fell through the expiry, rate-limit and hard-error classifiers alike: no pin drop from the status, charged against the full mid-stream reconnect cap, and the pin dropped only afterwards by the unproductive-streak rule. So the attempt right after the refusal went back to the address that had just refused, and the move that works, the one on gen=4, cost an extra round trip to reach.

On a redirect chain a 407 cannot mean "authenticate to your proxy". The request went out direct, which is exactly why CFNetwork logs it as an unexpected proxy response, and a genuinely configured proxy is answered by URLSession's own auth challenge long before a status code reaches the reader. It means the pinned lease is gone or an interception answered in its place, and one re-resolve through the source is the only productive response.

407 now classifies as resolved-address expiry alongside 401/403/404/410, and 402 and 451 with it: an expired subscription or a geo-refusing edge node is the same shape, where the source still mints working targets and this one stopped being one. Rate-limit statuses (429/503/509) stay out, because there the origin is metering us, the pin is fine, and re-resolving spends the request there is no room for.

A source that renumbers its clock from zero is a restart, not a programme boundary

When a live origin restarts its stream from its ring buffer with raw dts back at zero, FFmpeg's 33-bit wrap correction adds 2^33, so what reaches the producer is a large FORWARD jump:

[HLSSegmentProducer] video timeline rebase (live): jumpTicks=8487014192 srcDts=8589934592 lastSrcDts=102920400 oldShift=100915200 newShift=8587927592 continuationDts=2007000
[HLSSegmentProducer] seg-15.m4s captured (764054 B)      <- identical size to seg-6
[HLSSegmentProducer] seg-16.m4s captured (1044022 B)     <- identical size to seg-7

isSourceReplay opened with guard jumpTicks < 0, so the classifier that raises .sourceReplay never looked at it. The restart was absorbed as a programme boundary behind an EXT-X-DISCONTINUITY and the session re-served eleven seconds it had already played.

The anchor is the part that is easy to get wrong. A rewind lands near the first dts this session saw, because the server restarted the programme. An axis reset lands near zero no matter where the session joined the ring: oldShift=100915200 puts that join 1121 s into the origin's axis, and the restart lands at raw 0, so testing the wrap-corrected value against the first-seen dts would have missed it by three orders of magnitude.

Both shapes are recognized now and both end the pump for a host retune. The axis reset requires no recent unplanned reconnect, because the origin renumbers on the connection it already holds (gen=1->1, reconnects=0 in the trace) and the signature carries its own discriminator: a boundary inside one transport stream keeps its PCR axis running, and a genuine 33-bit wrap after ~26.5 h arrives as a continuous correction rather than a jump over the discontinuity threshold. It is live-only, since a sequential origin's archive chunks legitimately open their own axis at zero (#368).

Not in this release

The fourth finding from the same trace, that a trickling origin holds both of the reader's stall detectors at "flowing" because both are zero-byte detectors rather than throughput detectors, is real and still open. Measured against a loopback origin that delivers 100 bytes once a second for 45 s on a held connection: the reader posts one slow read: 46874ms stallWaits=71(71signaled) reconnects=0, and the producer's own no-cut watchdog, which classifies source starvation on a 35 s window, fires 30 ms after that read returns, at 48 s.

Returning a short read instead of holding out for the full request moves that to 47 s. One second, because the watchdog runs between av_read_frame calls and av_read_frame does not return until a whole packet is assembled, which at that rate takes longer than the window no matter how short the reads underneath it are. So the reader's floor is not where the time goes: the no-cut watchdog is inline in the loop it watches, which makes it depend on exactly the call it is supposed to time out. Lifting it onto a timer, with an interrupt callback to bound av_read_frame, is its own change and is not in this release.

Reported by tschuegy (#405), from a Syravo device trace.

Full changelog: https://github.com/superuser404notfound/AetherEngine/blob/main/CHANGELOG.md
Diff: 6.35.0...6.36.0

6.35.0 - The session says which codec and container it opened

Drop-in from 6.34.1. Additive API: two published fields naming the source a session actually opened, plus a SOURCE readout in aetherctl play.

A host could ask what is decoding, not what was opened

activeVideoDecoder answers "VideoToolbox H264 (HW)", which is a statement about the decoder, not about the source. The codec name itself existed at load (detectedCodecID) and in SourceProbe, and the running session published neither, so a host building a stats panel had to fall back on its own catalogue metadata.

That fallback fails in two ways at once. Catalogue metadata describes the file a library holds, which under a remux or a transcode is not what arrived. And a host whose item payload happens to be slim, a list response rather than a detail one, has nothing to show at all: the same panel renders in full for one item and half-empty for the next, with no difference in the playing media.

What a session publishes now

engine.sourceVideoCodecName   // "hevc", "h264", "av1", nil when the source carries no video
engine.sourceContainerFormat  // "matroska,webm", "mpegts", nil on the remote-HLS bypass

sourceVideoCodecName is the libavcodec spelling on every route. The probe path takes it from avcodec_get_name. The probe-free remote-HLS bypass has no libav context, so it maps the item's video sample type back to the same word ('hvc1' and 'dvh1' to hevc, 'avc1' and 'dvav' to h264), and a Dolby Vision tag resolves to its base-layer codec because the DV signaling already reaches the host through sourceVideoFormat. A test pins that agreement on one fixture: 'avc1' and a probe of the same bytes have to produce the identical string, so the field cannot drift apart per route.

sourceContainerFormat is what libavformat opened. It is nil on the bypass, where AVFoundation opens the source and there is no libav context to ask, rather than guessing.

Verified against h264/mp4, h264/mkv, hevc/mp4 and h264/mpegts:

SOURCE codec=h264 container=mov,mp4,m4a,3gp,3g2,mj2 1280x720 fps=25.000 bitrate=1294896 fmt=sdr
SOURCE codec=h264 container=matroska,webm 1280x720 fps=25.000 bitrate=0 fmt=sdr
SOURCE codec=hevc container=mov,mp4,m4a,3gp,3g2,mj2 1280x720 fps=25.000 bitrate=756722 fmt=sdr
SOURCE codec=h264 container=mpegts 1280x720 fps=25.000 bitrate=0 fmt=sdr

That line is aetherctl play's, printed from the session rather than from a separate probe, because the session is the state a host panel binds to and the two can disagree.

Changed

sourceVideoWidth and sourceVideoHeight are now @Published. They were readable but silent, so a SwiftUI panel bound to them never refreshed, including across the audio-switch reload that can change them.

Compatibility

Drop-in from 6.34.1. No behaviour change to playback: the new fields are read-only state, and no existing field changed meaning.

Full changelog: CHANGELOG.md · 6.34.1...6.35.0