Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ the public-API contract.

### Fixed

- **`maximumSubtitleOffsetSeconds = 0` stops the second download behind embedded subtitles (#1015).** Selecting an
embedded subtitle on a remote source started the forward prefetch, a second reader that downloads the video bytes
again: 243.6 MB against 104.5 MB for a minute of 1080p, MKV and MP4 alike. Its cues only serve an advance offset.
A host that declares a reach of 0 now reads subtitles from the main reader alone. The prefetch still runs for a
negative `softwareSubtitleDelaySeconds` and the OCR worker, and starts or stops mid-session when either changes.
The default (60) keeps today's behaviour.

- **Pluto TV ads have sound on the native path.** Pluto muxes audio into its programme segments, stitches
video-only ad segments on a new PID into the same variant, and carries the sound of both in its demuxed
`AUDIO` rendition. The engine played the muxed track, so every ad segment was cut without sound. A live
Expand Down
47 changes: 44 additions & 3 deletions Sources/AetherEngine/AetherEngine+Subtitles.swift
Original file line number Diff line number Diff line change
Expand Up @@ -760,10 +760,43 @@ extension AetherEngine {
/// #151: forward prefetch runs for VOD sessions only (live content past the edge does not
/// exist and the pump already rides it), needs an embedded drain target (external/sidecar
/// tracks hold whole files, CC is tap-fed) and a loaded source to open a side demuxer on.
///
/// AE#1015: and it needs something to serve. On an interleaved container the side reader
/// downloads the video a second time (measured: 104.5 MB against 243.6 MB for a minute of
/// 1080p MKV, the same for MP4), and the cues it buys are the ones beyond the pump's few
/// seconds of forward park, which only an advance offset or the OCR worker ever reads. A host
/// that declared no offset reach and has no negative software delay gets the pump's harvest,
/// which is exactly what a zero or positive offset consumes. The lead length is no lever: the
/// reader is sequential, so a shorter lead reads the same bytes, only later.
nonisolated static func shouldRunSubtitleForwardPrefetch(
isLive: Bool, hasEmbeddedDrainTargets: Bool, hasSource: Bool
isLive: Bool, hasEmbeddedDrainTargets: Bool, hasSource: Bool,
advanceReach: Double, ocrArmed: Bool
) -> Bool {
!isLive && hasEmbeddedDrainTargets && hasSource
!isLive && hasEmbeddedDrainTargets && hasSource && (ocrArmed || advanceReach > 0)
}

/// AE#1015: how far ahead of the playhead a cue can be wanted. The host's declared reach covers
/// both signs (the engine cannot see which way its overlay shifts); the software PiP delay is
/// the engine's own, and only its negative side reads ahead.
var subtitleAdvanceReach: Double {
max(maximumSubtitleOffsetSeconds, -softwareSubtitleDelaySeconds)
}

/// AE#1015: re-decide the forward prefetch after the reach changed mid-session. Starts it at the
/// playhead when an advance appeared, stops it when the last one went away; a running session
/// that is still wanted is left alone.
func syncSubtitleForwardPrefetchWithAdvanceReach() {
let wanted = Self.shouldRunSubtitleForwardPrefetch(
isLive: isLive,
hasEmbeddedDrainTargets: !embeddedSubtitleDrainTargets.isEmpty,
hasSource: loadedURL != nil,
advanceReach: subtitleAdvanceReach,
ocrArmed: subtitleOCRArmedOrdinal != nil)
if wanted, subtitleForwardPrefetchTask == nil {
startSubtitleForwardPrefetcher()
} else if !wanted, subtitleForwardPrefetchTask != nil {
cancelSubtitleForwardPrefetcher(reason: .noAdvanceReach)
}
}

/// #151: a drain-tick jump with an existing cursor (seek / producer re-anchor) restarts the
Expand Down Expand Up @@ -800,11 +833,19 @@ extension AetherEngine {
reanchor.request(anchor, seekGeneration: currentSeekGeneration)
return
}
// AE#1015: a session nothing reads ahead of is stopped under its own name, so the log
// does not read as a rebuild that never came back.
guard subtitleAdvanceReach > 0 || subtitleOCRArmedOrdinal != nil else {
cancelSubtitleForwardPrefetcher(reason: .noAdvanceReach)
return
}
cancelSubtitleForwardPrefetcher(reason: .prefetchRebuild)
guard Self.shouldRunSubtitleForwardPrefetch(
isLive: isLive,
hasEmbeddedDrainTargets: !embeddedSubtitleDrainTargets.isEmpty,
hasSource: loadedURL != nil),
hasSource: loadedURL != nil,
advanceReach: subtitleAdvanceReach,
ocrArmed: subtitleOCRArmedOrdinal != nil),
let store = activeSubtitlePacketStore,
let url = loadedURL else { return }
let isCustom = isCustomSource
Expand Down
10 changes: 8 additions & 2 deletions Sources/AetherEngine/AetherEngine.swift
Original file line number Diff line number Diff line change
Expand Up @@ -1001,6 +1001,7 @@ public final class AetherEngine: ObservableObject {
softwareHost?.updateSubtitleCompositor(cues: subtitleCues + secondarySubtitleCues,
enabled: pictureInPictureActive, delaySeconds: seconds)
EngineLog.emit("[AetherEngine] software subtitle delay=\(seconds)s", category: .engine)
syncSubtitleForwardPrefetchWithAdvanceReach() // AE#1015
}

/// Master enable for background playback (iOS: PiP + background audio; tvOS: PiP keepalive). Default on.
Expand Down Expand Up @@ -4244,8 +4245,12 @@ public final class AetherEngine: ObservableObject {
/// much of them the engine keeps: the history the byte budget never evicts and how far ahead the
/// drain decodes, each never less than `subtitleBitmapMinimumReachSeconds`, and the lead never
/// more than `subtitleDrainLeadSeconds`. A host without an offset control should set 0; the
/// default keeps the behavior from before this property existed. Text tracks are unaffected.
/// Negative and non-finite values are ignored. Takes effect on the next drain tick.
/// default keeps the behavior from before this property existed.
/// AE#1015: 0 also stops the forward subtitle prefetch (#151) for text and bitmap tracks alike,
/// unless `softwareSubtitleDelaySeconds` is negative or the OCR worker is armed. On a remote
/// interleaved source that reader downloads the video a second time.
/// Negative and non-finite values are ignored. Takes effect on the next drain tick; the
/// prefetch starts or stops at once.
/// A host that sets this also gives up the bitmap history behind that floor: every cue that
/// ended before it is dropped at once instead of being kept up to the 64 MiB budget, see
/// `bitmapCueHistoryByteBudget`.
Expand All @@ -4255,6 +4260,7 @@ public final class AetherEngine: ObservableObject {
maximumSubtitleOffsetSeconds = oldValue
} else {
subtitleOffsetReachDeclared = true
syncSubtitleForwardPrefetchWithAdvanceReach() // AE#1015
}
}
}
Expand Down
4 changes: 4 additions & 0 deletions Sources/AetherEngine/Subtitles/SubtitleDrainStopReason.swift
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,8 @@ enum SubtitleDrainStopReason: String, Sendable {
/// `startSubtitleForwardPrefetcher` is replacing a running session with a new one (a changed
/// lead, or a jump the in-place re-anchor cannot serve). The one reason that is not a loss.
case prefetchRebuild
/// AE#1015: the host's offset reach and the software delay no longer read ahead of the
/// playhead, so the second copy of the stream buys nothing. The drainer keeps running on the
/// pump's harvest.
case noAdvanceReach
}
89 changes: 85 additions & 4 deletions Tests/AetherEngineTests/Issue151SubtitleForwardPrefetchTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -160,13 +160,17 @@ struct Issue151SubtitleForwardPrefetchTests {
@Test("prefetch runs only for VOD sessions with embedded drain targets and a source")
func gatingRules() {
#expect(AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: true, hasSource: true))
isLive: false, hasEmbeddedDrainTargets: true, hasSource: true,
advanceReach: 60, ocrArmed: false))
#expect(!AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: true, hasEmbeddedDrainTargets: true, hasSource: true))
isLive: true, hasEmbeddedDrainTargets: true, hasSource: true,
advanceReach: 60, ocrArmed: false))
#expect(!AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: false, hasSource: true))
isLive: false, hasEmbeddedDrainTargets: false, hasSource: true,
advanceReach: 60, ocrArmed: false))
#expect(!AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: true, hasSource: false))
isLive: false, hasEmbeddedDrainTargets: true, hasSource: false,
advanceReach: 60, ocrArmed: false))
}

@Test("prefetch holds whenever its origin is paced or serial")
Expand Down Expand Up @@ -307,3 +311,80 @@ enum MatroskaSubtitleFixture {
return Data(header + segment)
}
}

/// AE#1015 (samimt55): selecting an embedded subtitle on a remote source doubled the bytes read
/// (104.5 MB against 243.6 MB for a minute of 1080p MKV, the same for MP4), because the #151
/// forward prefetch is a second reader of the interleaved stream. Its cues only serve an advance
/// offset or the OCR worker, so a host that declared no offset reach no longer pays for it.
@MainActor
struct SubtitleForwardPrefetchAdvanceReachTests {

private func engineWithLivePrefetcher() throws -> AetherEngine {
let engine = try AetherEngine()
engine.loadedURL = URL(string: "https://s/movie.mkv")!
engine.softwareSubtitlePacketStore = SubtitlePacketStore()
engine.subtitleDrainTargets[.primary] = 3
engine.subtitleForwardPrefetchTask = Task {}
return engine
}

@Test("AE#1015: no advance reach and no OCR worker means no prefetch, either one brings it back")
func gating() {
#expect(!AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: true, hasSource: true,
advanceReach: 0, ocrArmed: false))
#expect(AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: true, hasSource: true,
advanceReach: 0, ocrArmed: true))
#expect(AetherEngine.shouldRunSubtitleForwardPrefetch(
isLive: false, hasEmbeddedDrainTargets: true, hasSource: true,
advanceReach: 2, ocrArmed: false))
}

@Test("AE#1015: the default reach keeps today's prefetch")
func defaultReachReadsAhead() throws {
let engine = try AetherEngine()
#expect(engine.subtitleAdvanceReach == AetherEngine.subtitleCueBitmapFloorSeconds)
}

@Test("AE#1015: only a negative software delay reads ahead")
func softwareDelaySign() throws {
let engine = try AetherEngine()
engine.maximumSubtitleOffsetSeconds = 0
#expect(engine.subtitleAdvanceReach == 0)
engine.setSoftwareSubtitleDelay(4)
#expect(engine.subtitleAdvanceReach == 0)
engine.setSoftwareSubtitleDelay(-4)
#expect(engine.subtitleAdvanceReach == 4)
}

@Test("AE#1015: declaring zero reach mid-session stops a running prefetch under its own reason")
func zeroReachStopsRunningPrefetch() throws {
let engine = try engineWithLivePrefetcher()
engine.maximumSubtitleOffsetSeconds = 0
#expect(engine.subtitleForwardPrefetchTask == nil)
#expect(engine.lastSubtitleDrainStopReason == .noAdvanceReach)
}

@Test("AE#1015: a positive delay keeps it stopped, a negative one starts it, zero stops it again")
func softwareDelayStartsAndStops() throws {
let engine = try engineWithLivePrefetcher()
engine.maximumSubtitleOffsetSeconds = 0
engine.setSoftwareSubtitleDelay(3)
#expect(engine.subtitleForwardPrefetchTask == nil)
engine.setSoftwareSubtitleDelay(-3)
#expect(engine.subtitleForwardPrefetchTask != nil)
engine.setSoftwareSubtitleDelay(0)
#expect(engine.subtitleForwardPrefetchTask == nil)
#expect(engine.lastSubtitleDrainStopReason == .noAdvanceReach)
}

@Test("AE#1015: without an embedded drain target a reach change starts nothing")
func noTargetNoStart() throws {
let engine = try AetherEngine()
engine.loadedURL = URL(string: "https://s/movie.mkv")!
engine.maximumSubtitleOffsetSeconds = 0
engine.maximumSubtitleOffsetSeconds = 10
#expect(engine.subtitleForwardPrefetchTask == nil)
}
}
5 changes: 3 additions & 2 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1172,12 +1172,13 @@ See [Audio shaping](formats.md#audio-shaping) for what each setting does to the
- Before those, cues starting beyond the decode lead that the drain has not reached yet are dropped, furthest first (after a backward seek, the old position's cues). The drain decodes them again on its way there (AE#929).
- A seek back re-decodes what was dropped.

**`maximumSubtitleOffsetSeconds`** (default 60) sizes how much bitmap cue history the engine keeps:
**`maximumSubtitleOffsetSeconds`** (default 60) sizes how much bitmap cue history the engine keeps, and whether embedded subtitles cost a second read of a remote source:

- the history behind the playhead the 64 MiB budget never evicts, and how far ahead the drain decodes (at most 60 s). Both stay at least 15 s, and `softwareSubtitleDelaySeconds` is always covered.
- Setting it also drops every bitmap cue that ended behind that history at once, instead of keeping them up to the 64 MiB budget. Past the floor a cue serves no offset, and a seek back re-decodes its own window anyway.
- A host without a subtitle offset control should set 0. On a dense PGS track that keeps 15 s of decoded images on each side, instead of a protected 120 s plus 64 MiB of history (AE#929).
- Text tracks are unaffected. Negative and non-finite values are ignored. It takes effect on the next drain tick.
- **0 also turns off the forward subtitle prefetch (AE#1015).** That reader exists only to have cues ready for an advance offset. On a remote MKV or MP4 it downloads the video a second time: a minute of 1080p read 243.6 MB with an embedded SRT selected against 104.5 MB without. With 0, embedded subtitles come from the main reader alone, which covers a zero or positive offset. The prefetch still runs while `softwareSubtitleDelaySeconds` is negative or the OCR worker is armed (PiP, AirPlay, external display on a bitmap track), and it starts or stops at once when either changes.
- The text drain lead is unaffected. Negative and non-finite values are ignored. It takes effect on the next drain tick.

### Native renditions (PiP, AirPlay, external display)

Expand Down
1 change: 1 addition & 0 deletions docs/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -983,6 +983,7 @@ The tap's forward coverage ends at the producer's read position (its forward par
- While a bitmap track's OCR rendition is armed, the lead is raised to 270 s, which clears the OCR worker's own 240 s window by 30 s, so the store actually holds what the worker is about to read.
- So `subtitleCues` holds cues ahead of the playhead for a host-applied ADVANCE sync offset, text and bitmap alike.
- Live sessions skip it: content past the edge does not exist.
- **Only when something reads ahead (AE#1015).** It runs while the host's offset reach is above 0 (`maximumSubtitleOffsetSeconds`, default 60), `softwareSubtitleDelaySeconds` is negative, or the OCR worker is armed. On an interleaved container it downloads the video a second time, so with a reach of 0 the drainer reads from the tap alone.
- It is best-effort. If it cannot open or wedges, the tap-fed behavior above is unchanged.
- **Positioning (#234).** Its positioning seek is anchored on the subtitle stream's own timestamp axis. Left to libavformat's default reference, the anchor follows whichever stream is not fully discarded. The pacing stream that gives the park its control point (#230) would then move it onto video keyframes, and on Matroska that lands in a later cluster and drops a landing cue that starts further back than the destination.
- **Re-anchoring.** A playhead jump re-anchors the running session in place rather than rebuilding it, since a rebuild pays an open, the Matroska cue-index prewarm and a positioning seek, each a bounded range the origin delivers in full.
Expand Down
Loading