Do not unwrap a frame header on_error may have already taken (aborts on corrupt input) - #1496
Open
nruntas wants to merge 1 commit into
Open
Do not unwrap a frame header on_error may have already taken (aborts on corrupt input)#1496nruntas wants to merge 1 commit into
nruntas wants to merge 1 commit into
Conversation
`on_error` clears `f.frame_hdr` itself, so it must tolerate being reached when the header is already gone — and it can be. In the `c.fc.len() == 1` branch of `rav1d_submit_frame`, `rav1d_decode_frame` runs inline and tears the frame data down on failure; `rav1d_submit_frame` then calls `on_error` on that same data. The second call unwraps a `None`. Because every entry point into this crate is `extern "C"`, that panic cannot unwind across the ABI boundary: it aborts the process rather than surfacing as an error the caller can handle. A single corrupted byte in an otherwise valid AV1 bitstream is enough to take the host application down. Only single-frame-context decoding is affected, which is why it survives in threaded use: with `n_fc > 1` errors route through the task thread's `retval` instead. `n_fc = min(max_frame_delay, n_threads)`, so any caller pinning `max_frame_delay = 1` for single-shot decode selects the broken path. Checking the option instead of unwrapping it preserves behaviour exactly wherever the header exists, and turns the abort back into the error the caller already handles.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
on_errorinrav1d_submit_frameunwrapsf.frame_hdr, buton_errorisalso what clears
f.frame_hdr— so reaching it twice on the same frame datapanics. It can be reached twice, and because every entry point into this crate
is
extern "C", the panic cannot unwind across the ABI boundary: the processaborts instead of returning an error the caller can handle.
A single corrupted byte in an otherwise valid AV1 bitstream is enough to take
the host application down. We hit this in a media viewer, on a real
.mp4with one byte flipped inside its
mdat.Root cause
rav1d_submit_frame's single-frame-context branch:rav1d_decode_framehas already cleaned up by the time this runs, soon_errorgets frame data whose header is gone. Instrumentingon_errorwith#[track_caller]confirms it, reaching it from that call site withframe_hdr=false seq_hdr=false.Only
n_fc == 1is affected, which is presumably why it has survived: withmore than one frame context, decode errors route through the task thread's
retvaland this branch is never taken. Sincen_fc = min(max_frame_delay, n_threads), any caller pinningmax_frame_delay = 1— a natural choice for single-shot, deterministicdecoding — selects the broken path.
The change
Check the option rather than unwrapping it. Behaviour is identical wherever the
header exists (i.e. every path that works today); the only case that changes is
the one that currently aborts.
on_errorclearsf.frame_hdra few lineslater regardless.
Verification
Built a patched
rav1d1.1.0 and re-ran the case that aborted: 160 systematicmutations of a real AV1 MP4 (truncations, single-byte flips, and
0xFFFFFFFFwritten over header fields), decoded with
max_frame_delay = 1. Before: abortat
decode.rs:4997. After: every mutation returns cleanly, either a picture oran error, and no other assertion in our suite moved.
The same one-line change applies to
main, which is what this PR targets.Note
We also worked around it on our side by raising
max_frame_delayto 2, so thisPR is not urgent for us — but the abort is reachable by anyone decoding
single-threaded, and an unwinding panic would at least be catchable where an
abort is not.