Skip to content

Do not unwrap a frame header on_error may have already taken (aborts on corrupt input) - #1496

Open
nruntas wants to merge 1 commit into
memorysafety:mainfrom
nruntas:fix-on-error-double-teardown
Open

Do not unwrap a frame header on_error may have already taken (aborts on corrupt input)#1496
nruntas wants to merge 1 commit into
memorysafety:mainfrom
nruntas:fix-on-error-double-teardown

Conversation

@nruntas

@nruntas nruntas commented Jul 30, 2026

Copy link
Copy Markdown

Summary

on_error in rav1d_submit_frame unwraps f.frame_hdr, but on_error is
also what clears f.frame_hdr — so reaching it twice on the same frame data
panics. It can be reached twice, and because every entry point into this crate
is extern "C", the panic cannot unwind across the ABI boundary: the process
aborts instead of returning an error the caller can handle.

A single corrupted byte in an otherwise valid AV1 bitstream is enough to take
the host application down. We hit this in a media viewer, on a real .mp4
with one byte flipped inside its mdat.

Root cause

rav1d_submit_frame's single-frame-context branch:

if c.fc.len() == 1 {
    let res = rav1d_decode_frame(c, &fc);   // tears the frame data down on failure
    if res.is_err() {
        ...
        let mut f = fc.data.try_write().unwrap();
        on_error(fc, &mut f, ...);          // second teardown of the same data

rav1d_decode_frame has already cleaned up by the time this runs, so
on_error gets frame data whose header is gone. Instrumenting on_error with
#[track_caller] confirms it, reaching it from that call site with
frame_hdr=false seq_hdr=false.

Only n_fc == 1 is affected, which is presumably why it has survived: with
more than one frame context, decode errors route through the task thread's
retval and this branch is never taken. Since
n_fc = min(max_frame_delay, n_threads), any caller pinning
max_frame_delay = 1 — a natural choice for single-shot, deterministic
decoding — selects the broken path.

The change

Check the option rather than unwrapping it. Behaviour is identical wherever the
header exists (i.e. every path that works today); the only case that changes is
the one that currently aborts. on_error clears f.frame_hdr a few lines
later regardless.

Verification

Built a patched rav1d 1.1.0 and re-ran the case that aborted: 160 systematic
mutations of a real AV1 MP4 (truncations, single-byte flips, and 0xFFFFFFFF
written over header fields), decoded with max_frame_delay = 1. Before: abort
at decode.rs:4997. After: every mutation returns cleanly, either a picture or
an error, and no other assertion in our suite moved.

The same one-line change applies to main, which is what this PR targets.

Note

We also worked around it on our side by raising max_frame_delay to 2, so this
PR is not urgent for us — but the abort is reachable by anyone decoding
single-threaded, and an unwinding panic would at least be catchable where an
abort is not.

`on_error` clears `f.frame_hdr` itself, so it must tolerate being reached
when the header is already gone — and it can be. In the `c.fc.len() == 1`
branch of `rav1d_submit_frame`, `rav1d_decode_frame` runs inline and tears
the frame data down on failure; `rav1d_submit_frame` then calls `on_error`
on that same data. The second call unwraps a `None`.

Because every entry point into this crate is `extern "C"`, that panic cannot
unwind across the ABI boundary: it aborts the process rather than surfacing
as an error the caller can handle. A single corrupted byte in an otherwise
valid AV1 bitstream is enough to take the host application down.

Only single-frame-context decoding is affected, which is why it survives in
threaded use: with `n_fc > 1` errors route through the task thread's
`retval` instead. `n_fc = min(max_frame_delay, n_threads)`, so any caller
pinning `max_frame_delay = 1` for single-shot decode selects the broken path.

Checking the option instead of unwrapping it preserves behaviour exactly
wherever the header exists, and turns the abort back into the error the
caller already handles.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant