Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ opendubstage qc mix.wav --md qc.md # loudness, true peak, LRA, dead ai
| Spectral repair | Fill coughs, thumps and handling noise with the recording's own room tone; attenuate events under speech | `opendubstage specrepair` |
| De-rumble | Take car rumble and room boom out of a voice memo without touching the voice above 400 Hz | `opendubstage memofix` |
| Dialogue EQ match | Match one voice's long-term spectrum to another's with a gain-limited EQ, emitted as an `ffmpeg` filter chain | `opendubstage ltasmatch` |
| Dialogue leveller | Ride whole phrases from word timings, with pause-only ramps and explicit breath policy | `opendubstage leveller` |
| Music carve | Dip the music's 1–4 kHz band while someone speaks; bring music up in the gaps (posts); end cues on a phrase | `opendubstage carve` |
| Transitions | J-cuts, L-cuts and equal-power crossfades at chapter boundaries, on a cue sheet, without moving sync | `opendubstage transitions` |
| Mix QC | BS.1770-4 integrated loudness, true peak, LRA, dead air, clicks at edits, voice level jumps, music masking speech | `opendubstage qc` |
Expand All @@ -55,8 +56,7 @@ opendubstage qc mix.wav --md qc.md # loudness, true peak, LRA, dead ai
diff the result. It is not a replacement for RX on a difficult track. It is the batchable, checkable version of the
routine pass.
- **Auphonic** is a hosted service that levels, normalises loudness and reduces noise in one upload. OpenDubStage runs
offline, uploads nothing, and splits the work into separate steps with their parameters exposed. It has no dialogue
leveller yet; that is first on the [roadmap](ROADMAP.md).
offline, uploads nothing, and splits the work into separate steps with their parameters exposed. Its experimental [dialogue leveller](docs/leveller.md) uses explicit word timings; it does not infer speech or replace listening review.
- **pyloudnorm** is a BS.1770 loudness meter and normaliser library. `qc` measures gated integrated loudness and true
peak the same way, then adds what a broadcast QC pass asks for on top: LRA, dead air, clicks at cue edges, level
jumps between voices and masking of speech in the presence band, with blocking and advisory checks kept separate.
Expand All @@ -72,12 +72,13 @@ opendubstage qc mix.wav --md qc.md # loudness, true peak, LRA, dead ai
| | `memofix` | Removes rumble from a voice memo recorded in a car (or a boomy room). It learns a noise profile from the gaps between words, subtracts below 400 Hz only, then applies a gentle, clamped EQ toward the international long-term average speech spectrum. Nothing is resynthesised. |
| `opendubstage.match` | `ltasmatch` | Dialogue EQ match. It compares long-term average spectra on speech frames, then fits a smoothed, gain-limited set of peaking bands. The output is a plain `ffmpeg` `equalizer` chain, plus an early-reflections helper for putting a dry voice in a small room. |
| `opendubstage.mix` | `carve` | Speech-keyed dynamic EQ: the music's 1–4 kHz band dips while dialogue is active. It also provides *posts*, where music comes up in the gaps between lines and dips just before the next word, and *phrase-safe* music outs on a section, phrase or downbeat. |
| | `leveller` | Experimental phrase-constant gain rides, linked across channels, with sample-peak headroom caps and JSON audit report. See [method and timing schema](docs/leveller.md). |
| | `transitions` | A cue-sheet pass at chapter boundaries. It adds ambience pre-laps (J-cuts), tails (L-cuts) and equal-power music crossfades, and puts a short ambience gap between two foreground songs. It never moves a sync point, touches dialogue or extends a cue past its source. |
| `opendubstage.qc` | `qc` | Public-radio-style mix QC: integrated loudness (BS.1770-4 gating), true peak, LRA, dead air, clicks at cue edges, level jumps between voices, music/ambience masking of speech in the presence band, hard stops and tonal balance per voice class. Blocking and advisory checks are kept separate. |
| `opendubstage.foley` | `foley` | Sound for stills and archival footage. It analyses each picture, proposes sounds, then applies documentary placement rules (levels relative to dialogue, density caps, never on the cut, key-line protection, no-sound classes). The output is an auditable cue sheet. Library and local-model backends are pluggable. See [docs/foley.md](docs/foley.md). |
| `skills/` | | Agent skills for documentary sound design; see [skills/README.md](skills/README.md). |

The planned tools (dialogue leveller, room-tone synthesis, de-reverb, a true-peak limiter helper and an MLX VoxCPM LoRA
The planned tools (room-tone synthesis, de-reverb, a true-peak limiter helper and an MLX VoxCPM LoRA
training port) are listed in [ROADMAP.md](ROADMAP.md).

## Install
Expand Down
9 changes: 8 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,14 @@ plus objective checks, never by a metric alone.

## 0.2: planned tools

### Dialogue leveller
### Dialogue leveller (experimental word-timing implementation)

Implemented as `opendubstage leveller`; see [method, defaults and limitations](docs/leveller.md).
Synthetic invariants cover convergence when limits allow, word-safe gains and exact untouched samples.
VAD input, real-world listening validation and a release remain pending.
The current phrase meter is ungated K-weighted mean square, not standardized three-second short-term loudness.

Original goal:
A sentence-level leveller in the spirit of a dialogue-leveller plug-in, as opposed to a compressor:
- Measure short-term loudness per phrase (from word timings or a VAD) and ride each phrase toward a target with slow,
phrase-aligned gain moves. Gain changes happen in pauses, never mid-word.
Expand Down
74 changes: 74 additions & 0 deletions docs/leveller.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Experimental dialogue leveller

`opendubstage leveller input.wav output.wav --timings words.json --report report.json`

This offline gain rider takes explicit word timings. It does not detect speech, transcribe,
compress, denoise, or send audio anywhere. Input and output are mono/stereo PCM or floating-point
WAV files, with the input sample rate and subtype preserved. The full recording is held in RAM.

## Timing schema

```json
{
"words": [{"start": 0.5, "end": 1.0}, {"start": 1.1, "end": 1.8}],
"breaths": [{"start": 2.0, "end": 2.2}]
}
```

Times are seconds relative to sample zero. Each list must be ordered and non-overlapping,
within the recording, and have positive durations. Breaths must not intersect words.
Fractional sample edges round outward to protect words. Extra fields such as transcript
text are ignored and not copied into the report. Empty word lists are valid.

## Method and defaults

Words separated by at most `--phrase-gap 0.3` seconds form one phrase. An explicit breath
splits a phrase even when the gap is short. All channels receive the same gain, held constant
from the first word's start to the last word's end, including internal gaps. The meter resets
at each phrase and measures ungated K-weighted mean-square energy with mono/stereo channel
summing. This is a useful phrase descriptor, **not** EBU standardized 3-second short-term,
gated integrated loudness, or a claim of delivery compliance.

- `--target -24`: phrase target, inspired by A/85 dialogue level as a starting point.
- `--min-gain -12`, `--max-gain 6`: conservative gain bounds, in dB.
- `--tolerance 0.5`: leave phrases already this close to target untouched.
- `--floor -60`: do not boost near-silent phrases. Digital silence remains unchanged.
- `--ramp 0.15`: raised-cosine ramps in dB before/after each event, entirely outside words
and breaths. Each ramp requires the configured duration (rounded up, at least two samples),
using at most half the available gap. If a full ramp cannot fit at either internal
boundary, the entire event stays unchanged, with `insufficient_pause` recorded.
Thus a 150 ms ramp requires at least 300 ms of gap on that side. Farther gaps stay
untouched; recording start/end need no outside ramp. Choose shorter ramps explicitly
only after listening; the algorithm never silently substitutes a sharp transition.
- `--peak-ceiling -1`: cap positive gain against the maximum sample peak across the whole
affected phrase and ramp support. Existing over-ceiling material is not globally repaired.
This is a sample-peak guard, **not** a true-peak limiter; use `opendubstage qc` on the result.
- `--breath-policy keep`: keep explicitly marked breath samples bit-exact (the default).
`--breath-policy attenuate --breath-db -6` instead applies fixed attenuation, with the same
pause-only ramp protection. Unmarked breaths inside phrases receive that phrase's gain.

Other than the target's broadcast inspiration, these defaults are engineering starting
points chosen for restraint, not perceptually validated universal settings. In particular,
phrase boundaries and breath timings need review. When gain/headroom limits bind, the target
is deliberately not reached. The synthetic tests establish signal properties, not perceived
improvement; blind listening review is still needed before use on productions.

## Report and exactness

The JSON report uses schema `opendubstage.leveller/0.1`. It includes parameters, per-event
sample bounds and ramp supports, requested/applied gains, gain/headroom-limit flags,
before/after phrase loudness, and input/output sample peaks. Digital silence is represented
by `null`, never non-standard JSON infinity. All sample ranges are half-open `[start, end)`.
`changed_ranges` describes non-unity gain; `untouched_ranges` its complement. A changed range
can contain zero samples that numerically remain zero.

Measurements are made before WAV encoding. Unchanged decoded PCM/float samples are preserved,
not the entire WAV container or metadata. Changed PCM samples are quantized to the original
subtype. No-op does not mean "normalize the whole file." The API returns a new array and never
modifies the source. The CLI requires distinct input, output, timings and report paths; like
other tools it can overwrite existing output destinations, so choose fresh filenames.

```python
from opendubstage.mix.leveller import level
processed, report = level(audio_float_array, sample_rate, timings_dict)
```
1 change: 1 addition & 0 deletions opendubstage/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

# tool -> (module, one-line help). Modules are imported lazily so `--help` stays fast and light.
TOOLS = {
"leveller": ("opendubstage.mix.leveller", "phrase-safe dialogue gain rides from word timings"),
"specrepair": ("opendubstage.restore.specrepair", "spectral repair: room-tone replace, attenuate, LF thumps"),
"memofix": ("opendubstage.restore.memofix", "remove car/room rumble learned from word gaps, LTASS matching EQ"),
"ltasmatch": ("opendubstage.match.ltasmatch", "dialogue EQ match: fit an ffmpeg equalizer chain from LTAS"),
Expand Down
9 changes: 6 additions & 3 deletions opendubstage/foley/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,10 +176,13 @@ def room_of(s: Still, a: SceneAnalysis) -> str | None:
xf = r.bed_crossfade_s if i > 0 else 0.0
span = Interval(max(0.0, run[0].start - xf), run[-1].end)
for part in _subtract(span, list(seq.silences)):
sheet.cues.append(Cue(nid("b"), Layer.BED, bed, part.start, part.dur,
dur = part.dur
while part.start + dur > part.end: # preserve silence boundaries after float subtraction/addition
dur = math.nextafter(dur, 0.0)
sheet.cues.append(Cue(nid("b"), Layer.BED, bed, part.start, dur,
target_lufs(Layer.BED, seq.dialogue_lufs),
fade_in=min(r.bed_crossfade_s, part.dur / 2),
fade_out=min(r.bed_crossfade_s if i < len(runs) - 1 else 2.0, part.dur / 2),
fade_in=min(r.bed_crossfade_s, dur / 2),
fade_out=min(r.bed_crossfade_s if i < len(runs) - 1 else 2.0, dur / 2),
prompt=bed_prompt(bed, tod), room=ROOM_FOR_BED.get(bed)))

# 2. transitions: hand-requested only, <= N per chapter run, straddling the cut
Expand Down
Loading
Loading