Local story-to-video automation for long-form YouTube videos and cliffhanger Shorts.
Each story gets exactly one user-facing workspace:
runs/2026-08-16_16-27-short-story-title/
story.md
narration.wav
transcript.json
captions.ass
full.mp4
short.wav
short-transcript.json
short-captions.ass
short.mp4
thumbnail.png
runs/ is local and gitignored. Temporary model/runtime work remains under .work/.
The intended production flow is:
- Generate the story.
- Review/refine it.
- Save only the refined script to
story.md. - Generate narration once. Fish Audio S2 Pro is the default; Gemini and VibeVoice remain optional overrides.
- The reviewed story contains exactly one
[[SHORTS_CLIFFHANGER]]marker on its own line after the chosen cliffhanger sentence. - The pipeline strips that marker before TTS, transcribes once with WhisperX, and renders
full.mp4. - It aligns the pre-marker story words to
transcript.json, derives the cutoff timestamp programmatically, and rendersshort.mp4automatically. TTS and WhisperX are not rerun. - Save the final thumbnail as
thumbnail.pngin the same run folder.
There are no root input/ or output/ directories in the new workflow.
The story skills now live in this repository under skills/:
reddit-story- story coordinator.reddit-story-genres- genre-specific behavior.reddit-story-hooks- opening-hook guidance.reddit-story-review- refinement pass.reddit-story-thumbnail- thumbnail generation guidance.reddit-story-video- minimal instructions for driving this local pipeline and producing the full video + Short.
Only these three providers are maintained:
- Gemini TTS - API-based narration, up to two speakers.
- Microsoft VibeVoice 1.5B - local long-form narration with reusable presets.
- Fish Audio S2 Pro - local native Windows
s2.cpp, full unquantized F16 model, 20 CUDA transformer layers by default.
For Fish, production stories carry non-spoken gender=male / gender=female speaker metadata. The pipeline auto-casts male speakers to Ethan and female speakers to Sarah. --speaker-preset ID=PRESET is an optional override. Gemini and VibeVoice still require explicit per-speaker voices.
Requirements: Windows, Python 3.11, uv, Git, FFmpeg/FFprobe, and an NVIDIA GPU for the local accelerated paths.
.\setup.ps1For Gemini, set GOOGLE_API_KEY in .env.
Create a run first:
.\.venv\Scripts\python.exe main.py new-run --title "my story"Write the reviewed story to the printed folder's story.md. For unattended or scheduled production, enqueue the run instead of waiting for the heavy pipeline in the caller session:
.\.venv\Scripts\python.exe main.py enqueue `
--run-dir runs\2026-08-16_16-27_my-storyenqueue ensures a detached worker is alive, sends the run to that worker's in-memory FIFO queue over localhost IPC, and returns immediately. You can enqueue any number of runs while that worker is alive, and it processes exactly one video at a time. When the queue drains the worker exits; its queue disappears with it. A later worker always starts with an empty queue, so stale jobs are never recovered after a crash or restart. Per-job logs remain under .work/video-queue/logs/, but pending/running queue state is not persisted.
.\.venv\Scripts\python.exe main.py queue-statusUse main.py run only when you intentionally want a synchronous foreground generation, such as manual debugging.
Gemini synchronous example:
.\.venv\Scripts\python.exe main.py run `
--run-dir runs\2026-08-16_16-27_my-story `
--tts gemini `
--speaker-preset 0=Kore `
--speaker-preset 1=PuckVibeVoice synchronous example:
.\.venv\Scripts\python.exe main.py run `
--run-dir runs\2026-08-16_16-27_my-story `
--tts vibevoice `
--speaker-preset 0=Alice `
--speaker-preset 1=FrankDefault Fish synchronous example:
.\.venv\Scripts\python.exe main.py run `
--run-dir runs\2026-08-16_16-27_my-storyThe story metadata decides whether each slot uses the male or female default voice. Add --speaker-preset ID=PRESET only when you intentionally want to override that automatic casting.
Both queued and synchronous runs create short.mp4 automatically when story.md contains [[SHORTS_CLIFFHANGER]]. There is no separate AI cliffhanger-selection step and no manual timestamp argument.
main.py list shows existing runs, backgrounds, caption themes, VibeVoice presets, and saved Fish presets.
.\.venv\Scripts\python.exe app.pyThe UI now writes through the same run-folder pipeline. The default full-video mode keeps the background source dimensions.
app.py
main.py
setup.ps1
skills/
reddit_video/
pipeline.py
job_queue.py
runs.py
captions.py
tts.py
fish.py
tts_text.py
gemini-tts/
videos/
voice_presets/
patches/
runs/ # generated, gitignored
.work/ # internal, gitignored
vendor/ # runtimes/models, gitignored
.\.venv\Scripts\python.exe -m pytest -q