Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ DEEPSEEK_API_KEY=
# Optional. Enables YouTube Data API search for stronger B-roll discovery.
YOUTUBE_API_KEY=

# Optional. Enables licensed portrait stock-image search and download for slideshow campaigns.
# Request a key from https://www.pexels.com/api/documentation/
PEXELS_API_KEY=

# Optional. Transcription backend preference:
# auto | local-whispercpp | openai | youtube
TRANSCRIBE_PROVIDER=local-whispercpp
Expand Down
45 changes: 25 additions & 20 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,13 @@ permissions:
jobs:
verify:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version: [20, 22]

steps:
- name: Check out repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false

- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ matrix.node-version }}

- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
Expand All @@ -36,6 +27,13 @@ jobs:
- name: Install dependencies
run: bun install --frozen-lockfile

- name: Enforce Bun runtime commands
run: |
if rg "#!/usr/bin/env node|['\"]node['\"]" scripts tests; then
echo "Node runtime command found; use Bun or process.execPath." >&2
exit 1
fi

- name: Install ffmpeg (smoke tests render real MP4s)
run: sudo apt-get update && sudo apt-get install -y ffmpeg imagemagick

Expand All @@ -59,11 +57,6 @@ jobs:
with:
persist-credentials: false

- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22

- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
Expand All @@ -79,12 +72,29 @@ jobs:
tar -tzf .pack/clipcaptionai-*.tgz > pack-report.txt
bun - <<'EOF'
const files = require('node:fs').readFileSync('pack-report.txt', 'utf8').trim().split('\n');
const required = ['scripts/ebay/', 'scripts/logo/', 'bin/clipcaptionai.js', 'docs/WORKFLOWS.md'];
const required = [
'scripts/ebay/',
'scripts/logo/',
'scripts/adapters/stock.adapter.mjs',
'scripts/marketing/',
'templates/rotato/',
'bin/clipcaptionai.js',
'docs/MARKETING_PLATFORM.md',
'docs/VOICE_LIBRARY.md',
'docs/WORKFLOWS.md',
'desktop/worker/progress.mjs',
];
const missing = required.filter((p) => !files.some((f) => f.includes(p)));
if (missing.length > 0) {
console.error('Tarball is missing advertised files:', missing);
process.exit(1);
}
const tarball = require('node:fs').readdirSync('.pack').find((name) => name.endsWith('.tgz'));
const bin = require('node:child_process').execFileSync('tar', ['-xOf', `.pack/${tarball}`, 'package/bin/clipcaptionai.js'], {encoding: 'utf8'});
if (!bin.startsWith('#!/usr/bin/env bun\n')) {
console.error('Published CLI entrypoint must run with Bun.');
process.exit(1);
}
console.log('Tarball OK:', files.length, 'entries packed');
EOF

Expand All @@ -96,11 +106,6 @@ jobs:
with:
persist-credentials: false

- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22

- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ outputs/*
!outputs/.gitkeep
projects/*
!projects/.gitkeep
campaigns/*
!campaigns/.gitkeep
*.log
links.txt
broll-prompts*.txt
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@

All notable changes to ClipCaptionAI are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); the project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.2.0] — 2026-08-21

### Added
- Automatic repo-owned adapter discovery shared by the CLI and Electron desktop app
- Filesystem-backed detached jobs with logs, cancellation, recovery, and resource locks
- Remotion, Ollama, yt-dlp, workflow, Rotato, Higgsfield, capture, marketing, and Pexels stock adapters
- Generic campaign planning, cost estimation, approval, execution, inspection, technical QA, and export
- Native value-first carousels that render one provenance-tracked PNG per declared slide
- Reusable ElevenLabs phrase-library workflow and subject-aware portrait framing plans

### Changed
- Standardized development, commands, CI, and packaging on Bun/Bunx
- Desktop now observes the same adapter catalog and job store used by agents
- Marketing QA keeps technical, claims, visual-review, and publication states separate

### Security
- Paid generation requires matching plan approval, capability fingerprints, explicit live execution, and budget compliance
- Stock assets retain provider, creator, source, license, and content-hash provenance

## [0.1.1] — 2026-08-20 — production-readiness pass

### Added
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ On macOS with Homebrew:
brew install oven-sh/bun/bun ffmpeg
```

On Windows or Linux, install Node.js 20+ and FFmpeg using your normal package manager or the official installers, then continue with `bun run doctor` below.
On Windows or Linux, install Bun 1.3+ and FFmpeg using your normal package manager or the official installers, then continue with `bun run doctor` below.

Optional extras:

Expand Down Expand Up @@ -67,7 +67,7 @@ bun run clipkit -- video run \
--run-id first-video
```

When installed from the v0.1.0 release tarball, the equivalent clean first-run command is:
When installed from the v0.2.0 release tarball, the equivalent clean first-run command is:

```bash
clipcaptionai video run --example --run-id first-video
Expand Down Expand Up @@ -160,7 +160,7 @@ The menu is convenient for interactive editing. The direct `bun run clipkit -- .

`bun run desktop` starts an Electron shell that loads the same adapter catalog and observes the same persisted jobs as the CLI. Work continues if the window closes.

- Required: Node.js + `ffmpeg` + `ffprobe` + project CLI/runtime files
- Required: Bun + `ffmpeg` + `ffprobe` + project CLI/runtime files
- Optional: `yt-dlp`, Ollama; AI provider auto mode prefers local Ollama and falls back to configured DeepSeek/OpenAI
- Generic marketing campaigns use the same discovered adapters and job broker; see [Marketing platform](docs/MARKETING_PLATFORM.md).

Expand Down Expand Up @@ -210,6 +210,7 @@ Detailed walkthroughs for every workflow live in [docs/WORKFLOWS.md](docs/WORKFL
| `bun run transcribe:benchmark` | Compare local vs reference transcription providers | [Transcription Notes](docs/WORKFLOWS.md#transcription-notes) |
| `bun run smart:clips` | AI clip selection on one local video | [AI Clip Selection](docs/WORKFLOWS.md#ai-clip-selection) |
| `bun run render:clip` | Render one clip from captions JSON | [Single Clip Commands](docs/WORKFLOWS.md#single-clip-commands) |
| `bun run portrait:analyze` | Plan subject-aware 9:16 framing | [Subject-aware portrait framing](docs/WORKFLOWS.md#subject-aware-portrait-framing) |
| `bun run render:batch` | Batch render clips from a captions folder | — |
| `bun run rerender:clip` | Rerender after caption/fix edits | [Rerender](docs/WORKFLOWS.md#rerender) |
| `bun run moments:review` | Viral scorecard report for a moments run | [Find Important Moments Only](docs/WORKFLOWS.md#find-important-moments-only) |
Expand All @@ -226,6 +227,7 @@ Detailed walkthroughs for every workflow live in [docs/WORKFLOWS.md](docs/WORKFL
| `bun run ebay:render-blueprint-ad` / `ebay:render-blueprint-batch` | Product-safe preview ads from blueprints | [Competitive eBay Creative Blueprints](docs/WORKFLOWS.md#competitive-ebay-creative-blueprints) |
| `bun run ebay:competitive-*` | Post-blueprint pipeline: `competitive-loop`, `competitive-qa`, `prep-premium-renders`, `competitive-handoff`, `competitive-higgsfield-render`, `competitive-packets`, `competitive-research-queue/import/loop/process/rerun`, `collect-premium-renders`, `finalize-premium-ads`, `competitive-status`, `competitive-review` | [Competitive eBay Creative Blueprints](docs/WORKFLOWS.md#competitive-ebay-creative-blueprints) |
| `bun run voiceover:elevenlabs` | ElevenLabs narration file | [Demo Capture And Reviewed AI Assets](docs/WORKFLOWS.md#demo-capture-and-reviewed-ai-assets) |
| `bun run voiceover:library` | Build or resume a reusable ElevenLabs phrase library | [Voice Library](docs/VOICE_LIBRARY.md) |
| `bun run fal:image-edit` / `fal:reference-video` | fal.ai asset generation (opt-in, human-reviewed) | [Demo Capture And Reviewed AI Assets](docs/WORKFLOWS.md#demo-capture-and-reviewed-ai-assets) |
| `bun run sample:props` | Write Remotion Studio sample props | [Preview In Remotion Studio](docs/WORKFLOWS.md#preview-in-remotion-studio) |
| `bun run cleanup` | Clean temp files / old outputs | [Clean Up Generated Files](docs/WORKFLOWS.md#clean-up-generated-files) |
Expand All @@ -238,6 +240,7 @@ See `package.json` scripts for the full list (including `rotato`, `interview:qa`

- [docs/WORKFLOWS.md](docs/WORKFLOWS.md) — every workflow walkthrough, in depth
- [docs/AI_PROVIDERS.md](docs/AI_PROVIDERS.md) — provider keys, review gates, live-provider evidence
- [docs/VOICE_LIBRARY.md](docs/VOICE_LIBRARY.md) — restore or extend the released ElevenLabs phrase library
- [docs/AGENT_GUIDE.md](docs/AGENT_GUIDE.md) — automation guide for coding agents
- [docs/PRODUCTION_SUPPORT.md](docs/PRODUCTION_SUPPORT.md) — production support matrix
- [docs/GITHUB.md](docs/GITHUB.md) — GitHub-specific setup
Expand Down Expand Up @@ -286,7 +289,7 @@ bun test tests/ai-provider.test.mjs
bun test tests/clipkit-lib.test.mjs
```

Tests use Node's built-in test runner. Integration tests create temp dirs, run the actual CLI, generate real MP4s, and clean up. They skip gracefully when ffmpeg/ImageMagick are absent. `bun run check` runs typechecking plus the full test suite.
Tests use Bun's test runner. Integration tests create temp dirs, run the actual CLI, generate real MP4s, and clean up. They skip gracefully when ffmpeg/ImageMagick are absent. `bun run check` runs typechecking plus the full test suite.

## Related Projects

Expand Down
2 changes: 1 addition & 1 deletion bin/clipcaptionai.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/usr/bin/env node
#!/usr/bin/env bun
process.env.CCA_WORKSPACE_ROOT ||= process.cwd();
await import('../scripts/clipkit.mjs');
1 change: 1 addition & 0 deletions campaigns/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

8 changes: 8 additions & 0 deletions docs/AI_PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ bun run voiceover:elevenlabs -- \

Writes MP3 audio and a generation manifest with voice/model IDs, text hash, and request ID. Never writes the key or narration text into the manifest.

Build or resume the reusable phrase library within an explicit character budget:

```bash
bun run voiceover:library -- --budget 36000 --resume
```

The command writes one MP3 and non-secret manifest per phrase, does not retry ambiguous paid generation requests, checks the live subscription balance, and preserves a safety reserve. Use `--dry-run` before spending credits. Generated audio still requires human review for pronunciation, tone, and licensing suitability; see [the voice library guide](VOICE_LIBRARY.md).

### fal reviewed marketing assets

```bash
Expand Down
36 changes: 36 additions & 0 deletions docs/ASSET_RECOVERY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Asset Recovery Bundle

The `asset-recovery-2026-07-23` GitHub Release preserves the verified self-created ClipCaptionAI B-roll cards and the local public-source SFX library.

The B-roll bundle contains:

- `plan.svg`
- `render.svg`
- `qa.svg`

The separate `clipcaptionai-sfx-library.tar.gz` release asset contains all 256 local SFX files plus `sfx-library/index.json`.

Restore them from a fresh checkout with:

```bash
mkdir -p /tmp/clipcaptionai-asset-recovery
gh release download asset-recovery-2026-07-23 \
--repo jongan69/ClipCaptionAI \
--pattern 'clipcaptionai-cleared-assets.tar.gz' \
--dir /tmp/clipcaptionai-asset-recovery
tar -xzf /tmp/clipcaptionai-asset-recovery/clipcaptionai-cleared-assets.tar.gz \
-C .
```

To restore the SFX library as well:

```bash
gh release download asset-recovery-2026-07-23 \
--repo jongan69/ClipCaptionAI \
--pattern 'clipcaptionai-sfx-library.tar.gz' \
--dir /tmp/clipcaptionai-asset-recovery
tar -xzf /tmp/clipcaptionai-asset-recovery/clipcaptionai-sfx-library.tar.gz \
-C .
```

The SFX files are preserved because they are publicly available local assets, but public availability is not the same as verified commercial-use clearance. Review source terms before publishing a video commercially. The `music-library/` manifest explicitly marks its tracks `review_before_commercial_use`, so music remains excluded. The downloaded YouTube/movie `scene-library/` is intentionally excluded until its rights are reviewed.
40 changes: 40 additions & 0 deletions docs/MARKETING_PLATFORM.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,46 @@ clipcaptionai marketing export --run <run-id> --wait

Approval binds the plan hash, installed-tool capability fingerprint, CLI-derived estimate hash, and total credit budget. Any plan, estimate, adapter, CLI-help, or model change invalidates it. Live generation intents provide current installed-CLI cost argv and submission argv; paid submission additionally requires `--live-execution`. `--dry-run` never submits. Technical QA reports decoding, format, timing, stream, black-frame, silence, caption-zone, CTA, capture-freshness, and Rotato-template checks separately from claims, human visual review, and publication approval.

`execute` resolves the campaign timeline, renders the `MarketingTimeline` Remotion composition, normalizes audio to -16 LUFS by default, and registers a content-hashed final MP4. Timeline videos can be trimmed, muted, volume-adjusted, and fit for the placement; campaigns can add a `voice` narration track, a separate `music` bed, and brand colors. Set `audioTargetLufs` per variant when a delivery profile needs another target. `qa` requires that final artifact and never substitutes a source, generated, capture, or Rotato intermediate.

Product capture is command-only: manifests own argv arrays, cwd, outputs, seed, repository commit, and device profile. No shell interpolation or GUI automation is used.

## Native value-first slideshows

Slideshow campaigns use the same plan, budget, job, render, QA, and export flow as video campaigns. A variant declares `slides` instead of a raw `timeline`. Set `format: carousel` to render one finished PNG per slide, or retain the default `format: video` to expand the slides into an animated video with a CTA end card.

```yaml
variants:
- id: health-week-saveable
format: carousel
cta: Review your day with PrepAI
slides:
- src: /absolute/path/from/stock-manifest.json
eyebrow: SAVE THIS
headline: 5 realistic ways to support your health this week
body: Pick one that fits your needs and routine.
durationSeconds: 2.2
motion: push-in
sourceType: stock
attribution:
provider: pexels
creator: Photographer name
creatorUrl: https://www.pexels.com/@photographer
sourceUrl: https://www.pexels.com/photo/123
licenseUrl: https://www.pexels.com/license/
```

`motion` supports `push-in`, `pan-left`, and `pan-right` for video output; `textPosition` supports `top`, `center`, and `bottom`. Carousel output preserves the declared slide count exactly. Stock slides must include creator, source, provider, and license metadata. `execute` registers those source assets and `qa` fails the `stock-provenance` check when the record is incomplete.

Acquire images through the shared adapter before authoring the campaign:

```sh
clipcaptionai stock doctor
clipcaptionai stock download --query "healthy morning walk sunlight" --count 8 --out outputs/stock/health-week --wait --json
```

The adapter requests portrait Pexels originals using the API's `orientation=portrait` and `size=large` filters, then rejects images below 1080×1920. The manifest is the source of truth when an agent fills slide paths and attribution. Reuse the downloaded library across hook, ordering, wording, pacing, and CTA variants instead of downloading duplicate images.

Value slides must remain truthful and legible. Health campaigns require content/claims review; do not invent supplement, treatment, skin, weight-loss, or outcome claims. Stock subjects cannot be presented as customers or as endorsing the product. Promotion may be low-pressure, but it must remain identifiable rather than disguised as independent advice. Pexels downloads are per-campaign creative inputs, not a bulk collection or AI-training dataset; preserve the manifest's Pexels and photographer links.

Rotato templates live at `templates/rotato/<id>/template.json` beside their real `scene.rotato`. Generate `inspectFingerprint` from the current `rotato inspect <scene> --json` result. Semantic slots compile to inspected device indexes and overlay IDs; drift fails with `TEMPLATE_INSPECT_MISMATCH`.
16 changes: 8 additions & 8 deletions docs/RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# ClipCaptionAI v0.1.0 Beta
# ClipCaptionAI v0.2.0

ClipCaptionAI v0.1.0 is a CLI-first public beta. The supported first-run path is local and deterministic: it does not require a paid AI provider or an API key.
ClipCaptionAI v0.2.0 is a CLI-first video-production toolkit with an optional Electron desktop observer. Its supported first-run path is local and deterministic; provider-backed workflows are opt-in.

## Install from the release tarball

Prerequisites are Node.js 20 or newer plus `ffmpeg` and `ffprobe` on `PATH`.
Prerequisites are Bun 1.3 or newer plus `ffmpeg` and `ffprobe` on `PATH`.

```bash
bun install --global ./clipcaptionai-0.1.0.tgz
bun install --global ./clipcaptionai-0.2.0.tgz
clipcaptionai doctor
clipcaptionai video run --example --run-id first-video
clipcaptionai video qa --run outputs/video-runs/first-video
Expand All @@ -19,10 +19,10 @@ The finished MP4 and its versioned QA manifest are written below the current dir

| Platform | CLI beta | Desktop beta |
| --- | --- | --- |
| macOS 13+ on Apple silicon | Supported and release-smoked | Best effort; attached only when signed installation passes |
| macOS 13+ on Intel | Expected with Node.js and FFmpeg; not release-smoked | Not included in v0.1.0 |
| Windows 10/11 x64 | Expected with Node.js and FFmpeg; not release-smoked | Not included in v0.1.0 |
| Current x64 Linux | Expected with Node.js and FFmpeg; not release-smoked | Not included in v0.1.0 |
| macOS 13+ on Apple silicon | Supported and release-smoked | Attached when signing and packaging pass |
| macOS 13+ on Intel | Expected with Bun and FFmpeg; not release-smoked | Not included in v0.2.0 |
| Windows 10/11 x64 | Expected with Bun and FFmpeg; not release-smoked | Not included in v0.2.0 |
| Current x64 Linux | CI verified with Bun and FFmpeg | Build verified; installer not included in v0.2.0 |

## Optional providers

Expand Down
2 changes: 2 additions & 0 deletions docs/ROTATO-INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ clipcaptionai rotato render /path/to/scene.rotato --screen-media /path/to/captur

Rendering always runs `inspect --json` first. Template folders contain a real `scene.rotato` and `template.json`; semantic slots compile to inspected device indexes and overlay IDs. A changed inspect fingerprint or missing mapped ID fails with `TEMPLATE_INSPECT_MISMATCH`. `--screen-media` and `--screen-media-for` are mutually exclusive.

Use `clipcaptionai rotato templates --json` to discover the active local template library and its semantic screen slots. Set `CCA_ROTATO_TEMPLATES_ROOT` when the library lives outside the repository.

The wrapper uses safe argv execution and preserves Rotato app handoff, timeout, codec, size, quality, and wait flags. It never persists overlay mutations. Completed files are hashed and probed. `rotato raw` remains available for advanced debugging through the same safe argv path.

Rotato rendering success is not visual approval or publication readiness; marketing QA records those as separate states.
Loading
Loading