researchowl/docs/shortsmith-phases-4-6.md

# Phases 4–6 — more elaborate Shorts

**Roadmap document.** Phases 1–3 are live: spec generation with grounding (phase 2),
render via shortsmith, YouTube upload as private with a human publish click (phase 3).
Today shortsmith renders **silent** vector motion graphics from 8 templates
(`radar_sweep`, `track_map`, `data_card`, `scale_bars`, `orbit_track`, `signal_strips`,
`document_quote`, `counter_close`).

- **Repos touched:** mostly `git.chemavx.xyz/chemavx/shortsmith`. researchowl changes are
  small and called out explicitly per phase — the live-contract design (`GET /templates`
  injected into the prompt at generation time) means new templates and new spec fields
  reach the generator with little or no code here.
- **Contract rule for every phase:** additive and optional. An existing valid spec must
  stay valid; a pod running old researchowl must keep producing renderable specs. No
  breaking field renames, ever.

---

## 0. The gate — data before work

Do not start phase 4 until 3–4 real Shorts are published and have a week of YouTube
Analytics. The decision is not aesthetic, it is a retention curve:

| Signal in Analytics | Diagnosis | Order |
|---|---|---|
| Viewers swipe in the first 1–2 s | Silent open kills the hook | Phase 4 first |
| Retention holds, then decays evenly | Format works, ceiling is production value | Phase 5 first |
| Retention fine, views low | Distribution problem, not video problem | Neither — titles/tags/posting time |

Four videos of data beat any amount of a priori taste. The phases below are ordered by
the expected outcome (audio first), but the gate can reorder them.

---

## Phase 4 — sound

> **Corrected after reading the shortsmith source** (the first draft of this section
> assumed silent renders and proposed royalty-free tracks on disk — both wrong).
> shortsmith has synthesized audio by design: numpy only, no samples, no licensing
> exposure, deterministic to the sample, with the spec's `audio: {preset, silence}`
> block and the validated `sonar` composition guarded by a reference gate. The right
> 4a for that architecture is a wider palette of synthesized presets, not files.

### 4a. The preset palette — **shipped 2026-08-06**

- shortsmith: `pulse` (sub-bass heartbeat tightening past the midpoint — tension, for
  debunks) and `static` (shortwave noise bed, seeded crackles, faint drone — document
  drops) joined `sonar` and `none`. Same arc grammar (nothing starts inside a `silence`
  window, closing swell, same limiter chain); `static`'s noise comes from seeded legacy
  `RandomState` streams, which NEP 19 froze — deterministic on any numpy. `GET /audio`
  publishes the palette with one-line mood notes.
- researchowl: the client fetches the palette (404 → the baseline pair, fallback
  convention), `validate_spec` takes it as the live source of truth for
  `audio.preset`, and the prompt offers it with the mood notes so the model matches
  preset to narrative shape. The edit loop accepts it too: changing `audio.preset` in
  the `/short_spec` file and re-sending is the free way to audition the palette.

### Deferred from 4a: shot-boundary transition accents

A synthesized whoosh at each cut. Deliberately not done yet: `audio.py`'s presets are
fixed compositions that know nothing about the shots, and that one-way line is worth
keeping until the palette itself proves its value. If it happens, it is an opt-in
`Audio.transitions` flag with boundaries passed alongside `silence` — additive, and
`sonar`'s reference gate must not notice.

### 4b. Narration (TTS) + burned-in captions — **shipped 2026-08-06**

What actually landed, and where it differs from the plan below:

- **Piper as a standalone binary**, not the PyPI package: `piper-phonemize` is a
  compiled extension whose wheels chase the interpreter version, and the image's
  Python is pinned by digest. Binary, voice and config are all pinned by sha256 — the
  voice model *is* the channel's sound.
- **`--noise_scale 0 --noise_w 0` is load-bearing.** Measured before building anything:
  the same line twice gave 5.668 s and 5.796 s with different hashes. With the flags,
  three runs and one hash. Without that probe the phase would have shipped a renderer
  that quietly stopped being deterministic.
- **Declared duration became a floor**, as planned — plus a consequence the plan
  missed: `audio.silence` windows are written in absolute seconds, so a stretched shot
  slides them onto the wrong line. They are now remapped through the shot they pointed
  at.
- **One caption size for the whole video**, fitted against the longest group. Per-group
  fitting made the type jump between cues, which is the clearest tell of an
  auto-captioned video.
- researchowl got the grounding extension first, as the order below demanded, and one
  thing that order revealed: with the voice repeating on-screen figures, claims had to
  be de-duplicated by *canonical unit* ("35,000 FT" and "35,000 feet" are one claim) or
  every narrated Short would double its own review report.
- **researchowl's estimate of the voice had to be measured, not assumed** (2026-08-12).
  It shipped with 14.2 characters per second, taken from a single line, and that
  overshot every narration by about a fifth — four to six seconds on a whole Short,
  enough to make the spec writer rewrite videos that were already inside the target.
  Every generation since narration shipped had spent all three attempts on it.
  Synthesizing the 28 narration lines the bot had actually written gave 18.5 char/s
  **plus 0.25 s at every full stop**, which is the term that matters: Piper's
  `SENTENCE_SILENCE` is per sentence, so "Witness identities. Sensor details. Locations
  redacted." costs three quarters of a second that a characters-only model gives away.
  Estimates now land within half a second of the three rendered MP4s. The lesson is the
  older one restated: a constant taken from one sample is a guess with a decimal point.

Original plan, kept for the record:

### 4b (as planned). Narration (TTS) + burned-in captions

These two ship together: most Shorts are watched muted, so the captions matter more than
the voice — but both come from the same new field.

- Spec: each shot gains an optional `narration` string. **When present, the shot's
  duration is derived from the synthesized audio length plus padding** (clamped to the
  template's min/max) — voice-first timing, not text-squeezed-into-a-window. `duration`
  stays valid for shots without narration.
- shortsmith: TTS engine local and free — Piper or Kokoro, CPU-realtime, **model version
  pinned in the image** so renders stay reproducible. One voice, always the same: the
  voice is channel identity, not a per-video choice. Captions burned from the narration
  text, word-grouped, styled like the existing typography.
- Fallback (repo convention): if TTS fails, render with music only and report it —
  degrade, don't die. The job must not fail because a phoneme did.
- researchowl — two small, real changes:
  1. **Grounding must cover `narration`.** It is exactly the field an LLM fills with
     confident paraphrase. `grounding.py` extracts from spec props today; add the
     narration strings to the extraction. Same deterministic path, no LLM.
  2. Prompt: narration guidance (spoken register, ≤ ~25 words per shot, hook in the
     first line — the first two seconds decide the swipe).
- Optional later upgrade: ElevenLabs behind an env var (~$0.10–0.30/Short) if the local
  voice grates. Start local; the constraint on volume is review time, not money.

### Phase 4 tests

| Area | Assert |
|---|---|
| Contract | Spec without `audio`/`narration` still validates and renders (today's golden spec passes untouched) |
| Timing | Narrated shot duration == audio length + padding, clamped to template bounds |
| Fallback | TTS failure produces a music-only render plus a warning, not a failed job |
| Grounding (researchowl) | A fabricated figure placed only in `narration` is flagged |
| Determinism | Same spec twice → byte-identical audio track (pinned model) |

---

## Phase 5 — archival assets (the Ken Burns template)

The most differentiating work for this niche: real declassified documents, newspaper
clippings and official photos, panned and zoomed with a highlight box. It is what
separates the channel from generic AI slop — and it extends the grounding philosophy to
imagery, which is why the editorial rule below is load-bearing.

**The rule: only assets that come from the session's own sources.** No stock, no image
search, no "looks right". If the scraper didn't see it, the Short doesn't show it.

- shortsmith:
  - New template `archive_pan`: props = asset reference, start/end crop keyframes,
    optional highlight rectangle, mandatory `credit` line (rendered small, always).
  - New endpoint `POST /assets` — content-addressed upload (sha256 as the id), so specs
    reference immutable hashes and re-renders can't silently swap an image.
- researchowl:
  - Asset extraction during scraping: `og:image`, inline images above a size floor, PDF
    pages rendered to PNG. Stored under `/data/assets/{session_id}/` with **the source
    URL persisted per asset** — provenance is the whole point.
  - Spec generation: the prompt receives the asset list (id, source URL, dimensions,
    nearby text) and may use `archive_pan` shots.
  - The claims report grows an assets section: every asset used, with its origin URL, so
    the human gate reviews imagery the same way it reviews figures.
- Rights posture: US federal government works (Blue Book, NARA scans, official releases)
  are public domain — the bulk of this channel's material. Anything else the human
  reviewer judges at the gate that already exists; the credit line renders regardless.
- This is the largest phase. It is two deliverables in truth (asset pipeline; template)
  and the pipeline is useful alone — extracted assets improve the *blog* posts too.

### Phase 5 tests

| Area | Assert |
|---|---|
| Assets | Content-addressing: same bytes → same id; spec referencing an unknown hash is rejected at validation with the exact path |
| Provenance | Every stored asset carries a source URL; claims report lists all used assets |
| Template | Keyframe interpolation renders deterministically; missing `credit` fails validation |
| Editorial | A spec referencing an asset from another session is rejected |

---

## Phase 6 — `short_es` for Zona de Exclusión

Nearly free once the format is proven (phase 2 doc, §11, still true): shortsmith draws
whatever strings it gets and does not care about language; with phase 4, TTS needs a
Spanish voice (Piper has good ones — pin it like the English one). The work is the
prompt, the narrative shapes, and the stopword list — all researchowl, all small.

Do it only after the EN format has produced retention worth copying. A bad format in two
languages is twice the bad format.

---

## Explicitly out — generative video

No Veo, no Sora, no Runway. Not primarily for cost: a documentary channel whose premise
is *"the numbers come from primary sources"* cannot mix in fabricated "recreations"
without undermining exactly what the grounding pipeline protects. If ever revisited, it
would need an on-screen RECREATION label and a very good reason. There is no current
reason.

---

## Free work — no phase required

Available any time, zero researchowl changes, because the contract is fetched live:

- **New vector templates** in shortsmith: `timeline`, `before_after`, `map_zoom`. Same
  discriminated-union pattern as the existing eight; they appear in the prompt
  automatically on the next generation.
- **Narrative tuning** in the prompt: sharper first-shot hook, the
  hook → evidence → unresolved question → CTA arc. Costs one commit, no deploy risk
  beyond a prompt change.

---

## Implementation order

One change at a time, verified before the next — and phase-gated by the Analytics data
from §0.

1. Publish 3–4 Shorts with the current pipeline. Read the retention curves.
2. ~~Phase 4a~~ — done (the preset palette, see above; the user chose to skip the gate
   for the mechanism and let the published Shorts test the palette itself).
3. Phase 4b (narration + captions), grounding extension in researchowl **first** — build
   the check before the thing it checks, same reasoning as phase 2 §12.
4. Free-work templates whenever convenient; they ride along.
5. Phase 5, asset pipeline before template — the pipeline is useful alone.
6. Phase 6 last, and only if the numbers say the format earned a second language.