feat(short): la paleta de audio viva — GET /audio en el contrato, el prompt y el bucle de edición
Build & Deploy ResearchOwl / build-and-push (push) Successful in 9s

shortsmith ya compone la banda sonora (sonar); ahora publica una paleta
(pulse, static) y este repo la consume en vivo: el prompt la ofrece con sus
notas de mood, validate_spec la usa como fuente de verdad para audio.preset,
y editar el preset en /short_spec es la manera gratis de escucharlas. Sin
/audio (404 o caída) todo cae a la paleta base y nada deja de renderizar.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
ChemaVX
2026-08-06 14:41:27 +00:00
co-authored by Claude Fable 5
parent 1fd0c1b3d6
commit 93a506b636
11 changed files with 259 additions and 34 deletions
+7
View File
@@ -92,6 +92,13 @@ saves the edited spec as a new output, and renders. No LLM in that path: it is
free. The session comes from the filename, so it works even if the chat has
researched something else since.
**Soundtrack:** every Short carries a synthesized score — shortsmith composes
it deterministically, no samples, no licensing. The palette comes live from
`GET /audio` (the audio half of what `GET /templates` does for shots): `sonar`
for case files, `pulse` for debunks, `static` for document drops. The model
picks one to match the narrative shape, and the cheapest way to audition them
is the edit loop — change `audio.preset` in the spec file and re-send it.
Full spec of the phase: `docs/shortsmith-phase2-spec.md`.
## YouTube (`/upload_short`)
+26 -18
View File
@@ -34,26 +34,34 @@ the expected outcome (audio first), but the gate can reorder them.
## Phase 4 — sound
The largest single lever. A silent Short is penalised de facto: retention is the
algorithm's primary signal and silence invites the swipe at second one. Three deliverables,
shipped in this order because each is useful without the next.
> **Corrected after reading the shortsmith source** (the first draft of this section
> assumed silent renders and proposed royalty-free tracks on disk — both wrong).
> shortsmith has synthesized audio by design: numpy only, no samples, no licensing
> exposure, deterministic to the sample, with the spec's `audio: {preset, silence}`
> block and the validated `sonar` composition guarded by a reference gate. The right
> 4a for that architecture is a wider palette of synthesized presets, not files.
### 4a. Music bed + transition SFX
### 4a. The preset palette — **shipped 2026-08-06**
- shortsmith: new **optional** top-level spec block:
- shortsmith: `pulse` (sub-bass heartbeat tightening past the midpoint — tension, for
debunks) and `static` (shortwave noise bed, seeded crackles, faint drone — document
drops) joined `sonar` and `none`. Same arc grammar (nothing starts inside a `silence`
window, closing swell, same limiter chain); `static`'s noise comes from seeded legacy
`RandomState` streams, which NEP 19 froze — deterministic on any numpy. `GET /audio`
publishes the palette with one-line mood notes.
- researchowl: the client fetches the palette (404 → the baseline pair, fallback
convention), `validate_spec` takes it as the live source of truth for
`audio.preset`, and the prompt offers it with the mood notes so the model matches
preset to narrative shape. The edit loop accepts it too: changing `audio.preset` in
the `/short_spec` file and re-sending is the free way to audition the palette.
```json
"audio": { "track": "static_dread", "volume_db": -14 }
```
### Deferred from 4a: shot-boundary transition accents
A small curated set of royalty-free tracks (35, matching the channel's tone) baked
into the shortsmith image as assets. Track names are advertised in the `GET /templates`
response (a new `audio.tracks` key) so the prompt can list them without hardcoding —
same reasoning as the template schemas. ffmpeg mixes at final assembly; a short whoosh
on each shot boundary comes free from the same mix.
- researchowl: **zero code**. The prompt gains one line ("pick a track from: …",
fed from the contract fetch). Old specs without `audio` render silent, as today.
- Determinism holds: same spec, same output.
A synthesized whoosh at each cut. Deliberately not done yet: `audio.py`'s presets are
fixed compositions that know nothing about the shots, and that one-way line is worth
keeping until the palette itself proves its value. If it happens, it is an opt-in
`Audio.transitions` flag with boundaries passed alongside `silence` — additive, and
`sonar`'s reference gate must not notice.
### 4b. Narration (TTS) + burned-in captions
@@ -172,8 +180,8 @@ One change at a time, verified before the next — and phase-gated by the Analyt
from §0.
1. Publish 34 Shorts with the current pipeline. Read the retention curves.
2. Phase 4a (music). Smallest change, immediate feel upgrade, proves the `audio` contract
extension end to end.
2. ~~Phase 4a~~ — done (the preset palette, see above; the user chose to skip the gate
for the mechanism and let the published Shorts test the palette itself).
3. Phase 4b (narration + captions), grounding extension in researchowl **first** — build
the check before the thing it checks, same reasoning as phase 2 §12.
4. Free-work templates whenever convenient; they ride along.
+2 -1
View File
@@ -605,7 +605,8 @@ async def cmd_short_spec(update: Update, ctx: ContextTypes.DEFAULT_TYPE):
filename=f"short_{session['id']}_spec.json",
caption=f"🎬 Shot spec — {session['topic']}\n{created} UTC\n\n"
f"Edítalo y mándamelo de vuelta como fichero para "
f"re-renderizar sin pagar otra generación.",
f"re-renderizar sin pagar otra generación. La banda sonora "
f"también: audio.preset acepta sonar, pulse o static.",
)
except Exception as e:
logger.error("short_spec failed", error=str(e))
+21 -4
View File
@@ -28,7 +28,8 @@ from src.config import settings
from src.db.database import OutputType, ResearchDB
from src.generator.grounding import GroundingReport, check_grounding
from src.generator.shortsmith import (
ShortsmithClient, ShortsmithError, ShortsmithRejected, ShortsmithUnavailable,
BASELINE_PRESETS, ShortsmithClient, ShortsmithError, ShortsmithRejected,
ShortsmithUnavailable,
)
from src.generator.shortspec import ShortSpecWriter, SpecWriteFailed
from src.generator.spec_contract import SpecInvalid, editorial_notes, validate_spec
@@ -138,6 +139,15 @@ class ShortProducer:
directory.mkdir(parents=True, exist_ok=True)
return directory / f"{session_id}.mp4"
async def _presets(self) -> dict[str, str]:
"""La paleta de audio, en vivo. Nunca tumba nada: sin ella se ofrece la
base y el Short sale con sonar, que es lo que salía siempre."""
try:
return await self.client.audio_presets()
except Exception as e:
logger.warning("GET /audio falló — paleta base", error=str(e))
return dict(BASELINE_PRESETS)
# --- pipeline -----------------------------------------------------------
async def produce(self, session_id: int,
@@ -171,15 +181,19 @@ class ShortProducer:
logger.warning("Short sin URL de artículo — se sigue con el dominio pelado",
session_id=session_id)
# 2. El contrato, en vivo. Sin él no hay prompt que escribir.
# 2. El contrato, en vivo. Sin él no hay prompt que escribir. La paleta
# de audio mejora el prompt pero no lo define: si /audio falla, se
# ofrece la paleta base y el render sale igual.
templates = await self.client.templates()
presets = await self._presets()
# 3. El spec.
started = time.monotonic()
llm_call = self.llm_override or self._llm_call(session_id)
writer = ShortSpecWriter(
llm_call, templates,
refresh_templates=lambda: self.client.templates(refresh=True))
refresh_templates=lambda: self.client.templates(refresh=True),
presets=presets)
try:
written = await writer.write(
topic, context, article_url=result.article_url,
@@ -251,9 +265,12 @@ class ShortProducer:
result = ShortResult(topic=topic, spec=spec)
# 1. El contrato, en vivo — las mismas rutas verbatim que ve el modelo.
# La paleta también: editar audio.preset a "pulse" es justo el tipo
# de retoque para el que existe este camino.
templates = await self.client.templates()
presets = await self._presets()
try:
validate_spec(spec, templates)
validate_spec(spec, templates, presets=presets)
except SpecInvalid as e:
result.failure = ("El spec editado no pasa el contrato: "
+ "; ".join(e.errors[:6]))
+42
View File
@@ -42,10 +42,20 @@ __all__ = [
"ShortsmithUnavailable",
"ShortsmithRejected",
"JobResult",
"BASELINE_PRESETS",
"POLL_INTERVAL",
"POLL_CEILING",
]
#: La paleta que existía antes de que shortsmith publicara `GET /audio`. Es el
#: fallback cuando el endpoint no está (404 = shortsmith anterior) o no se pudo
#: consultar: la paleta mejora el spec, no lo define, y quedarse en sonar nunca
#: rompe un render.
BASELINE_PRESETS = {
"sonar": "low drone and sonar pings tightening toward the close",
"none": "digital silence",
}
class ShortsmithError(Exception):
"""Cualquier fallo hablando con shortsmith."""
@@ -86,6 +96,7 @@ class JobResult:
#: petición cuando una validación falla, por si el renderizador se actualizó a
#: mitad de una run.
_templates_cache: dict[str, dict[str, Any]] = {}
_presets_cache: dict[str, dict[str, str]] = {}
class ShortsmithClient:
@@ -136,6 +147,37 @@ class ShortsmithClient:
logger.info("shortsmith templates fetched", n=len(data))
return data
async def audio_presets(self, refresh: bool = False) -> dict[str, str]:
"""La paleta de audio: nombre de preset -> nota de una línea (`GET /audio`).
La mitad de audio del contrato vivo: shortsmith añade un preset y el
prompt lo ofrece sin tocar este repo. Un 404 es un shortsmith anterior
al endpoint y devuelve la paleta base, sin error — fallbacks siempre.
"""
if not refresh and self.base_url in _presets_cache:
return _presets_cache[self.base_url]
try:
async with self._session(30) as sess:
async with sess.get(f"{self.base_url}/audio") as resp:
if resp.status == 404:
data = dict(BASELINE_PRESETS)
elif resp.status != 200:
body = await resp.text()
raise ShortsmithError(
f"GET /audio devolvió {resp.status}: {body[:200]}")
else:
payload = await resp.json()
presets = payload.get("presets")
data = (presets if isinstance(presets, dict) and presets
else dict(BASELINE_PRESETS))
except aiohttp.ClientError as e:
raise ShortsmithUnavailable(f"shortsmith inalcanzable: {e}") from e
except asyncio.TimeoutError as e:
raise ShortsmithUnavailable("shortsmith no respondió a /audio") from e
_presets_cache[self.base_url] = data
logger.info("shortsmith audio presets fetched", presets=sorted(data))
return data
async def render(self, spec: dict[str, Any]) -> str:
"""`POST /render`. Devuelve el job_id. 422 -> ShortsmithRejected."""
try:
+16 -3
View File
@@ -98,7 +98,9 @@ and your whole answer becomes unparseable. This is the single most common way \
this task fails.
- meta.id is a lowercase slug (letters, digits, - and _). meta.title is the \
title a human reads, not a filename.
- version is 1. Keep meta at 1080x1920 and audio preset "sonar".
- version is 1. Keep meta at 1080x1920.
- audio.preset — pick the one whose mood fits the shape you chose:
{presets}
- The closing shot carries the domain, uppercase, no protocol: {domain}
# 4. Grounding — this is the part that matters
@@ -144,6 +146,14 @@ Return the JSON object now."""
PALETTE_FALLBACK = "ink, amber, amber_dark, muted, dim, red"
#: Si el caller no trae la paleta de audio, el prompt solo ofrece lo que
#: cualquier shortsmith renderiza.
PRESETS_FALLBACK = {"sonar": "low drone and sonar pings", "none": "digital silence"}
def _describe_presets(presets: dict[str, str]) -> str:
return "\n".join(f' "{name}"{note}' for name, note in sorted(presets.items()))
@dataclass
class SpecResult:
@@ -305,12 +315,14 @@ LLMCall = Callable[[str, str], Awaitable[str]]
class ShortSpecWriter:
def __init__(self, llm_call: LLMCall, templates: dict[str, dict],
refresh_templates: Optional[Callable[[], Awaitable[dict]]] = None):
refresh_templates: Optional[Callable[[], Awaitable[dict]]] = None,
presets: Optional[dict[str, str]] = None):
self.llm_call = llm_call
self.templates = templates
#: Se vuelve a pedir el contrato si una validación falla: el
#: renderizador puede haberse actualizado a mitad de la run.
self.refresh_templates = refresh_templates
self.presets = presets or dict(PRESETS_FALLBACK)
def build_prompt(self, topic: str, context: str, article_url: Optional[str],
domain: str) -> str:
@@ -322,6 +334,7 @@ class ShortSpecWriter:
shapes=NARRATIVE_SHAPES,
templates=describe_templates(self.templates),
colors=_palette(self.templates),
presets=_describe_presets(self.presets),
domain=domain,
target_min=TARGET_MIN_DURATION,
target_max=TARGET_MAX_DURATION,
@@ -359,7 +372,7 @@ class ShortSpecWriter:
last_spec = spec
try:
validate_spec(spec, self.templates)
validate_spec(spec, self.templates, presets=self.presets)
except SpecInvalid as e:
history.append(e.errors)
feedback = _format_errors(e.errors)
+19 -6
View File
@@ -21,7 +21,7 @@ cubierto.
from __future__ import annotations
import re
from typing import Any, Optional
from typing import Any, Iterable, Optional
__all__ = [
"SpecInvalid",
@@ -183,14 +183,23 @@ def _check_meta(meta: Any) -> list[str]:
return errors
def _check_audio(audio: Any, total: float) -> list[str]:
#: Los presets que existían antes de `GET /audio`. Solo es el default cuando el
#: caller no pasa la paleta viva; con ella, un preset nuevo en shortsmith llega
#: aquí sin tocar este repo — el mismo pacto que las plantillas.
BASELINE_PRESET_NAMES = ("sonar", "none")
def _check_audio(audio: Any, total: float,
presets: Optional[Iterable[str]] = None) -> list[str]:
if audio is None:
return []
if not isinstance(audio, dict):
return ["audio: se esperaba un objeto"]
errors = []
if audio.get("preset", "sonar") not in ("sonar", "none"):
errors.append(f"audio.preset: {audio.get('preset')!r} no es 'sonar' ni 'none'")
known = tuple(presets) if presets else BASELINE_PRESET_NAMES
if audio.get("preset", "sonar") not in known:
errors.append(f"audio.preset: {audio.get('preset')!r} no existe "
f"(los presets son: {', '.join(sorted(known))})")
silence = audio.get("silence", [])
if not isinstance(silence, list):
return errors + ["audio.silence: se esperaba una lista de pares [inicio, fin]"]
@@ -223,12 +232,16 @@ def _total_duration(spec: dict) -> float:
return total
def validate_spec(spec: Any, templates: dict[str, dict]) -> None:
def validate_spec(spec: Any, templates: dict[str, dict],
presets: Optional[Iterable[str]] = None) -> None:
"""Lanza `SpecInvalid` con TODAS las rutas que fallan.
Se devuelven todos los errores de golpe a propósito: el bucle de reintento
se los da al modelo verbatim y arreglar cinco de una vez sale más barato
que cinco vueltas.
`presets` es la paleta viva de `GET /audio`; sin ella se valida contra la
paleta base, que nunca acepta nada que un shortsmith viejo no renderice.
"""
errors: list[str] = []
if not isinstance(spec, dict):
@@ -284,7 +297,7 @@ def validate_spec(spec: Any, templates: dict[str, dict]) -> None:
if total > MAX_TOTAL_DURATION:
errors.append(f"shots: la duración total ({total:.2f}s) pasa del límite "
f"de {MAX_TOTAL_DURATION}s")
errors.extend(_check_audio(spec.get("audio"), total))
errors.extend(_check_audio(spec.get("audio"), total, presets))
if errors:
raise SpecInvalid(errors)
+58
View File
@@ -59,6 +59,14 @@ class FakeProcessor:
return self.chunks
PRESETS = {
"sonar": "low drone — the case-file mood",
"pulse": "sub-bass heartbeat — debunks",
"static": "shortwave static — document drops",
"none": "digital silence",
}
class FakeClient:
"""shortsmith de mentira. `fail_at` decide dónde se rompe."""
@@ -73,6 +81,11 @@ class FakeClient:
raise ShortsmithUnavailable("no hay nadie al otro lado")
return TEMPLATES
async def audio_presets(self, refresh=False):
if self.fail_at == "audio":
raise ShortsmithUnavailable("sin /audio")
return dict(PRESETS)
async def render(self, spec):
if self.fail_at == "render":
raise ShortsmithUnavailable("conexión rechazada")
@@ -450,3 +463,48 @@ async def test_rerender_respects_the_kill_switch(tmp_path, monkeypatch):
with pytest.raises(ShortsDisabled):
await p.rerender(153, json.loads(json.dumps(SPEC)))
# --- la paleta de audio (GET /audio) ----------------------------------------
@pytest.mark.asyncio
async def test_an_edited_preset_from_the_live_palette_renders(tmp_path, monkeypatch):
"""El retoque para el que existe el bucle de edición: cambiar la banda
sonora a "pulse" sin pagar otra generación."""
edited = json.loads(json.dumps(SPEC))
edited["audio"] = {"preset": "pulse"}
client = FakeClient()
p = producer(tmp_path, monkeypatch, client=client, llm=_llm_prohibido)
result = await p.rerender(153, edited)
assert result.has_video
assert client.rendered["audio"]["preset"] == "pulse"
@pytest.mark.asyncio
async def test_a_preset_the_renderer_does_not_know_is_rejected_with_the_palette(
tmp_path, monkeypatch):
edited = json.loads(json.dumps(SPEC))
edited["audio"] = {"preset": "vaporwave"}
client = FakeClient()
p = producer(tmp_path, monkeypatch, client=client, llm=_llm_prohibido)
result = await p.rerender(153, edited)
assert not result.has_video
assert "audio.preset" in result.failure and "pulse" in result.failure
assert client.rendered is None
@pytest.mark.asyncio
async def test_a_dead_audio_endpoint_never_blocks_a_sonar_render(tmp_path, monkeypatch):
"""La paleta mejora el prompt, no lo define: sin /audio se cae a la base y
un spec con sonar renderiza igual."""
p = producer(tmp_path, monkeypatch, client=FakeClient(fail_at="audio"),
llm=_llm_prohibido)
result = await p.rerender(153, json.loads(json.dumps(SPEC)))
assert result.has_video
+26 -2
View File
@@ -10,8 +10,8 @@ from pathlib import Path
import pytest
from src.generator.shortsmith import (
JobResult, ShortsmithClient, ShortsmithError, ShortsmithRejected,
ShortsmithUnavailable, _templates_cache,
BASELINE_PRESETS, JobResult, ShortsmithClient, ShortsmithError,
ShortsmithRejected, ShortsmithUnavailable, _presets_cache, _templates_cache,
)
EXAMPLE = Path(__file__).resolve().parents[1] / "src/generator/examples/jal1628.json"
@@ -100,6 +100,30 @@ async def test_templates_cached_per_process():
_templates_cache.clear()
@pytest.mark.asyncio
async def test_audio_presets_come_from_the_service():
_presets_cache.clear()
client = ShortsmithClient("http://fake:8080")
palette = {"sonar": "drone", "pulse": "heartbeat"}
session = patch_session(client, {"/audio": FakeResp(200, {"presets": palette})})
assert await client.audio_presets() == palette
await client.audio_presets()
assert len(session.calls) == 1, "la segunda llamada debe salir de la caché"
_presets_cache.clear()
@pytest.mark.asyncio
async def test_a_shortsmith_without_audio_endpoint_yields_the_baseline():
"""404 = shortsmith anterior a la paleta. Fallback, no error."""
_presets_cache.clear()
client = ShortsmithClient("http://fake:8080")
patch_session(client, {"/audio": FakeResp(404, body="not found")})
assert await client.audio_presets() == BASELINE_PRESETS
_presets_cache.clear()
@pytest.mark.asyncio
async def test_render_returns_job_id(spec):
client = ShortsmithClient("http://fake:8080")
+17
View File
@@ -118,6 +118,23 @@ def test_prompt_says_out_loud_that_there_is_no_article_yet():
assert "No article URL yet" in w.build_prompt("X", "m", None, "X.TEST")
def test_prompt_offers_the_live_audio_palette():
"""La mitad de audio del contrato vivo: los presets y sus notas de mood
vienen de GET /audio, no de este repo."""
palette = {"sonar": "the case-file mood", "pulse": "tension, built for debunks"}
w, _ = writer("{}", presets=palette)
prompt = w.build_prompt("X", "m", None, "X.TEST")
assert '"pulse" — tension, built for debunks' in prompt
assert "whose mood fits the shape" in prompt
def test_prompt_without_a_palette_only_offers_the_baseline():
w, _ = writer("{}")
prompt = w.build_prompt("X", "m", None, "X.TEST")
assert '"sonar"' in prompt and '"none"' in prompt
assert '"pulse"' not in prompt
# --- bucle ------------------------------------------------------------------
@pytest.mark.asyncio
+25
View File
@@ -184,6 +184,31 @@ def test_silence_window_cannot_run_past_the_end():
assert any("se sale de la duración total" in e for e in errors_of(bad))
def test_the_live_palette_widens_what_a_preset_may_be():
"""Con la paleta de GET /audio, un preset nuevo en shortsmith llega aquí
sin tocar este repo el mismo pacto que las plantillas."""
doc = spec_with(shot(duration=25.0))
doc["audio"] = {"preset": "pulse"}
validate_spec(doc, TEMPLATES, presets=("sonar", "pulse", "static", "none"))
def test_without_the_palette_only_the_baseline_presets_pass():
"""El default es conservador a propósito: nunca acepta lo que un shortsmith
viejo no renderice."""
doc = spec_with(shot(duration=25.0))
doc["audio"] = {"preset": "pulse"}
assert any("audio.preset" in e for e in errors_of(doc))
def test_an_unknown_preset_error_names_the_palette():
doc = spec_with(shot(duration=25.0))
doc["audio"] = {"preset": "vaporwave"}
with pytest.raises(SpecInvalid) as exc:
validate_spec(doc, TEMPLATES, presets=("sonar", "pulse", "none"))
line = next(e for e in exc.value.errors if "audio.preset" in e)
assert "pulse" in line and "vaporwave" in line
def test_extra_root_key_is_rejected():
bad = spec_with(shot(duration=25.0)); bad["narrative_shape"] = "case_file"
assert any(e.startswith("narrative_shape:") for e in errors_of(bad))