Files

T

ChemaVXandClaude Fable 5 3d74857d85 fix(ghost): guard anti-duplicados dentro de publish_draft, solo tras POST aceptado

El guard anterior vivía en el fallback de autofill y casaba con CUALQUIER
draft del mismo título: un fallo pre-POST (Ollama caído) con un draft viejo
del mismo topic descartaba en silencio el contenido recién generado.

Ahora la recuperación vive dentro de publish_draft y solo se activa cuando
el POST fue aceptado (2xx) y falla la lectura de la respuesta — el escenario
exacto del incidente br del 2026-07-04 — con filtro since=attempt_start en
find_draft_by_title. Cubre a todos los callers: autofill, bare publish y
/publish (antes sin proteger: reintento del usuario = draft duplicado).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-05 15:54:49 +00:00

2.8 KiB

Raw Blame History

ResearchOwl — Known Issues & Operational Gotchas

Brotli / aiohttp incompatibility

Do NOT install any brotli backend (Brotli, brotlicffi) while on aiohttp 3.14.x. aiohttp's incremental brotli decompressor is broken and fails intermittently depending on stream chunking ("Can not decode content-encoding: br" on VALID streams — the same bytes decompress fine offline; httpx is unaffected).

Extra trap: merely installing a backend makes aiohttp advertise br in the default Accept-Encoding of every session that doesn't set one explicitly. On 2026-07-04 a scraper fix installed brotlicffi and inadvertently enabled br project-wide: Ghost/Cloudflare responded in brotli, the decode blew up AFTER Ghost had already accepted the POST, and the publish fallback re-published → duplicate drafts + silently lost SEO notices.

Fix applied (commits 397546a + 76c927f + 7d07375, guard relocated 2026-07-05):

no brotli backend in requirements.txt (comment there explains why),
explicit Accept-Encoding: gzip, deflate on all Ghost/autofill aiohttp sessions and on the scraper HEADERS (57f341f),
duplicate guard INSIDE GhostPublisher.publish_draft (covers every caller, including /publish): if the POST is accepted (2xx) but reading the response fails, it recovers the just-created draft via find_draft_by_title(title, since=attempt_start) instead of raising. The guard only fires after an accepted POST — a pre-POST failure (Ollama down, menu fetch, links) never triggers it, so a stale same-title draft from a previous run can no longer swallow freshly generated content.

Re-test with disclosure.org before ever re-enabling br (e.g. after an aiohttp upgrade).

SQLite WAL mode + read-only mounts

The database runs in WAL mode, so even read-only access needs the -shm file writable, or must open with sqlite3 "file:...?immutable=1". Backups (daily CronJob researchowl-db-backup, 03:00 Europe/Madrid, PVC researchowl-backups, 7-day retention) inherit WAL mode: to inspect one from a read-only mount use ?immutable=1; to restore, copy it to a writable location first.

DDG (duckduckgo_search) blocks the event loop

DDGS() is synchronous (blocking requests inside). Never call it directly from async code — always wrap in loop.run_in_executor() (see _ddg_text_sync / _ddg_videos_sync in src/scraper/exhaustive.py). Direct calls froze the entire Telegram bot during searches until fixed on 2026-07-04 (8dfd011).

Google News RSS is a dead end from this infrastructure

news.google.com/rss entry links point to /rss/articles/CBMi… redirects that hit a consent wall from EU IPs, and the inner AU_yqL id is only resolvable via Google's private batchexecute API. Do not retry. The news seed uses Bing News RSS instead (ENABLE_NEWS_SEED, real publisher URL in the ?url= param of apiclick.aspx — unwrapped by _unwrap_news_link).