diff --git a/KNOWN-ISSUES.md b/KNOWN-ISSUES.md index 61ba832..7ee2d11 100644 --- a/KNOWN-ISSUES.md +++ b/KNOWN-ISSUES.md @@ -60,3 +60,24 @@ hit a consent wall from EU IPs, and the inner `AU_yqL` id is only resolvable via Google's private batchexecute API. Do not retry. The news seed uses Bing News RSS instead (`ENABLE_NEWS_SEED`, real publisher URL in the `?url=` param of apiclick.aspx — unwrapped by `_unwrap_news_link`). + +## Large sources can OOM-kill the pod + +On 2026-07-10 the pod was OOMKilled (memory limit was 1Gi) mid-research: a +batch of 20 concurrent sources hit a 98k-word document plus several large PDFs +at once, and pdfplumber's parse spiked RAM past the limit. The in-memory +research task died with the pod and its session sat in `running` forever. + +Mitigations now in place: + +- Memory limit raised to 2Gi (`k8s-manifests/researchowl/deployment.yaml`). +- PDFs capped at 15MB (was 50MB), checked both via Content-Length and actual + body size; pdfplumber runs in `run_in_executor` (it is sync + CPU-heavy and + also froze the event loop, same class of bug as DDGS) and flushes its page + cache per page. +- Extracted content is truncated to `max_content_length` (300k chars) before + hitting `source_contents`. +- On startup the bot marks orphaned `running` sessions as `interrupted`. + +If a research still dies, the scraped sources survive in the DB: `/process` +re-chunks and scores them without re-scraping.