docs: gotcha de OOM por fuentes grandes en KNOWN-ISSUES
Build & Deploy ResearchOwl / build-and-push (push) Successful in 7s
Build & Deploy ResearchOwl / build-and-push (push) Successful in 7s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
187d29a372
commit
4ee9ad064e
@@ -60,3 +60,24 @@ hit a consent wall from EU IPs, and the inner `AU_yqL` id is only resolvable via
|
||||
Google's private batchexecute API. Do not retry. The news seed uses Bing News
|
||||
RSS instead (`ENABLE_NEWS_SEED`, real publisher URL in the `?url=` param of
|
||||
apiclick.aspx — unwrapped by `_unwrap_news_link`).
|
||||
|
||||
## Large sources can OOM-kill the pod
|
||||
|
||||
On 2026-07-10 the pod was OOMKilled (memory limit was 1Gi) mid-research: a
|
||||
batch of 20 concurrent sources hit a 98k-word document plus several large PDFs
|
||||
at once, and pdfplumber's parse spiked RAM past the limit. The in-memory
|
||||
research task died with the pod and its session sat in `running` forever.
|
||||
|
||||
Mitigations now in place:
|
||||
|
||||
- Memory limit raised to 2Gi (`k8s-manifests/researchowl/deployment.yaml`).
|
||||
- PDFs capped at 15MB (was 50MB), checked both via Content-Length and actual
|
||||
body size; pdfplumber runs in `run_in_executor` (it is sync + CPU-heavy and
|
||||
also froze the event loop, same class of bug as DDGS) and flushes its page
|
||||
cache per page.
|
||||
- Extracted content is truncated to `max_content_length` (300k chars) before
|
||||
hitting `source_contents`.
|
||||
- On startup the bot marks orphaned `running` sessions as `interrupted`.
|
||||
|
||||
If a research still dies, the scraped sources survive in the DB: `/process`
|
||||
re-chunks and scores them without re-scraping.
|
||||
|
||||
Reference in New Issue
Block a user