docs: gotcha de OOM por fuentes grandes en KNOWN-ISSUES
Build & Deploy ResearchOwl / build-and-push (push) Successful in 7s
Build & Deploy ResearchOwl / build-and-push (push) Successful in 7s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
187d29a372
commit
4ee9ad064e
@@ -60,3 +60,24 @@ hit a consent wall from EU IPs, and the inner `AU_yqL` id is only resolvable via
|
|||||||
Google's private batchexecute API. Do not retry. The news seed uses Bing News
|
Google's private batchexecute API. Do not retry. The news seed uses Bing News
|
||||||
RSS instead (`ENABLE_NEWS_SEED`, real publisher URL in the `?url=` param of
|
RSS instead (`ENABLE_NEWS_SEED`, real publisher URL in the `?url=` param of
|
||||||
apiclick.aspx — unwrapped by `_unwrap_news_link`).
|
apiclick.aspx — unwrapped by `_unwrap_news_link`).
|
||||||
|
|
||||||
|
## Large sources can OOM-kill the pod
|
||||||
|
|
||||||
|
On 2026-07-10 the pod was OOMKilled (memory limit was 1Gi) mid-research: a
|
||||||
|
batch of 20 concurrent sources hit a 98k-word document plus several large PDFs
|
||||||
|
at once, and pdfplumber's parse spiked RAM past the limit. The in-memory
|
||||||
|
research task died with the pod and its session sat in `running` forever.
|
||||||
|
|
||||||
|
Mitigations now in place:
|
||||||
|
|
||||||
|
- Memory limit raised to 2Gi (`k8s-manifests/researchowl/deployment.yaml`).
|
||||||
|
- PDFs capped at 15MB (was 50MB), checked both via Content-Length and actual
|
||||||
|
body size; pdfplumber runs in `run_in_executor` (it is sync + CPU-heavy and
|
||||||
|
also froze the event loop, same class of bug as DDGS) and flushes its page
|
||||||
|
cache per page.
|
||||||
|
- Extracted content is truncated to `max_content_length` (300k chars) before
|
||||||
|
hitting `source_contents`.
|
||||||
|
- On startup the bot marks orphaned `running` sessions as `interrupted`.
|
||||||
|
|
||||||
|
If a research still dies, the scraped sources survive in the DB: `/process`
|
||||||
|
re-chunks and scores them without re-scraping.
|
||||||
|
|||||||
Reference in New Issue
Block a user