feat(scraper): seed de noticias Bing News RSS tras ENABLE_NEWS_SEED (off)
Build & Deploy ResearchOwl / build-and-push (push) Successful in 7s

Nuevo _seed_news: registra el feed de Bing News como fuente rss y
_extract_rss (F3) siembra sus entries — sin lógica de parseo duplicada.
_unwrap_news_link extrae la URL real del publisher del ?url= de
apiclick.aspx.

Bing y no Google News como pedía el plan: verificado en vivo que los
links de news.google.com/rss/articles/ acaban en muro de consent (UE) y
el id AU_yqL solo se resuelve vía batchexecute interno — habría sembrado
URLs muertas. Los de Bing llevan la URL real en texto plano.

Flag off por defecto: se activará deliberadamente vía deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
ChemaVX
2026-07-04 10:59:00 +00:00
co-authored by Claude Fable 5
parent 8dfd0116b2
commit a23810b9cc
3 changed files with 53 additions and 2 deletions
+12
View File
@@ -36,3 +36,15 @@ def test_simple_chunk():
chunks = simple_chunk(text, chunk_size=100, overlap=20)
assert len(chunks) > 1
assert all(isinstance(c, str) for c in chunks)
def test_unwrap_news_link():
from src.scraper.exhaustive import _unwrap_news_link
wrapped = ("http://www.bing.com/news/apiclick.aspx?ref=FexRss&aid=&tid=x"
"&url=https://example.com/article&c=1&mkt=es-es")
assert _unwrap_news_link(wrapped) == "https://example.com/article"
# Links normales pasan intactos
assert _unwrap_news_link("https://thedebrief.org/foo") == "https://thedebrief.org/foo"
# apiclick sin url= no revienta
assert _unwrap_news_link("http://www.bing.com/news/apiclick.aspx?ref=x") \
== "http://www.bing.com/news/apiclick.aspx?ref=x"