Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eef721a9ed | ||
|
|
cef8a7f2e1 | ||
|
|
1dde4d27eb | ||
|
|
7804d25c51 | ||
|
|
0816e19740 | ||
|
|
2b326ad54f | ||
|
|
124b6d8558 | ||
|
|
6c544e46e2 | ||
|
|
0ac48ba7f8 | ||
|
|
eb4f67414a | ||
|
|
919fe1617a | ||
|
|
117d2b33b2 |
@@ -1,9 +1,10 @@
|
||||
name: CI/CD
|
||||
|
||||
# Decommission (2026-07): trigger switched from push:main to manual only, so
|
||||
# documentation commits don't rebuild/redeploy images. Run from the Gitea UI
|
||||
# (Actions → Run workflow) if a rebuild is ever needed again.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
REGISTRY: gitea.gitea.svc.cluster.local:3000
|
||||
@@ -178,7 +179,8 @@ jobs:
|
||||
# keep their current (still existing) tag in the registry.
|
||||
if [ "${{ steps.changes.outputs.build_bot }}" = "true" ]; then
|
||||
sed -i "s|image: .*polymarket-bot[^-].*|image: git.chemavx.xyz/chemavx/polymarket-bot:${TAG}|g" \
|
||||
polymarket-bot/deployment-bot.yaml
|
||||
polymarket-bot/deployment-bot.yaml \
|
||||
polymarket-bot/cronjob-outcomes.yaml
|
||||
fi
|
||||
if [ "${{ steps.changes.outputs.build_api }}" = "true" ]; then
|
||||
sed -i "s|image: .*polymarket-bot-api.*|image: git.chemavx.xyz/chemavx/polymarket-bot-api:${TAG}|g" \
|
||||
|
||||
@@ -1,7 +1,118 @@
|
||||
# polymarket-bot
|
||||
|
||||
Bot de paper-trading para Polymarket con estrategia bayesiana, API FastAPI y
|
||||
dashboard React. Corre en k3s vía GitOps (Gitea Actions → registry → ArgoCD).
|
||||
Bot de **paper trading** para mercados de predicción de Polymarket: estrategia
|
||||
bayesiana sobre el precio de mercado como prior, señales externas en log-odds,
|
||||
gates de riesgo en cascada y un motor de replay determinista para calibración
|
||||
offline. API FastAPI + dashboard React. Desplegado en k3s vía GitOps
|
||||
(Gitea Actions → registry → ArgoCD), con PostgreSQL, secrets por Infisical y
|
||||
observabilidad completa (Grafana, uptime-kuma, Telegram).
|
||||
|
||||
> **📦 Estado del proyecto: ARCHIVADO (julio 2026).**
|
||||
> El 26 de mayo de 2026 la DGOJ [ordenó el bloqueo cautelar de Polymarket y
|
||||
> Kalshi en España](https://www.ordenacionjuego.es/novedades/dgoj-abre-expediente-sancionador-plataformas-polymarket-kalshi-ordena-bloqueo-sus-webs)
|
||||
> por operar sin licencia de juego ([CoinDesk](https://www.coindesk.com/policy/2026/05/26/spain-joins-growing-list-of-countries-shutting-out-polymarket-and-kalshi)).
|
||||
> Con el bloqueo a nivel de ISP la fuente de datos primaria desaparece, así que
|
||||
> el proyecto se jubila de forma ordenada: datos respaldados y verificados,
|
||||
> resultados documentados en el [informe final](docs/informe-final.md), e
|
||||
> infraestructura apagada vía GitOps. El cierre es parte de la historia del
|
||||
> proyecto, no un abandono — los módulos centrales son agnósticos de la fuente
|
||||
> y el pivote natural es Manifold Markets (API abierta, play-money, sin
|
||||
> fricción regulatoria).
|
||||
|
||||
## El problema y la solución
|
||||
|
||||
Los mercados de predicción publican probabilidades implícitas en el precio.
|
||||
La hipótesis: en mercados de bajo volumen hay ineficiencias detectables
|
||||
combinando el precio con señales externas (noticias, sentimiento cripto,
|
||||
mercados espejo en otras plataformas). El bot la pone a prueba **sin arriesgar
|
||||
dinero**, con la disciplina de un sistema real:
|
||||
|
||||
1. **Prior desde el precio** de Polymarket (mid del order book CLOB).
|
||||
2. **Ajuste bayesiano en log-odds** con señales independientes: GNews (con
|
||||
presupuesto de 5 consultas/día y *guardrail* que limita el ajuste a
|
||||
prior±0,25), Fear&Greed, momentum BTC, dominancia BTC, y Manifold Markets
|
||||
como señal cruzada (en modo observacional tras detectarse un bug de
|
||||
inversión — auditado en 165k filas).
|
||||
3. **Gates en cascada** antes de operar: prior extremo (<0,08 / >0,92), edge
|
||||
bruto mínimo por régimen de volatilidad, edge neto positivo tras comisión
|
||||
(2%) y spread, confianza ≥0,55, conflicto de familias, guard de reentrada.
|
||||
4. **Agrupación por familias de mercados**: mercados hermanos del mismo evento
|
||||
(candidatos de una misma elección, umbrales de un mismo precio) comparten
|
||||
`family_key`; solo se mantiene la posición de mayor edge para no apilar la
|
||||
misma apuesta con dos nombres.
|
||||
5. **Sizing por Kelly fraccionado** con techo por posición sobre un bankroll
|
||||
simulado de $10.000.
|
||||
|
||||
Resultado honesto en 81 días: 12 trades (n estadísticamente irrelevante, y el
|
||||
informe lo subraya), realized **+$247,78**, y — más interesante — **~223.000
|
||||
evaluaciones archivadas con su decisión reproducible**, porque el sistema
|
||||
prefirió no operar antes que operar sin ventaja medible.
|
||||
Detalle completo: [docs/informe-final.md](docs/informe-final.md).
|
||||
|
||||
## Arquitectura
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph ext["Fuentes externas"]
|
||||
PM["Polymarket<br/>CLOB + Gamma API"]
|
||||
GN["GNews"]
|
||||
MF["Manifold Markets"]
|
||||
CG["CoinGecko /<br/>alternative.me"]
|
||||
end
|
||||
|
||||
subgraph k3s["k3s · namespace polymarket-bot"]
|
||||
BOT["bot (ciclo ~64s)<br/>estrategia bayesiana + gates"]
|
||||
PG[("PostgreSQL 16<br/>trades · signals · replay")]
|
||||
API["api (FastAPI :8000)"]
|
||||
DASH["dashboard (React/nginx)"]
|
||||
CJ1["CronJob metrics-retention 00:10"]
|
||||
CJ2["CronJob outcomes-joiner 00:30"]
|
||||
end
|
||||
|
||||
subgraph obs["Observabilidad"]
|
||||
GRAF["Grafana"]
|
||||
UK["uptime-kuma"]
|
||||
TG["Telegram"]
|
||||
end
|
||||
|
||||
PM --> BOT
|
||||
GN --> BOT
|
||||
MF --> BOT
|
||||
CG --> BOT
|
||||
BOT --> PG
|
||||
PG --> API
|
||||
API --> DASH
|
||||
API --> GRAF
|
||||
UK --> DASH
|
||||
BOT --> TG
|
||||
PM --> CJ2
|
||||
CJ2 --> PG
|
||||
CJ1 --> PG
|
||||
|
||||
subgraph ci["GitOps"]
|
||||
GA["Gitea Actions"] --> REG["registry"] --> ARGO["ArgoCD<br/>prune + selfHeal"]
|
||||
end
|
||||
ARGO -.-> k3s
|
||||
```
|
||||
|
||||
## Decisiones de diseño
|
||||
|
||||
- **Paper trading por diseño, no como demo**: el bot exigía ≥10 mercados
|
||||
resueltos y ≥30 días antes de considerarse promocionable a dinero real, y
|
||||
reportaba `n/a (insufficient_sample)` en vez de métricas infladas.
|
||||
- **Todo skip es un dato**: cada evaluación archiva prior, estimación, edge
|
||||
bruto/neto, descomposición por feature en log-odds y el gate exacto que la
|
||||
bloqueó (~55k filas/día en `signals`).
|
||||
- **Replay determinista** (julio 2026): inyección de reloj + re-ejecución de
|
||||
ciclos archivados con 22.697/22.697 decisiones idénticas; joiner diario de
|
||||
resoluciones UMA para calibrar sobre *todas* las evaluaciones, no solo los
|
||||
trades.
|
||||
- **Los errores se cierran y se documentan**: el bug de inversión de Manifold
|
||||
y los conflictos de familia cerraron 5 posiciones en abril; la señal pasó a
|
||||
observacional-only con auditoría completa en vez de eliminarse.
|
||||
- **GitOps estricto**: nunca `kubectl` para cambios persistentes (ArgoCD con
|
||||
`selfHeal` revierte cualquier parche manual); secrets fuera de git vía
|
||||
Infisical operator; smoke test PostSync con notificación a Telegram.
|
||||
|
||||
## Componentes
|
||||
|
||||
@@ -11,10 +122,19 @@ dashboard React. Corre en k3s vía GitOps (Gitea Actions → registry → ArgoCD
|
||||
| api | `api/` (+ `bot/` como librería) | `polymarket-bot-api` | `uvicorn api.main:app` |
|
||||
| dashboard | `dashboard/` | `polymarket-bot-dashboard` | nginx estático |
|
||||
|
||||
Dashboard: https://polymarket.chemavx.xyz
|
||||
Dashboard (hasta el apagado): https://polymarket.chemavx.xyz
|
||||
|
||||
## Stack
|
||||
|
||||
Python 3 (asyncio) · FastAPI · React + Vite · PostgreSQL 16 · k3s · ArgoCD ·
|
||||
Gitea Actions + BuildKit · Infisical (secrets) · Grafana + Prometheus ·
|
||||
uptime-kuma · Telegram Bot API.
|
||||
|
||||
## CI/CD
|
||||
|
||||
> ⚠️ **Decomisión**: el trigger automático `push:main` se desactivó en la Fase 1
|
||||
> del archivado; el workflow queda solo bajo `workflow_dispatch` manual.
|
||||
|
||||
`.gitea/workflows/ci.yml` construye **solo las imágenes cuyos ficheros
|
||||
cambiaron** en el push (diff contra `github.event.before`):
|
||||
|
||||
@@ -33,3 +153,17 @@ sincroniza vía webhook en segundos.
|
||||
```bash
|
||||
python3 -m pytest tests/ -q
|
||||
```
|
||||
|
||||
## Datos y backup
|
||||
|
||||
Base de datos respaldada y verificada (2026-07-06): `pg_dump -Fc` + exports
|
||||
CSV.gz de las tablas de señales y auditoría, con checksums y restore de prueba
|
||||
(recuentos idénticos 11/11 tablas). Ubicación: almacenamiento de backups del
|
||||
clúster con sync offsite. El dump final se toma en la fase de apagado.
|
||||
|
||||
## Documentación
|
||||
|
||||
- [Informe final de paper trading](docs/informe-final.md) — resultados,
|
||||
pipeline de datos y estado de la BD al cierre.
|
||||
- `docs/pivot-manifold.md` — notas de diseño del posible pivote (Fase 5,
|
||||
pendiente).
|
||||
|
||||
+245
@@ -650,6 +650,251 @@ class Database:
|
||||
cooldown_reason = EXCLUDED.cooldown_reason
|
||||
""", poly_market_id, last_status, retry_after, cooldown_reason)
|
||||
|
||||
# ── Replay R0: snapshot recorder ─────────────────────────────────────────
|
||||
|
||||
async def save_ext_snapshot(self, cycle_ts, ext) -> None:
|
||||
"""Persist the ExternalSignals snapshot for one cycle (Replay R0)."""
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.execute("""
|
||||
INSERT INTO ext_snapshots (
|
||||
cycle_ts, btc_price, btc_change_24h, eth_price, eth_change_24h,
|
||||
btc_dominance, fear_greed_index, fear_greed_label,
|
||||
total_market_cap_change, valid
|
||||
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
|
||||
ON CONFLICT (cycle_ts) DO NOTHING
|
||||
""",
|
||||
cycle_ts, ext.btc_price, ext.btc_change_24h,
|
||||
ext.eth_price, ext.eth_change_24h, ext.btc_dominance,
|
||||
ext.fear_greed_index, ext.fear_greed_label,
|
||||
ext.total_market_cap_change, ext.valid,
|
||||
)
|
||||
|
||||
async def upsert_markets(self, markets: list) -> None:
|
||||
"""Refresh market metadata (Replay R0) — replay rebuilds Market from here."""
|
||||
rows = [
|
||||
(m.id, m.condition_id, m.question, m.category, m.end_date, m.active)
|
||||
for m in markets
|
||||
]
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.executemany("""
|
||||
INSERT INTO markets (id, condition_id, question, category, end_date, active, last_seen)
|
||||
VALUES ($1,$2,$3,$4,$5,$6, now())
|
||||
ON CONFLICT (id) DO UPDATE SET
|
||||
condition_id = EXCLUDED.condition_id,
|
||||
question = EXCLUDED.question,
|
||||
category = EXCLUDED.category,
|
||||
end_date = EXCLUDED.end_date,
|
||||
active = EXCLUDED.active,
|
||||
last_seen = now()
|
||||
""", rows)
|
||||
|
||||
async def save_signal_records(self, cycle_ts, records: list[dict]) -> None:
|
||||
"""Batch-insert one cycle's decision records into signals (Replay R0)."""
|
||||
if not records:
|
||||
return
|
||||
rows = [
|
||||
(
|
||||
r["market_id"], cycle_ts, cycle_ts,
|
||||
r["polymarket_price"], r["category"], r["volume_24h"],
|
||||
r["skip_reason"], r["family_key"],
|
||||
r["prior_prob"], r["estimated_prob"], r["raw_final_prob"],
|
||||
r["edge_gross"], r["edge_net"], r["regime_min_edge"],
|
||||
r["days_to_resolution"], r["confidence"], r["direction"],
|
||||
r["passed_gross"], r["passed_net"],
|
||||
r["news_sentiment"], r["news_budget_skipped"],
|
||||
r["guardrail_applied"], r["guardrail_changed_decision"],
|
||||
r["feat_fg_lo"], r["feat_mom_lo"], r["feat_news_lo"],
|
||||
r["feat_mfld_lo"], r["feat_btc_dom_lo"],
|
||||
r["edge_gross"], # legacy `edge` column mirrors edge_gross
|
||||
r["acted_on"],
|
||||
)
|
||||
for r in records
|
||||
]
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.executemany("""
|
||||
INSERT INTO signals (
|
||||
market_id, timestamp, cycle_ts,
|
||||
polymarket_price, category, volume_24h,
|
||||
skip_reason, family_key,
|
||||
prior_prob, estimated_prob, raw_final_prob,
|
||||
edge_gross, edge_net, regime_min_edge,
|
||||
days_to_resolution, confidence, direction,
|
||||
passed_gross, passed_net,
|
||||
news_sentiment, news_budget_skipped,
|
||||
guardrail_applied, guardrail_changed_decision,
|
||||
feat_fg_lo, feat_mom_lo, feat_news_lo,
|
||||
feat_mfld_lo, feat_btc_dom_lo,
|
||||
edge, acted_on
|
||||
) VALUES (
|
||||
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,
|
||||
$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30
|
||||
)
|
||||
""", rows)
|
||||
|
||||
async def prune_signal_records(self, retention_days: int) -> int:
|
||||
"""Delete archive rows older than retention_days; returns rows deleted."""
|
||||
async with self._pool.acquire() as conn:
|
||||
result = await conn.execute(
|
||||
"DELETE FROM signals WHERE timestamp < now() - ($1 || ' days')::interval",
|
||||
str(retention_days),
|
||||
)
|
||||
await conn.execute(
|
||||
"DELETE FROM ext_snapshots WHERE cycle_ts < now() - ($1 || ' days')::interval",
|
||||
str(retention_days),
|
||||
)
|
||||
try:
|
||||
return int(result.split()[-1])
|
||||
except (ValueError, IndexError):
|
||||
return 0
|
||||
|
||||
# ── Replay R1: replay core ───────────────────────────────────────────────
|
||||
|
||||
async def get_replay_cycles(self, from_ts, to_ts) -> list:
|
||||
"""Return the cycle_ts values with archived decisions in [from_ts, to_ts)."""
|
||||
async with self._pool.acquire() as conn:
|
||||
rows = await conn.fetch("""
|
||||
SELECT DISTINCT cycle_ts FROM signals
|
||||
WHERE cycle_ts >= $1 AND cycle_ts < $2
|
||||
ORDER BY cycle_ts
|
||||
""", from_ts, to_ts)
|
||||
return [r["cycle_ts"] for r in rows]
|
||||
|
||||
async def get_ext_snapshot(self, cycle_ts) -> Optional[dict]:
|
||||
"""Return one cycle's ExternalSignals snapshot, or None if missing."""
|
||||
async with self._pool.acquire() as conn:
|
||||
row = await conn.fetchrow(
|
||||
"SELECT * FROM ext_snapshots WHERE cycle_ts = $1", cycle_ts
|
||||
)
|
||||
return dict(row) if row else None
|
||||
|
||||
async def get_cycle_signal_rows(self, cycle_ts) -> list[dict]:
|
||||
"""Return one cycle's archived decision rows in original evaluation
|
||||
order (id = insertion order = the order main.py evaluated them)."""
|
||||
async with self._pool.acquire() as conn:
|
||||
rows = await conn.fetch(
|
||||
"SELECT * FROM signals WHERE cycle_ts = $1 ORDER BY id", cycle_ts
|
||||
)
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
async def get_markets_by_ids(self, market_ids: list[str]) -> dict[str, dict]:
|
||||
"""Return market metadata rows keyed by id (for Market reconstruction)."""
|
||||
if not market_ids:
|
||||
return {}
|
||||
async with self._pool.acquire() as conn:
|
||||
rows = await conn.fetch(
|
||||
"SELECT * FROM markets WHERE id = ANY($1::text[])", market_ids
|
||||
)
|
||||
return {r["id"]: dict(r) for r in rows}
|
||||
|
||||
async def save_replay_run(self, run: dict) -> None:
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.execute("""
|
||||
INSERT INTO replay_runs (
|
||||
run_id, git_sha, config_hash, config_json,
|
||||
from_ts, to_ts, cycles, decisions, matched, mismatched, note
|
||||
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11)
|
||||
""",
|
||||
run["run_id"], run["git_sha"], run["config_hash"],
|
||||
run["config_json"], run["from_ts"], run["to_ts"],
|
||||
run["cycles"], run["decisions"], run["matched"],
|
||||
run["mismatched"], run["note"],
|
||||
)
|
||||
|
||||
async def save_replay_decisions(self, run_id: str, decisions: list[dict]) -> None:
|
||||
if not decisions:
|
||||
return
|
||||
rows = [
|
||||
(
|
||||
run_id, d["cycle_ts"], d["market_id"],
|
||||
d["skip_reason"], d["prior_prob"], d["estimated_prob"],
|
||||
d["raw_final_prob"], d["edge_gross"], d["edge_net"],
|
||||
d["regime_min_edge"], d["days_to_resolution"],
|
||||
d["confidence"], d["direction"], d["would_trade"],
|
||||
d["recorded_skip_reason"], d["matched"], d["mismatch_field"],
|
||||
)
|
||||
for d in decisions
|
||||
]
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.executemany("""
|
||||
INSERT INTO replay_decisions (
|
||||
run_id, cycle_ts, market_id,
|
||||
skip_reason, prior_prob, estimated_prob,
|
||||
raw_final_prob, edge_gross, edge_net,
|
||||
regime_min_edge, days_to_resolution,
|
||||
confidence, direction, would_trade,
|
||||
recorded_skip_reason, matched, mismatch_field
|
||||
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17)
|
||||
""", rows)
|
||||
|
||||
# ── Replay R2: outcomes + calibration metrics ────────────────────────────
|
||||
|
||||
async def get_unresolved_archived_market_ids(self) -> list[str]:
|
||||
"""Archived markets (present in signals) with no stored outcome yet."""
|
||||
async with self._pool.acquire() as conn:
|
||||
rows = await conn.fetch("""
|
||||
SELECT DISTINCT s.market_id FROM signals s
|
||||
LEFT JOIN market_outcomes o ON o.market_id = s.market_id
|
||||
WHERE o.market_id IS NULL
|
||||
ORDER BY s.market_id
|
||||
""")
|
||||
return [r["market_id"] for r in rows]
|
||||
|
||||
async def upsert_market_outcome(
|
||||
self, market_id: str, outcome: float, resolved_at
|
||||
) -> None:
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.execute("""
|
||||
INSERT INTO market_outcomes (market_id, outcome, resolved_at)
|
||||
VALUES ($1, $2, $3)
|
||||
ON CONFLICT (market_id) DO UPDATE
|
||||
SET outcome = EXCLUDED.outcome,
|
||||
resolved_at = EXCLUDED.resolved_at,
|
||||
fetched_at = NOW()
|
||||
""", market_id, outcome, resolved_at)
|
||||
|
||||
async def get_outcome_coverage(self) -> dict:
|
||||
"""How much of the archive is scorable: resolved vs archived markets."""
|
||||
async with self._pool.acquire() as conn:
|
||||
row = await conn.fetchrow("""
|
||||
SELECT
|
||||
(SELECT COUNT(DISTINCT market_id) FROM signals) AS archived,
|
||||
(SELECT COUNT(*) FROM market_outcomes
|
||||
WHERE market_id IN (SELECT DISTINCT market_id FROM signals)
|
||||
) AS resolved
|
||||
""")
|
||||
return dict(row)
|
||||
|
||||
async def get_calibration_rows(self, run_id: Optional[str] = None) -> list[dict]:
|
||||
"""Every archived evaluation with a full estimate AND a known outcome.
|
||||
|
||||
run_id None scores the R0 archive (signals); a run_id scores that
|
||||
replay run's re-estimates instead (counterfactual calibration).
|
||||
Rows without estimated_prob (skipped before estimation: prior_extreme,
|
||||
unsupported, family, no_signals) carry no model prediction to score.
|
||||
"""
|
||||
async with self._pool.acquire() as conn:
|
||||
if run_id is None:
|
||||
rows = await conn.fetch("""
|
||||
SELECT s.market_id, s.category,
|
||||
s.estimated_prob, s.prior_prob, o.outcome
|
||||
FROM signals s
|
||||
JOIN market_outcomes o ON o.market_id = s.market_id
|
||||
WHERE s.estimated_prob IS NOT NULL
|
||||
AND s.prior_prob IS NOT NULL
|
||||
""")
|
||||
else:
|
||||
rows = await conn.fetch("""
|
||||
SELECT d.market_id, m.category,
|
||||
d.estimated_prob, d.prior_prob, o.outcome
|
||||
FROM replay_decisions d
|
||||
JOIN market_outcomes o ON o.market_id = d.market_id
|
||||
LEFT JOIN markets m ON m.id = d.market_id
|
||||
WHERE d.run_id = $1
|
||||
AND d.estimated_prob IS NOT NULL
|
||||
AND d.prior_prob IS NOT NULL
|
||||
""", run_id)
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
async def mark_manifold_audit_used(self, audit_id: str) -> None:
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.execute(
|
||||
|
||||
@@ -318,3 +318,137 @@ CREATE TABLE IF NOT EXISTS manifold_eval_cooldown (
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_mfld_cooldown_retry ON manifold_eval_cooldown(retry_after);
|
||||
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
-- Replay R0: snapshot recorder — the archive the replay engine reads from
|
||||
--
|
||||
-- The signals table (Phase 2/5 schema) never had a writer; R0 makes it the
|
||||
-- per-(market, cycle) decision archive. One row per evaluated market per
|
||||
-- cycle, carrying both the INPUTS the strategy saw (external signals, news
|
||||
-- sentiment, per-feature log-odds) and the OUTPUTS it produced (probs, edges,
|
||||
-- gates, skip_reason). A replay run rebuilds Market/ExternalSignals from
|
||||
-- these rows plus ext_snapshots and re-executes evaluate() deterministically.
|
||||
--
|
||||
-- cycle_ts groups all rows of one trading cycle and joins them to their
|
||||
-- ext_snapshots row (same timestamp; no FK to keep writes independent).
|
||||
-- days_to_resolution is persisted so replay does not depend on wall-clock.
|
||||
-- news_budget_skipped distinguishes "GNews had nothing" from "GNews was not
|
||||
-- asked this cycle" (5-query budget) — without it politics replay would treat
|
||||
-- budget starvation as absence of news.
|
||||
-- Retention: rows older than SIGNALS_RETENTION_DAYS (default 90) are pruned.
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS cycle_ts TIMESTAMPTZ;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS category TEXT;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS prior_prob DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS raw_final_prob DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS days_to_resolution INTEGER;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS volume_24h DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS news_sentiment DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS news_budget_skipped BOOLEAN;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS guardrail_applied BOOLEAN;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS guardrail_changed_decision BOOLEAN;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS feat_fg_lo DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS feat_mom_lo DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS feat_news_lo DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS feat_mfld_lo DOUBLE PRECISION;
|
||||
ALTER TABLE signals ADD COLUMN IF NOT EXISTS feat_btc_dom_lo DOUBLE PRECISION;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_signals_cycle ON signals(cycle_ts);
|
||||
|
||||
-- One row per trading cycle: the ExternalSignals snapshot every market in
|
||||
-- that cycle was evaluated against. Written once per cycle before the
|
||||
-- evaluation loop; signals rows join on cycle_ts.
|
||||
CREATE TABLE IF NOT EXISTS ext_snapshots (
|
||||
cycle_ts TIMESTAMPTZ PRIMARY KEY,
|
||||
btc_price DOUBLE PRECISION,
|
||||
btc_change_24h DOUBLE PRECISION,
|
||||
eth_price DOUBLE PRECISION,
|
||||
eth_change_24h DOUBLE PRECISION,
|
||||
btc_dominance DOUBLE PRECISION,
|
||||
fear_greed_index INTEGER,
|
||||
fear_greed_label TEXT,
|
||||
total_market_cap_change DOUBLE PRECISION,
|
||||
valid BOOLEAN
|
||||
);
|
||||
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
-- Replay R1: replay core — re-execute evaluate() over the R0 archive
|
||||
--
|
||||
-- A replay run reads cycles from signals + ext_snapshots + markets, rebuilds
|
||||
-- the exact inputs (including archived news_sentiment — GNews is never called),
|
||||
-- re-runs BayesianStrategy.evaluate() with the archived cycle_ts as clock, and
|
||||
-- writes one replay_decisions row per (cycle, market).
|
||||
--
|
||||
-- replay_runs tags every run with the code (git_sha) and strategy constants
|
||||
-- (config_hash) that produced it: two runs over the same window with different
|
||||
-- config_hash values are a counterfactual comparison; same config_hash against
|
||||
-- the recorded rows is a determinism check (mismatches should be 0, modulo
|
||||
-- day-boundary crossings between cycle_ts and the original wall-clock).
|
||||
--
|
||||
-- matched: replayed decision equals the recorded one (skip_reason, probs,
|
||||
-- confidence, direction). NULL when not comparable — e.g. reentry_guard
|
||||
-- rows, recorded outside evaluate() with no decision fields to compare;
|
||||
-- the replay still re-evaluates them, which is extra calibration data.
|
||||
-- mismatch_field: first field that differed, for triage.
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
CREATE TABLE IF NOT EXISTS replay_runs (
|
||||
run_id TEXT PRIMARY KEY,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
git_sha TEXT,
|
||||
config_hash TEXT,
|
||||
config_json TEXT,
|
||||
from_ts TIMESTAMPTZ,
|
||||
to_ts TIMESTAMPTZ,
|
||||
cycles INTEGER,
|
||||
decisions INTEGER,
|
||||
matched INTEGER,
|
||||
mismatched INTEGER,
|
||||
note TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS replay_decisions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
run_id TEXT NOT NULL,
|
||||
cycle_ts TIMESTAMPTZ NOT NULL,
|
||||
market_id TEXT NOT NULL,
|
||||
-- replayed outputs (same semantics as the signals columns)
|
||||
skip_reason TEXT,
|
||||
prior_prob DOUBLE PRECISION,
|
||||
estimated_prob DOUBLE PRECISION,
|
||||
raw_final_prob DOUBLE PRECISION,
|
||||
edge_gross DOUBLE PRECISION,
|
||||
edge_net DOUBLE PRECISION,
|
||||
regime_min_edge DOUBLE PRECISION,
|
||||
days_to_resolution INTEGER,
|
||||
confidence DOUBLE PRECISION,
|
||||
direction TEXT,
|
||||
would_trade BOOLEAN,
|
||||
-- fidelity vs the recorded signals row
|
||||
recorded_skip_reason TEXT,
|
||||
matched BOOLEAN,
|
||||
mismatch_field TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_replay_decisions_run ON replay_decisions(run_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_replay_decisions_mkt ON replay_decisions(market_id);
|
||||
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
-- Replay R2: outcomes + calibration metrics
|
||||
--
|
||||
-- One row per resolved market, fetched from the Gamma API via
|
||||
-- get_market_resolution() (UMA-final only: a market closed but still in
|
||||
-- proposal/dispute is not stored). outcome is the final YES price:
|
||||
-- 1.0 = YES won, 0.0 = NO won.
|
||||
--
|
||||
-- Joining signals (or replay_decisions) to market_outcomes scores every
|
||||
-- archived estimate against reality — Brier / log-loss of estimated_prob
|
||||
-- benchmarked against the market price (prior_prob) on the same rows,
|
||||
-- answering "does the model add value over the market?" across ALL
|
||||
-- evaluations, not just executed trades.
|
||||
-- ─────────────────────────────────────────────────────────────────────────────
|
||||
CREATE TABLE IF NOT EXISTS market_outcomes (
|
||||
market_id TEXT PRIMARY KEY,
|
||||
outcome DOUBLE PRECISION NOT NULL,
|
||||
resolved_at TIMESTAMPTZ,
|
||||
fetched_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
+43
@@ -43,6 +43,14 @@ PAPER_BANKROLL = float(os.getenv("PAPER_BANKROLL", "10000"))
|
||||
# position per 10 minutes.
|
||||
RESOLUTION_CHECK_INTERVAL = 10
|
||||
|
||||
# Replay R0: persist per-(market, cycle) decision records + the ExternalSignals
|
||||
# snapshot each cycle, so the replay engine can re-run past decisions. The
|
||||
# recorder must never break trading — every write is wrapped in try/except.
|
||||
SIGNAL_RECORDER_ENABLED = os.getenv("SIGNAL_RECORDER_ENABLED", "true").lower() == "true"
|
||||
SIGNALS_RETENTION_DAYS = int(os.getenv("SIGNALS_RETENTION_DAYS", "90"))
|
||||
# Prune the archive roughly once a day at the 60s cycle cadence.
|
||||
SIGNALS_PRUNE_INTERVAL_CYCLES = 1440
|
||||
|
||||
|
||||
async def check_resolutions(
|
||||
poly: PolymarketClient,
|
||||
@@ -122,6 +130,16 @@ async def run_trading_loop(
|
||||
# 2. Get external signals
|
||||
ext_data = await external.get_all_signals()
|
||||
|
||||
# 2b. Replay R0: archive this cycle's inputs (ext snapshot + market
|
||||
# metadata). cycle_ts groups all signals rows of this cycle.
|
||||
cycle_ts = datetime.now(timezone.utc)
|
||||
if SIGNAL_RECORDER_ENABLED:
|
||||
try:
|
||||
await db.save_ext_snapshot(cycle_ts, ext_data)
|
||||
await db.upsert_markets(markets)
|
||||
except Exception as exc:
|
||||
log.warning("Signal recorder (inputs) failed: %s", exc)
|
||||
|
||||
# 3. Build occupied_families from the current open portfolio positions.
|
||||
# This prevents re-entering a family where we already hold a position.
|
||||
# We also pull from DB to survive pod restarts.
|
||||
@@ -176,6 +194,7 @@ async def run_trading_loop(
|
||||
|
||||
reentry_guard_count = 0
|
||||
cycle_trades = 0
|
||||
traded_market_ids: set[str] = set()
|
||||
for market in markets:
|
||||
if market.id in inverted_guard:
|
||||
log.info(
|
||||
@@ -183,6 +202,7 @@ async def run_trading_loop(
|
||||
market.id, market.question[:60],
|
||||
)
|
||||
reentry_guard_count += 1
|
||||
strategy.record_skip(market, "reentry_guard")
|
||||
continue
|
||||
|
||||
# evaluate() returns None for all skips — reasons are logged internally
|
||||
@@ -214,6 +234,7 @@ async def run_trading_loop(
|
||||
# Block this family for the rest of the cycle (Phase 2)
|
||||
occupied_families.add(signal.family_key)
|
||||
cycle_trades += 1
|
||||
traded_market_ids.add(market.id)
|
||||
# Mark manifold audit record as used in this trade
|
||||
if signal.mfld_audit_id:
|
||||
try:
|
||||
@@ -221,6 +242,28 @@ async def run_trading_loop(
|
||||
except Exception as exc:
|
||||
log.warning("Failed to mark manifold audit used: %s", exc)
|
||||
|
||||
# 7b. Replay R0: flush this cycle's decision records to the archive.
|
||||
# acted_on marks records whose signal actually became a trade
|
||||
# (evaluate() can emit a signal that risk sizing later rejects).
|
||||
records = strategy.drain_cycle_records()
|
||||
if SIGNAL_RECORDER_ENABLED and records:
|
||||
for rec in records:
|
||||
if rec["market_id"] in traded_market_ids:
|
||||
rec["acted_on"] = True
|
||||
try:
|
||||
await db.save_signal_records(cycle_ts, records)
|
||||
except Exception as exc:
|
||||
log.warning("Signal recorder (records) failed: %s", exc)
|
||||
if cycle_count % SIGNALS_PRUNE_INTERVAL_CYCLES == 1:
|
||||
try:
|
||||
pruned = await db.prune_signal_records(SIGNALS_RETENTION_DAYS)
|
||||
log.info(
|
||||
"Signal archive pruned: %d rows older than %d days removed",
|
||||
pruned, SIGNALS_RETENTION_DAYS,
|
||||
)
|
||||
except Exception as exc:
|
||||
log.warning("Signal archive prune failed: %s", exc)
|
||||
|
||||
# 8. [CYCLE SUMMARY] — one block per cycle, stable format for grep/compare
|
||||
stats = strategy.get_cycle_stats()
|
||||
legacy_incomplete_count = await db.get_legacy_incomplete_count()
|
||||
|
||||
+208
@@ -0,0 +1,208 @@
|
||||
"""
|
||||
Replay R2 — outcomes + calibration metrics.
|
||||
|
||||
Two phases, one CLI:
|
||||
|
||||
1. Fetch: for every archived market (present in `signals`) without a stored
|
||||
outcome, ask the Gamma API via PolymarketClient.get_market_resolution()
|
||||
— the same UMA-finality gate the trading loop uses to settle positions.
|
||||
Definitive resolutions are upserted into `market_outcomes`; open, disputed
|
||||
or ambiguous markets are simply retried on the next invocation. There is
|
||||
no data-loss urgency here (unlike the R0 recorder): Gamma reports past
|
||||
resolutions at any time, so running this lazily loses nothing.
|
||||
|
||||
2. Score: join archived estimates to outcomes and compute Brier / log-loss of
|
||||
estimated_prob, benchmarked against the market price (prior_prob) on the
|
||||
same rows. This scores ALL evaluations with a full estimate — the sample
|
||||
multiplier the phase plan calls for — not just executed trades. With
|
||||
--run-id it scores a replay run's re-estimates instead (counterfactual
|
||||
calibration: did config X predict better than the market?).
|
||||
|
||||
Reading the numbers: lower is better for both metrics; model < prior means
|
||||
the model added information over the market price. Micro averages weight
|
||||
every evaluation equally, so long-lived markets (~1 evaluation/min while in
|
||||
the universe) dominate; macro averages score each market once (mean of its
|
||||
evaluations) and answer the same question per market. Evaluations of one
|
||||
market minutes apart are highly autocorrelated — n_evaluations overstates
|
||||
the effective sample size, n_markets is the honest one.
|
||||
|
||||
CLI:
|
||||
python -m bot.outcomes # fetch new outcomes, then score archive
|
||||
python -m bot.outcomes --fetch-only
|
||||
python -m bot.outcomes --metrics-only
|
||||
python -m bot.outcomes --run-id UUID # score a replay run (implies no fetch)
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import logging
|
||||
import math
|
||||
from collections import defaultdict
|
||||
from typing import Optional
|
||||
|
||||
from bot.data.db import Database
|
||||
from bot.data.polymarket import PolymarketClient
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Clip probabilities before log() so a (theoretical) hard 0/1 estimate on a
|
||||
# wrong outcome scores ~20.7 nats instead of infinity poisoning the mean.
|
||||
LOGLOSS_EPS = 1e-9
|
||||
|
||||
|
||||
async def fetch_outcomes(poly, market_ids: list[str]) -> list[dict]:
|
||||
"""Resolve archived markets against Gamma; returns only definitive ones.
|
||||
|
||||
Sequential on purpose: ~50 markets per invocation, and the Gamma API has
|
||||
no bulk endpoint. get_market_resolution() already returns None on API
|
||||
errors and resolved=False on open/disputed/ambiguous markets.
|
||||
"""
|
||||
resolved = []
|
||||
for market_id in market_ids:
|
||||
res = await poly.get_market_resolution(market_id)
|
||||
if res is None or not res.resolved or res.resolution is None:
|
||||
continue
|
||||
resolved.append({
|
||||
"market_id": market_id,
|
||||
"outcome": res.resolution,
|
||||
"resolved_at": res.resolved_at,
|
||||
})
|
||||
return resolved
|
||||
|
||||
|
||||
def _logloss(p: float, outcome: float) -> float:
|
||||
p = min(max(p, LOGLOSS_EPS), 1.0 - LOGLOSS_EPS)
|
||||
return -math.log(p) if outcome == 1.0 else -math.log(1.0 - p)
|
||||
|
||||
|
||||
def compute_calibration(rows: list[dict]) -> Optional[dict]:
|
||||
"""Score estimated_prob vs prior_prob against outcomes; None if no rows.
|
||||
|
||||
rows: dicts with market_id, category, estimated_prob, prior_prob, outcome.
|
||||
Pure function — the CLI feeds it DB rows, tests feed it literals.
|
||||
"""
|
||||
if not rows:
|
||||
return None
|
||||
|
||||
n = len(rows)
|
||||
brier_model = sum((r["estimated_prob"] - r["outcome"]) ** 2 for r in rows) / n
|
||||
brier_prior = sum((r["prior_prob"] - r["outcome"]) ** 2 for r in rows) / n
|
||||
logloss_model = sum(_logloss(r["estimated_prob"], r["outcome"]) for r in rows) / n
|
||||
logloss_prior = sum(_logloss(r["prior_prob"], r["outcome"]) for r in rows) / n
|
||||
|
||||
by_market: dict[str, list[dict]] = defaultdict(list)
|
||||
for r in rows:
|
||||
by_market[r["market_id"]].append(r)
|
||||
market_briers = [
|
||||
(
|
||||
sum((r["estimated_prob"] - r["outcome"]) ** 2 for r in mrows) / len(mrows),
|
||||
sum((r["prior_prob"] - r["outcome"]) ** 2 for r in mrows) / len(mrows),
|
||||
)
|
||||
for mrows in by_market.values()
|
||||
]
|
||||
brier_model_macro = sum(b[0] for b in market_briers) / len(market_briers)
|
||||
brier_prior_macro = sum(b[1] for b in market_briers) / len(market_briers)
|
||||
|
||||
by_category: dict[str, list[dict]] = defaultdict(list)
|
||||
for r in rows:
|
||||
by_category[r["category"] or "unknown"].append(r)
|
||||
per_category = {
|
||||
cat: {
|
||||
"n": len(crows),
|
||||
"markets": len({r["market_id"] for r in crows}),
|
||||
"brier_model": sum((r["estimated_prob"] - r["outcome"]) ** 2
|
||||
for r in crows) / len(crows),
|
||||
"brier_prior": sum((r["prior_prob"] - r["outcome"]) ** 2
|
||||
for r in crows) / len(crows),
|
||||
}
|
||||
for cat, crows in sorted(by_category.items())
|
||||
}
|
||||
|
||||
return {
|
||||
"n_evaluations": n,
|
||||
"n_markets": len(by_market),
|
||||
"brier_model": brier_model,
|
||||
"brier_prior": brier_prior,
|
||||
"brier_model_macro": brier_model_macro,
|
||||
"brier_prior_macro": brier_prior_macro,
|
||||
"logloss_model": logloss_model,
|
||||
"logloss_prior": logloss_prior,
|
||||
"per_category": per_category,
|
||||
}
|
||||
|
||||
|
||||
def print_report(metrics: Optional[dict], source: str) -> None:
|
||||
if metrics is None:
|
||||
print(f"calibration : no scorable rows yet for {source} "
|
||||
"(no archived estimate has a resolved outcome)")
|
||||
return
|
||||
print(f"calibration : {source} — {metrics['n_evaluations']} evaluations, "
|
||||
f"{metrics['n_markets']} markets")
|
||||
print(f"{'':14s}{'model':>10s}{'market':>10s}{'delta':>10s}")
|
||||
for label, m_key, p_key in (
|
||||
("Brier micro", "brier_model", "brier_prior"),
|
||||
("Brier macro", "brier_model_macro", "brier_prior_macro"),
|
||||
("logloss micro", "logloss_model", "logloss_prior"),
|
||||
):
|
||||
m, p = metrics[m_key], metrics[p_key]
|
||||
print(f" {label:12s}{m:>10.4f}{p:>10.4f}{m - p:>+10.4f}")
|
||||
print(" (delta < 0 = model beats the market price)")
|
||||
for cat, c in metrics["per_category"].items():
|
||||
print(f" {cat:12s}n={c['n']:<6d} markets={c['markets']:<3d} "
|
||||
f"brier model {c['brier_model']:.4f} vs market {c['brier_prior']:.4f}")
|
||||
|
||||
|
||||
async def _amain(args: argparse.Namespace) -> None:
|
||||
db = Database()
|
||||
await db.connect()
|
||||
try:
|
||||
if not args.metrics_only and args.run_id is None:
|
||||
pending = await db.get_unresolved_archived_market_ids()
|
||||
poly = PolymarketClient()
|
||||
try:
|
||||
resolved = await fetch_outcomes(poly, pending)
|
||||
finally:
|
||||
await poly.close()
|
||||
for out in resolved:
|
||||
await db.upsert_market_outcome(
|
||||
out["market_id"], out["outcome"], out["resolved_at"]
|
||||
)
|
||||
print(f"outcomes : {len(resolved)} newly resolved "
|
||||
f"(of {len(pending)} pending markets checked)")
|
||||
|
||||
coverage = await db.get_outcome_coverage()
|
||||
print(f"coverage : {coverage['resolved']}/{coverage['archived']} "
|
||||
"archived markets resolved")
|
||||
|
||||
if args.fetch_only:
|
||||
return
|
||||
rows = await db.get_calibration_rows(run_id=args.run_id)
|
||||
source = f"replay run {args.run_id}" if args.run_id else "R0 archive"
|
||||
print_report(compute_calibration(rows), source)
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m bot.outcomes",
|
||||
description="Fetch market resolutions and score archived estimates.",
|
||||
)
|
||||
parser.add_argument("--fetch-only", action="store_true",
|
||||
help="only fetch/store outcomes, skip metrics")
|
||||
parser.add_argument("--metrics-only", action="store_true",
|
||||
help="skip the Gamma fetch, score what is stored")
|
||||
parser.add_argument("--run-id", default=None,
|
||||
help="score a replay run's re-estimates instead of "
|
||||
"the R0 archive (implies --metrics-only)")
|
||||
args = parser.parse_args()
|
||||
if args.fetch_only and args.metrics_only:
|
||||
parser.error("--fetch-only and --metrics-only are mutually exclusive")
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
asyncio.run(_amain(args))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+394
@@ -0,0 +1,394 @@
|
||||
"""
|
||||
Replay R1 — replay core.
|
||||
|
||||
Re-executes BayesianStrategy.evaluate() over the R0 archive (signals +
|
||||
ext_snapshots + markets) and stores the outcome in replay_runs /
|
||||
replay_decisions.
|
||||
|
||||
Determinism contract: evaluate() is a pure function of
|
||||
(market, ext, occupied_families, as_of) plus the news client, so a replay
|
||||
rebuilds exactly those four inputs from the archive:
|
||||
|
||||
market — metadata from `markets`, per-cycle price/volume from `signals`
|
||||
ext — the cycle's `ext_snapshots` row
|
||||
families — a family-skipped row replays with its own family_key occupied;
|
||||
every other row replays with no occupancy (the recorded
|
||||
skip_reason already reflects the original portfolio state)
|
||||
as_of — the archived cycle_ts (clock injection, Replay R1)
|
||||
|
||||
GNews is never called: ReplayNews feeds back the archived news_sentiment.
|
||||
The per-cycle query budget is bypassed (reset before every market) because
|
||||
the archived sentiment already encodes the budget's effect — a
|
||||
budget-skipped market was recorded with sentiment 0.0.
|
||||
|
||||
Manifold and the DB are not wired into the replayed strategy (manifold=None,
|
||||
db=None): the signal is observational-only in production (feat_mfld_lo is
|
||||
always 0.0 in the archive), so the replay reproduces decisions without
|
||||
touching cooldowns or audit tables. If MANIFOLD_SIGNAL_ENABLED is ever
|
||||
turned on, replayed decisions will diverge from recorded ones and the
|
||||
matched/mismatch_field columns will say so.
|
||||
|
||||
Run tagging: every run stores the git sha and a hash of the strategy
|
||||
constants. Same config_hash vs the archive = determinism check (expect 0
|
||||
mismatches, modulo UTC-day-boundary crossings between cycle_ts and the
|
||||
original wall-clock). Different config_hash = counterfactual run.
|
||||
|
||||
CLI:
|
||||
python -m bot.replay --from 2026-07-02T00:00:00Z --to 2026-07-03 --note "..."
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import uuid
|
||||
from collections import Counter
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
|
||||
import bot.strategy.bayesian as bayesian
|
||||
from bot.data.db import Database
|
||||
from bot.data.external import ExternalSignals
|
||||
from bot.data.polymarket import Market
|
||||
from bot.strategy.bayesian import BayesianStrategy
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Absolute float tolerance for recorded-vs-replayed comparison. Archived
|
||||
# values are float8 (exact IEEE-754 round-trip of Python floats), so any real
|
||||
# divergence is far larger than this.
|
||||
FLOAT_TOL = 1e-9
|
||||
|
||||
# Strategy constants that define a replay configuration. Hashed into
|
||||
# replay_runs.config_hash; read from the module at call time so a
|
||||
# counterfactual run can monkeypatch them and be tagged distinctly.
|
||||
CONFIG_KEYS = (
|
||||
"SPREAD_ESTIMATE",
|
||||
"COMMISSION_RATE",
|
||||
"MIN_CONFIDENCE",
|
||||
"NEWS_LOGODDS_WEIGHT",
|
||||
"MANIFOLD_LOGODDS_WEIGHT",
|
||||
"MANIFOLD_SIGNAL_ENABLED",
|
||||
"NEWS_GUARDRAIL_ENABLED",
|
||||
"MAX_NEWS_ONLY_PROB_SHIFT",
|
||||
"NEWS_MATERIAL_LOGODDS_THRESHOLD",
|
||||
"MAX_NEWS_QUERIES_PER_CYCLE",
|
||||
)
|
||||
|
||||
# Rows recorded outside evaluate() (via record_skip) carry no decision fields;
|
||||
# the replay still re-evaluates them for calibration but cannot compare.
|
||||
NON_COMPARABLE_SKIPS = {"reentry_guard"}
|
||||
|
||||
|
||||
def strategy_config() -> dict:
|
||||
return {k: getattr(bayesian, k) for k in CONFIG_KEYS}
|
||||
|
||||
|
||||
def strategy_config_hash() -> str:
|
||||
blob = json.dumps(strategy_config(), sort_keys=True)
|
||||
return hashlib.sha256(blob.encode()).hexdigest()[:12]
|
||||
|
||||
|
||||
def _git_sha() -> str:
|
||||
sha = os.getenv("GIT_SHA", "")
|
||||
if sha:
|
||||
return sha
|
||||
try:
|
||||
return subprocess.run(
|
||||
["git", "rev-parse", "--short", "HEAD"],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
).stdout.strip() or "unknown"
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return "unknown"
|
||||
|
||||
|
||||
class ReplayNews:
|
||||
"""NewsClient stand-in that feeds archived sentiment back into evaluate().
|
||||
|
||||
No HTTP, no cache: the engine sets `sentiment` to the archived value
|
||||
before each evaluate() call. Values below evaluate()'s 0.05 materiality
|
||||
threshold were archived as 0.0, so the round-trip is exact.
|
||||
"""
|
||||
enabled = True
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.sentiment: float = 0.0
|
||||
|
||||
async def get_sentiment(self, question: str) -> float:
|
||||
return self.sentiment
|
||||
|
||||
def get_freshness(self, question: str) -> float:
|
||||
return 1.0 # only used by gnews_priority(), which replay never calls
|
||||
|
||||
|
||||
def build_ext(snapshot: dict) -> ExternalSignals:
|
||||
"""Rebuild the ExternalSignals a cycle was evaluated against."""
|
||||
return ExternalSignals(
|
||||
btc_price=snapshot["btc_price"],
|
||||
btc_change_24h=snapshot["btc_change_24h"],
|
||||
eth_price=snapshot["eth_price"],
|
||||
eth_change_24h=snapshot["eth_change_24h"],
|
||||
btc_dominance=snapshot["btc_dominance"],
|
||||
fear_greed_index=snapshot["fear_greed_index"],
|
||||
fear_greed_label=snapshot["fear_greed_label"],
|
||||
total_market_cap_change=snapshot["total_market_cap_change"],
|
||||
valid=snapshot["valid"],
|
||||
)
|
||||
|
||||
|
||||
def build_market(market_row: dict, signal_row: dict) -> Market:
|
||||
"""Rebuild a Market: metadata from `markets`, per-cycle state from `signals`.
|
||||
|
||||
Token ids are irrelevant to evaluate() and left empty; no_price is the
|
||||
YES complement (evaluate() never reads it either).
|
||||
"""
|
||||
yes_price = signal_row["polymarket_price"]
|
||||
return Market(
|
||||
id=market_row["id"],
|
||||
condition_id=market_row["condition_id"] or "",
|
||||
question=market_row["question"],
|
||||
yes_token_id="",
|
||||
no_token_id="",
|
||||
yes_price=yes_price,
|
||||
no_price=1.0 - yes_price,
|
||||
volume_24h=signal_row["volume_24h"] or 0.0,
|
||||
end_date=market_row["end_date"] or "",
|
||||
active=True,
|
||||
category=signal_row["category"] or (market_row["category"] or ""),
|
||||
)
|
||||
|
||||
|
||||
def _compare(recorded: dict, replayed: dict) -> Optional[str]:
|
||||
"""Return the first field where replayed diverges from recorded, or None."""
|
||||
if recorded["skip_reason"] != replayed["skip_reason"]:
|
||||
return "skip_reason"
|
||||
for field in ("prior_prob", "estimated_prob", "raw_final_prob",
|
||||
"edge_net", "confidence"):
|
||||
a, b = recorded[field], replayed[field]
|
||||
if a is None and b is None:
|
||||
continue
|
||||
if a is None or b is None or abs(a - b) > FLOAT_TOL:
|
||||
return field
|
||||
if recorded["direction"] != replayed["direction"]:
|
||||
return "direction"
|
||||
return None
|
||||
|
||||
|
||||
async def replay_cycle(
|
||||
cycle_ts: datetime,
|
||||
snapshot: dict,
|
||||
signal_rows: list[dict],
|
||||
market_rows: dict[str, dict],
|
||||
) -> list[dict]:
|
||||
"""Re-evaluate one archived cycle; returns one decision dict per row.
|
||||
|
||||
Pure with respect to the DB — everything it needs is passed in, so tests
|
||||
can drive it with synthetic rows.
|
||||
"""
|
||||
news = ReplayNews()
|
||||
strategy = BayesianStrategy(news=news, manifold=None, db=None)
|
||||
ext = build_ext(snapshot)
|
||||
decisions: list[dict] = []
|
||||
|
||||
for row in signal_rows:
|
||||
recorded_skip = row["skip_reason"]
|
||||
decision = {
|
||||
"cycle_ts": cycle_ts,
|
||||
"market_id": row["market_id"],
|
||||
"skip_reason": None,
|
||||
"prior_prob": None,
|
||||
"estimated_prob": None,
|
||||
"raw_final_prob": None,
|
||||
"edge_gross": None,
|
||||
"edge_net": None,
|
||||
"regime_min_edge": None,
|
||||
"days_to_resolution": None,
|
||||
"confidence": None,
|
||||
"direction": None,
|
||||
"would_trade": None,
|
||||
"recorded_skip_reason": recorded_skip,
|
||||
"matched": None,
|
||||
"mismatch_field": None,
|
||||
}
|
||||
|
||||
market_row = market_rows.get(row["market_id"])
|
||||
if market_row is None:
|
||||
# Should not happen (R0 upserts markets every cycle) — record the
|
||||
# gap instead of crashing the run.
|
||||
decision["matched"] = False
|
||||
decision["mismatch_field"] = "market_missing"
|
||||
decisions.append(decision)
|
||||
continue
|
||||
|
||||
market = build_market(market_row, row)
|
||||
# A family-skipped row replays against its own occupied family; all
|
||||
# other rows replay unoccupied — their recorded skip_reason already
|
||||
# reflects whatever portfolio state existed, and evaluate() checks
|
||||
# the family gate before anything portfolio-dependent.
|
||||
families = (
|
||||
{row["family_key"]}
|
||||
if recorded_skip == "family" and row["family_key"]
|
||||
else set()
|
||||
)
|
||||
news.sentiment = row["news_sentiment"] or 0.0
|
||||
# Bypass the per-cycle GNews budget: archived sentiment already
|
||||
# encodes it (budget-skipped markets were recorded with 0.0).
|
||||
strategy._news_queries_this_cycle = 0
|
||||
|
||||
signal = await strategy.evaluate(market, ext, families, as_of=cycle_ts)
|
||||
rec = strategy.drain_cycle_records()[-1]
|
||||
|
||||
decision.update(
|
||||
skip_reason=rec["skip_reason"],
|
||||
prior_prob=rec["prior_prob"],
|
||||
estimated_prob=rec["estimated_prob"],
|
||||
raw_final_prob=rec["raw_final_prob"],
|
||||
edge_gross=rec["edge_gross"],
|
||||
edge_net=rec["edge_net"],
|
||||
regime_min_edge=rec["regime_min_edge"],
|
||||
days_to_resolution=rec["days_to_resolution"],
|
||||
confidence=rec["confidence"],
|
||||
direction=rec["direction"],
|
||||
would_trade=signal is not None,
|
||||
)
|
||||
if recorded_skip in NON_COMPARABLE_SKIPS:
|
||||
decision["matched"] = None # re-evaluated for calibration only
|
||||
else:
|
||||
mismatch = _compare(row, rec)
|
||||
decision["matched"] = mismatch is None
|
||||
decision["mismatch_field"] = mismatch
|
||||
decisions.append(decision)
|
||||
|
||||
return decisions
|
||||
|
||||
|
||||
async def run_replay(
|
||||
db: Database,
|
||||
from_ts: datetime,
|
||||
to_ts: datetime,
|
||||
note: str = "",
|
||||
limit_cycles: Optional[int] = None,
|
||||
) -> dict:
|
||||
"""Replay every archived cycle in [from_ts, to_ts) and persist the run.
|
||||
|
||||
Returns the replay_runs row (plus a mismatch_fields Counter) for reporting.
|
||||
"""
|
||||
run_id = str(uuid.uuid4())
|
||||
cycles = await db.get_replay_cycles(from_ts, to_ts)
|
||||
if limit_cycles:
|
||||
cycles = cycles[:limit_cycles]
|
||||
|
||||
decisions_total = 0
|
||||
matched = 0
|
||||
mismatched = 0
|
||||
mismatch_fields: Counter = Counter()
|
||||
skipped_cycles = 0
|
||||
|
||||
for cycle_ts in cycles:
|
||||
snapshot = await db.get_ext_snapshot(cycle_ts)
|
||||
if snapshot is None:
|
||||
skipped_cycles += 1
|
||||
log.warning("Replay: no ext_snapshot for cycle %s — skipped", cycle_ts)
|
||||
continue
|
||||
signal_rows = await db.get_cycle_signal_rows(cycle_ts)
|
||||
market_rows = await db.get_markets_by_ids(
|
||||
[r["market_id"] for r in signal_rows]
|
||||
)
|
||||
decisions = await replay_cycle(cycle_ts, snapshot, signal_rows, market_rows)
|
||||
await db.save_replay_decisions(run_id, decisions)
|
||||
|
||||
decisions_total += len(decisions)
|
||||
for d in decisions:
|
||||
if d["matched"] is True:
|
||||
matched += 1
|
||||
elif d["matched"] is False:
|
||||
mismatched += 1
|
||||
mismatch_fields[d["mismatch_field"]] += 1
|
||||
|
||||
run = {
|
||||
"run_id": run_id,
|
||||
"git_sha": _git_sha(),
|
||||
"config_hash": strategy_config_hash(),
|
||||
"config_json": json.dumps(strategy_config(), sort_keys=True),
|
||||
"from_ts": from_ts,
|
||||
"to_ts": to_ts,
|
||||
"cycles": len(cycles) - skipped_cycles,
|
||||
"decisions": decisions_total,
|
||||
"matched": matched,
|
||||
"mismatched": mismatched,
|
||||
"note": note,
|
||||
}
|
||||
await db.save_replay_run(run)
|
||||
run["mismatch_fields"] = dict(mismatch_fields)
|
||||
run["skipped_cycles"] = skipped_cycles
|
||||
return run
|
||||
|
||||
|
||||
def _parse_ts(value: str) -> datetime:
|
||||
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt
|
||||
|
||||
|
||||
async def _amain(args: argparse.Namespace) -> None:
|
||||
db = Database()
|
||||
await db.connect()
|
||||
try:
|
||||
run = await run_replay(
|
||||
db,
|
||||
from_ts=args.from_ts,
|
||||
to_ts=args.to_ts,
|
||||
note=args.note,
|
||||
limit_cycles=args.limit_cycles,
|
||||
)
|
||||
finally:
|
||||
await db.disconnect()
|
||||
|
||||
comparable = run["matched"] + run["mismatched"]
|
||||
print(f"run_id : {run['run_id']}")
|
||||
print(f"git_sha : {run['git_sha']} config_hash: {run['config_hash']}")
|
||||
print(f"window : {run['from_ts'].isoformat()} → {run['to_ts'].isoformat()}")
|
||||
print(f"cycles : {run['cycles']} (skipped: {run['skipped_cycles']})")
|
||||
print(f"decisions : {run['decisions']} ({comparable} comparable)")
|
||||
print(f"matched : {run['matched']}")
|
||||
print(f"mismatched : {run['mismatched']}")
|
||||
if run["mismatch_fields"]:
|
||||
for field, count in sorted(run["mismatch_fields"].items(), key=lambda x: -x[1]):
|
||||
print(f" {field}: {count}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m bot.replay",
|
||||
description="Replay archived trading cycles through the current strategy.",
|
||||
)
|
||||
now = datetime.now(timezone.utc)
|
||||
parser.add_argument(
|
||||
"--from", dest="from_ts", type=_parse_ts,
|
||||
default=now - timedelta(hours=24),
|
||||
help="window start, ISO-8601 (default: 24h ago)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--to", dest="to_ts", type=_parse_ts, default=now,
|
||||
help="window end, ISO-8601, exclusive (default: now)",
|
||||
)
|
||||
parser.add_argument("--note", default="", help="free-text tag for replay_runs")
|
||||
parser.add_argument(
|
||||
"--limit-cycles", type=int, default=None,
|
||||
help="replay at most N cycles (smoke runs)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
# evaluate() logs one INFO line per market — thousands per replay window.
|
||||
logging.getLogger("bot.strategy.bayesian").setLevel(logging.WARNING)
|
||||
asyncio.run(_amain(args))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+108
-4
@@ -167,15 +167,22 @@ def _regime_min_edge(category: str, days_to_resolution: int) -> float:
|
||||
return 0.10 # tech, crypto/finance, events, default
|
||||
|
||||
|
||||
def _days_to_resolution(end_date: str) -> int:
|
||||
"""Return calendar days until market resolution, or 30 if unknown."""
|
||||
def _days_to_resolution(end_date: str, as_of: Optional[datetime] = None) -> int:
|
||||
"""Return calendar days until market resolution, or 30 if unknown.
|
||||
|
||||
as_of (Replay R1): reference clock for the computation. None (production)
|
||||
means wall-clock now; a replay run passes the archived cycle_ts so
|
||||
days-to-resolution — and therefore the regime edge threshold — is computed
|
||||
against the moment the decision was originally made.
|
||||
"""
|
||||
if not end_date:
|
||||
return 30 # conservative: treat as medium-term
|
||||
try:
|
||||
dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
days = (dt - datetime.now(timezone.utc)).days
|
||||
now = as_of if as_of is not None else datetime.now(timezone.utc)
|
||||
days = (dt - now).days
|
||||
return max(0, days)
|
||||
except (ValueError, TypeError):
|
||||
return 30
|
||||
@@ -361,6 +368,9 @@ class BayesianStrategy:
|
||||
self._news_shifts: list[float] = [] # final_prob - prior, signed
|
||||
self._news_guardrail_applied: int = 0
|
||||
self._news_changed_decisions: int = 0
|
||||
# Replay R0: per-(market, cycle) decision records, drained by main.py
|
||||
# into the signals table after each evaluation loop.
|
||||
self._cycle_records: list[dict] = []
|
||||
|
||||
def reset_cycle(self) -> None:
|
||||
"""Call once at the start of each trading cycle to reset per-cycle counters."""
|
||||
@@ -375,6 +385,51 @@ class BayesianStrategy:
|
||||
self._news_shifts = []
|
||||
self._news_guardrail_applied = 0
|
||||
self._news_changed_decisions = 0
|
||||
self._cycle_records = []
|
||||
|
||||
def record_skip(self, market: Market, skip_reason: str) -> None:
|
||||
"""Record a skip decided OUTSIDE evaluate() (e.g. reentry_guard in main)."""
|
||||
self._record(market, skip_reason=skip_reason)
|
||||
|
||||
def drain_cycle_records(self) -> list[dict]:
|
||||
"""Return and clear this cycle's decision records (Replay R0)."""
|
||||
records, self._cycle_records = self._cycle_records, []
|
||||
return records
|
||||
|
||||
def _record(self, market: Market, skip_reason: Optional[str], **fields) -> None:
|
||||
"""Append one decision record. Early skips leave most fields None —
|
||||
the archive still shows the market existed and why it went no further."""
|
||||
rec = {
|
||||
"market_id": market.id,
|
||||
"polymarket_price": market.yes_price,
|
||||
"category": market.category,
|
||||
"volume_24h": market.volume_24h,
|
||||
"skip_reason": skip_reason,
|
||||
"family_key": None,
|
||||
"prior_prob": None,
|
||||
"estimated_prob": None,
|
||||
"raw_final_prob": None,
|
||||
"edge_gross": None,
|
||||
"edge_net": None,
|
||||
"regime_min_edge": None,
|
||||
"days_to_resolution": None,
|
||||
"confidence": None,
|
||||
"direction": None,
|
||||
"passed_gross": None,
|
||||
"passed_net": None,
|
||||
"news_sentiment": None,
|
||||
"news_budget_skipped": None,
|
||||
"guardrail_applied": None,
|
||||
"guardrail_changed_decision": None,
|
||||
"feat_fg_lo": None,
|
||||
"feat_mom_lo": None,
|
||||
"feat_news_lo": None,
|
||||
"feat_mfld_lo": None,
|
||||
"feat_btc_dom_lo": None,
|
||||
"acted_on": False,
|
||||
}
|
||||
rec.update(fields)
|
||||
self._cycle_records.append(rec)
|
||||
|
||||
def get_cycle_stats(self) -> dict:
|
||||
"""Return per-cycle counters for the [CYCLE SUMMARY] log block."""
|
||||
@@ -409,10 +464,17 @@ class BayesianStrategy:
|
||||
market: Market,
|
||||
ext: ExternalSignals,
|
||||
occupied_families: set[str],
|
||||
as_of: Optional[datetime] = None,
|
||||
) -> Optional[TradingSignal]:
|
||||
"""
|
||||
Evaluate a market and return a TradingSignal if actionable.
|
||||
|
||||
as_of (Replay R1): clock injection — None in production (wall-clock
|
||||
now); a replay passes the archived cycle_ts so the regime threshold
|
||||
matches the original decision moment. Only days-to-resolution
|
||||
depends on the clock; everything else is a pure function of
|
||||
(market, ext, occupied_families) and the news/manifold clients.
|
||||
|
||||
Returns None with a structured log line in all skip cases.
|
||||
Skip reasons (Phase 5 observability):
|
||||
SKIP_UNSUPPORTED — category not supported
|
||||
@@ -467,6 +529,7 @@ class BayesianStrategy:
|
||||
"SKIP_UNSUPPORTED %-50s | cat=%r",
|
||||
market.question[:50], category,
|
||||
)
|
||||
self._record(market, skip_reason="unsupported")
|
||||
return None
|
||||
|
||||
if not ext.valid:
|
||||
@@ -474,6 +537,7 @@ class BayesianStrategy:
|
||||
"SKIP_NO_SIGNALS %-50s | reason=external data unavailable",
|
||||
market.question[:50],
|
||||
)
|
||||
self._record(market, skip_reason="no_signals")
|
||||
return None
|
||||
|
||||
# ── Phase 1: prior + prior-extreme filter ────────────────────────────
|
||||
@@ -485,6 +549,7 @@ class BayesianStrategy:
|
||||
"SKIP_PRIOR_EXTREME %-50s | cat=%-12s | prior=%.3f | reason=prior<0.08",
|
||||
market.question[:50], category, market.yes_price,
|
||||
)
|
||||
self._record(market, skip_reason="prior_extreme", prior_prob=prior)
|
||||
return None
|
||||
if market.yes_price > 0.92:
|
||||
self._skip_prior_extreme += 1
|
||||
@@ -492,6 +557,7 @@ class BayesianStrategy:
|
||||
"SKIP_PRIOR_EXTREME %-50s | cat=%-12s | prior=%.3f | reason=prior>0.92",
|
||||
market.question[:50], category, market.yes_price,
|
||||
)
|
||||
self._record(market, skip_reason="prior_extreme", prior_prob=prior)
|
||||
return None
|
||||
|
||||
# ── Phase 2: family deduplication ────────────────────────────────────
|
||||
@@ -502,10 +568,11 @@ class BayesianStrategy:
|
||||
"SKIP_FAMILY %-50s | cat=%-12s | family=%s",
|
||||
market.question[:50], category, family,
|
||||
)
|
||||
self._record(market, skip_reason="family", prior_prob=prior, family_key=family)
|
||||
return None
|
||||
|
||||
# ── Phase 4: regime min-edge ─────────────────────────────────────────
|
||||
days = _days_to_resolution(market.end_date)
|
||||
days = _days_to_resolution(market.end_date, as_of)
|
||||
regime_min = _regime_min_edge(category, days)
|
||||
|
||||
# ── Bayesian probability estimation ──────────────────────────────────
|
||||
@@ -576,6 +643,10 @@ class BayesianStrategy:
|
||||
# highest-value markets reach this block first.
|
||||
news_log_adj = 0.0
|
||||
news_sentiment = 0.0
|
||||
# Replay R0: True when GNews was never consulted for this market this
|
||||
# cycle (budget exhausted) — a replay must not read feat_news_lo=0.0 as
|
||||
# "there was no news".
|
||||
news_budget_skipped = False
|
||||
# self._news.enabled gates the whole block: with no GNews API key the
|
||||
# client is a no-op, so we must not consume (or report) query budget for
|
||||
# it — see NewsClient.enabled.
|
||||
@@ -588,6 +659,7 @@ class BayesianStrategy:
|
||||
news_log_adj = sentiment * NEWS_LOGODDS_WEIGHT
|
||||
sources.append(f"GNews: {sentiment:+.2f}")
|
||||
else:
|
||||
news_budget_skipped = True
|
||||
log.info(
|
||||
"SKIP_GNEWS_PRIORITY %-50s | reason=cycle budget %d reached",
|
||||
market.question[:50], MAX_NEWS_QUERIES_PER_CYCLE,
|
||||
@@ -825,6 +897,38 @@ class BayesianStrategy:
|
||||
MAX_NEWS_ONLY_PROB_SHIFT,
|
||||
)
|
||||
|
||||
# Replay R0: full decision record — same fields for skip and trade paths.
|
||||
# skip_reason granularity: "edge_net" when the edge gate failed,
|
||||
# "confidence" when only the confidence gate blocked the trade.
|
||||
self._record(
|
||||
market,
|
||||
skip_reason=(
|
||||
None if can_trade
|
||||
else ("edge_net" if not passed_net else "confidence")
|
||||
),
|
||||
family_key=family,
|
||||
prior_prob=prior,
|
||||
estimated_prob=estimated_prob,
|
||||
raw_final_prob=raw_final_prob,
|
||||
edge_gross=edge_gross,
|
||||
edge_net=edge_net,
|
||||
regime_min_edge=regime_min,
|
||||
days_to_resolution=days,
|
||||
confidence=confidence,
|
||||
direction=direction,
|
||||
passed_gross=passed_gross,
|
||||
passed_net=passed_net,
|
||||
news_sentiment=news_sentiment,
|
||||
news_budget_skipped=news_budget_skipped,
|
||||
guardrail_applied=news_guardrail_applied,
|
||||
guardrail_changed_decision=guardrail_changed_trade_decision,
|
||||
feat_fg_lo=feat_fg_lo,
|
||||
feat_mom_lo=feat_mom_lo,
|
||||
feat_news_lo=feat_news_lo,
|
||||
feat_mfld_lo=feat_mfld_lo,
|
||||
feat_btc_dom_lo=feat_btc_dom_lo,
|
||||
)
|
||||
|
||||
if not can_trade:
|
||||
# Increment the appropriate edge-net counter
|
||||
if edge_net <= 0:
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 119 KiB |
@@ -0,0 +1,165 @@
|
||||
# Informe final de paper trading — polymarket-bot
|
||||
|
||||
**Snapshot: 2026-07-06** · Generado desde la base de datos de producción con el bot
|
||||
aún operativo, como parte de la Fase 1 del plan de decomisión (bloqueo ISP de
|
||||
Polymarket en España, mayo 2026).
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Advertencia de tamaño muestral (léase primero)
|
||||
|
||||
**12 trades ejecutados, de los cuales solo 2 llegaron a resolución, no constituyen
|
||||
evidencia estadística de nada.** Ni el P&L positivo demuestra que la estrategia
|
||||
funcione, ni su ausencia lo habría refutado. El propio bot lo sabía: sus criterios
|
||||
de promoción a dinero real exigían ≥10 mercados resueltos y ≥30 días de
|
||||
observación (`promotion_ready: false`, win rate y Sharpe deliberadamente
|
||||
reportados como `n/a (insufficient_sample)` en toda la instrumentación).
|
||||
|
||||
El valor demostrable de este proyecto no está en el P&L: está en la
|
||||
infraestructura de evaluación, el pipeline de datos y la disciplina de gates que
|
||||
impidió que el bot tradease sin ventaja medible. Este informe presenta el P&L
|
||||
como lo que es — **una anécdota** — y dedica el grueso al sistema.
|
||||
|
||||
---
|
||||
|
||||
## 1. Resultados de trading (anecdóticos)
|
||||
|
||||
Bankroll simulado: **$10.000** (USDC virtual) · Capital desplegado: **$2.447** ·
|
||||
Periodo de actividad: **14–22 de abril de 2026** (los 12 trades se abrieron en 9
|
||||
días; después, el endurecimiento de los gates de edge y confianza dejó el bot en
|
||||
0 trades — funcionando, evaluando ~1.350 ciclos/día, sin encontrar ventaja neta
|
||||
que superara sus propios umbrales).
|
||||
|
||||
### P&L realizado: **+$247,78** (2 mercados resueltos)
|
||||
|
||||
| Mercado | Dirección | Entrada | Resultado | P&L neto |
|
||||
|---|---|---|---|---|
|
||||
| Will Ken Paxton win the 2026 Texas Republican Primary? | BUY_YES | 0,618 | ✅ YES (2026-06-11) | **+$299,06** |
|
||||
| Will NVIDIA be the largest company in the world…? | BUY_NO | 0,156 | ❌ YES (2026-06-30) | **−$51,28** |
|
||||
|
||||
P&L neto de comisiones (2% simulado por lado). 1 acierto / 1 fallo: n=2.
|
||||
|
||||
### P&L no realizado (mark-to-market): **+$809,59** (5 posiciones abiertas)
|
||||
|
||||
Marcado a los precios reales de Polymarket (Gamma API) el 2026-07-06. Estas
|
||||
posiciones **no están cerradas**: el número de abajo cambiaría cada día que los
|
||||
mercados sigan moviéndose, y varios no resuelven hasta noviembre de 2026.
|
||||
|
||||
| Mercado | Dirección | Entrada | Precio actual (lado) | Coste neto | MTM P&L |
|
||||
|---|---|---|---|---|---|
|
||||
| Karen Bass gana la alcaldía de LA 2026 | BUY_YES | 0,268 | 0,595 | $510,00 | **+$600,07** |
|
||||
| OpenAI IPO antes del 31-dic-2026 | BUY_NO | 0,618 | 0,785 | $510,00 | **+$125,11** |
|
||||
| Demócratas ganan el Senado de Texas 2026 | BUY_YES | 0,418 | 0,435 | $509,49 | **+$10,32** |
|
||||
| Demócratas ganan gobernación de Ohio 2026 | BUY_NO | 0,458 | 0,520 | $407,74 | **+$46,12** |
|
||||
| Republicanos ganan gobernación de Nebraska 2026 | BUY_NO | 0,158 | 0,170 | $510,00 | **+$27,97** |
|
||||
|
||||
> Nota metodológica: el tracker interno reportaba `unrealized_pnl_est = +$522,73`,
|
||||
> pero esa cifra es la **expectativa del propio modelo a fecha de entrada**
|
||||
> (edge_net × coste), no un marcado a mercado. La tabla de arriba usa precios
|
||||
> reales de mercado del día del snapshot, que es el criterio honesto. Ambas
|
||||
> cifras coinciden en el signo; ninguna de las dos es un resultado realizado.
|
||||
|
||||
### Los otros 5 cierres (sin P&L): la historia real de abril
|
||||
|
||||
De los 7 trades cerrados, solo 2 cerraron por resolución. Los otros 5 los cerró
|
||||
el propio bot al detectar defectos en su lógica — y son la parte más
|
||||
instructiva del histórico:
|
||||
|
||||
- **3 por conflicto de familia**: dos posiciones sobre mercados hermanos del
|
||||
mismo evento (p. ej. YES-demócratas y YES-republicanos en la misma
|
||||
gobernación) — el agrupador de familias los detectó y cerró el lado débil.
|
||||
- **2 por el bug de inversión de Manifold**: la señal cruzada de Manifold se
|
||||
aplicaba invertida en mercados espejo. Detectado por instrumentación, las
|
||||
posiciones afectadas se cerraron y el matching se movió a modo
|
||||
observacional-only con auditoría completa (165.538 filas en
|
||||
`manifold_match_audit`).
|
||||
- **1 excluido de métricas** (`invalid_manifold_match_legacy`, P&L $0,00).
|
||||
|
||||
Después de esa semana el bot no volvió a operar: los gates endurecidos
|
||||
(edge neto > mínimo por régimen, confianza ≥ 0,55, guardrail de noticias
|
||||
prior±0,25) filtraron el 100% de las ~223.000 evaluaciones posteriores.
|
||||
**Un sistema de paper trading cuyo resultado principal es "no tradees sin
|
||||
ventaja" es un resultado**, y está instrumentado hasta el último skip.
|
||||
|
||||
---
|
||||
|
||||
## 2. El sistema (el activo real)
|
||||
|
||||
### Pipeline de señales
|
||||
|
||||
| Métrica | Valor (snapshot 2026-07-06) |
|
||||
|---|---|
|
||||
| Ciclos de evaluación archivados | **5.356** (cadencia ~64 s) |
|
||||
| Evaluaciones mercado-ciclo archivadas | **222.738** |
|
||||
| Mercados distintos evaluados | 87 (events 114,9k · politics 96,0k · crypto/finance 7,7k · tech 4,2k evaluaciones) |
|
||||
| Archivo `signals` operativo desde | 2026-07-02 (replay R0) — crece ~55k filas/día |
|
||||
| Histórico de métricas diarias | **81 días** (2026-04-14 → 2026-07-06, `metrics_daily`) |
|
||||
| Snapshots de contexto externo | 5.356 (BTC, dominancia, Fear&Greed por ciclo) |
|
||||
|
||||
Cada evaluación archiva el prior, la probabilidad estimada, el edge bruto/neto,
|
||||
la descomposición por feature (Fear&Greed, momentum, noticias, Manifold,
|
||||
dominancia BTC en log-odds), el gate que la bloqueó y el contexto de mercado —
|
||||
suficiente para reproducir la decisión completa offline.
|
||||
|
||||
### Motor de replay (R0–R2, julio 2026)
|
||||
|
||||
- Inyección de reloj + replay determinista de ciclos archivados:
|
||||
**22.697/22.697 decisiones idénticas** al reproducir el histórico.
|
||||
- Joiner diario de resoluciones (CronJob 00:30 UTC) contra la Gamma API:
|
||||
cobertura de outcomes 9/87 mercados al snapshot, creciendo sola.
|
||||
- Diseñado para calibrar sobre **todas** las evaluaciones (no solo trades),
|
||||
esquivando el problema del n=12.
|
||||
|
||||
### Estrategia (congelada desde 2026-07-03)
|
||||
|
||||
Prior desde el precio de Polymarket → ajuste bayesiano en log-odds con señales
|
||||
externas (GNews con presupuesto de 5 consultas/día y guardrail prior±0,25;
|
||||
Fear&Greed; momentum BTC; Manifold observacional) → gates en cascada: prior
|
||||
extremo, edge bruto por régimen de volatilidad, edge neto tras comisión 2% y
|
||||
spread, confianza mínima 0,55, conflictos de familia, guard de reentrada.
|
||||
Sizing por Kelly fraccionado con techo por posición.
|
||||
|
||||
### Infraestructura
|
||||
|
||||
3 imágenes (bot / API FastAPI / dashboard) construidas por Gitea Actions solo
|
||||
cuando cambian sus fuentes, desplegadas por ArgoCD (prune + selfHeal) en k3s;
|
||||
PostgreSQL 16 en StatefulSet; secrets vía Infisical operator; smoke test
|
||||
PostSync contra la API; backups nocturnos con sync offsite (rclone → MEGA);
|
||||
observabilidad en Grafana + uptime-kuma + notificaciones Telegram (deploys,
|
||||
checkpoints del bot, fallos de sync). Retención de métricas y join de outcomes
|
||||
como CronJobs. **88 días de uptime del StatefulSet al snapshot.**
|
||||
|
||||
---
|
||||
|
||||
## 3. Estado de la base de datos al snapshot
|
||||
|
||||
| Tabla | Filas | Contenido |
|
||||
|---|---|---|
|
||||
| `signals` | 222.549* | Archivo de evaluaciones por ciclo (desde 2026-07-02) |
|
||||
| `manifold_match_audit` | 165.538 | Auditoría completa del matching Polymarket↔Manifold |
|
||||
| `replay_decisions` | 22.697 | Decisiones reproducidas por el motor de replay |
|
||||
| `ext_snapshots` | 5.353* | Contexto externo por ciclo |
|
||||
| `metrics_daily` | 516 | Cierres diarios (81 días) + snapshots intradía de hoy |
|
||||
| `markets` | 87 | Universo evaluado |
|
||||
| `trades` | 12 | Ledger completo de paper trades |
|
||||
| `market_outcomes` | 9 | Resoluciones UMA-final joineadas |
|
||||
| `replay_runs` / `manifold_eval_cooldown` / `checkpoint_alerts` | 1 / 76 / 3 | Metadatos |
|
||||
|
||||
\* La BD sigue viva; las tablas por ciclo crecen ~55k y ~1,3k filas/día
|
||||
respectivamente. Backup verificado del 2026-07-06 en
|
||||
`/data/backups/backups/polymarket-decommission/` (pg_dump -Fc + CSV.gz +
|
||||
checksums; restore de prueba con recuentos idénticos 11/11 tablas). El dump
|
||||
final se hará en la Fase 3, justo antes del apagado.
|
||||
|
||||
---
|
||||
|
||||
## 4. Conclusión
|
||||
|
||||
El proyecto se archiva por una causa externa (bloqueo regulatorio del origen de
|
||||
datos), no por fracaso técnico. Lo que queda: un motor de evaluación bayesiano
|
||||
determinista y auditable, un pipeline de datos que archivó cada decisión con su
|
||||
contexto completo, y la evidencia — anecdótica en P&L, sólida en ingeniería —
|
||||
de que el sistema prefería no operar antes que operar sin ventaja. Todos los
|
||||
módulos centrales (edge, familias, replay, observabilidad) son agnósticos de la
|
||||
fuente y quedan listos para un eventual pivote a Manifold
|
||||
(`docs/pivot-manifold.md`, Fase 5).
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 58 KiB |
@@ -0,0 +1,174 @@
|
||||
"""Replay R2 tests — outcome fetching and calibration scoring."""
|
||||
import asyncio
|
||||
import math
|
||||
|
||||
import pytest
|
||||
|
||||
from bot.data.polymarket import MarketResolution
|
||||
from bot.outcomes import (
|
||||
LOGLOSS_EPS,
|
||||
compute_calibration,
|
||||
fetch_outcomes,
|
||||
print_report,
|
||||
)
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
class FakePoly:
|
||||
"""get_market_resolution stand-in driven by a dict of canned responses."""
|
||||
|
||||
def __init__(self, responses: dict):
|
||||
self.responses = responses
|
||||
self.calls: list[str] = []
|
||||
|
||||
async def get_market_resolution(self, market_id: str):
|
||||
self.calls.append(market_id)
|
||||
return self.responses.get(market_id)
|
||||
|
||||
|
||||
RESOLVED_AT = datetime(2026, 7, 1, 12, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def _row(market_id="m1", category="politics", est=0.6, prior=0.5, outcome=1.0):
|
||||
return {
|
||||
"market_id": market_id,
|
||||
"category": category,
|
||||
"estimated_prob": est,
|
||||
"prior_prob": prior,
|
||||
"outcome": outcome,
|
||||
}
|
||||
|
||||
|
||||
# ── fetch_outcomes ───────────────────────────────────────────────────────────
|
||||
|
||||
def test_fetch_keeps_only_definitive_resolutions():
|
||||
poly = FakePoly({
|
||||
"yes": MarketResolution(resolved=True, resolution=1.0,
|
||||
resolved_at=RESOLVED_AT),
|
||||
"no": MarketResolution(resolved=True, resolution=0.0,
|
||||
resolved_at=None),
|
||||
"open": MarketResolution(resolved=False),
|
||||
"disputed": MarketResolution(resolved=False),
|
||||
"apierror": None, # get_market_resolution returns None on HTTP errors
|
||||
})
|
||||
out = asyncio.run(
|
||||
fetch_outcomes(poly, ["yes", "no", "open", "disputed", "apierror"])
|
||||
)
|
||||
assert poly.calls == ["yes", "no", "open", "disputed", "apierror"]
|
||||
assert out == [
|
||||
{"market_id": "yes", "outcome": 1.0, "resolved_at": RESOLVED_AT},
|
||||
{"market_id": "no", "outcome": 0.0, "resolved_at": None},
|
||||
]
|
||||
|
||||
|
||||
def test_fetch_empty_list_is_noop():
|
||||
poly = FakePoly({})
|
||||
assert asyncio.run(fetch_outcomes(poly, [])) == []
|
||||
assert poly.calls == []
|
||||
|
||||
|
||||
# ── compute_calibration ──────────────────────────────────────────────────────
|
||||
|
||||
def test_no_rows_returns_none():
|
||||
assert compute_calibration([]) is None
|
||||
|
||||
|
||||
def test_single_row_known_values():
|
||||
m = compute_calibration([_row(est=0.8, prior=0.6, outcome=1.0)])
|
||||
assert m["n_evaluations"] == 1
|
||||
assert m["n_markets"] == 1
|
||||
assert m["brier_model"] == pytest.approx((0.8 - 1.0) ** 2)
|
||||
assert m["brier_prior"] == pytest.approx((0.6 - 1.0) ** 2)
|
||||
assert m["logloss_model"] == pytest.approx(-math.log(0.8))
|
||||
assert m["logloss_prior"] == pytest.approx(-math.log(0.6))
|
||||
# one market: macro == micro
|
||||
assert m["brier_model_macro"] == pytest.approx(m["brier_model"])
|
||||
assert m["brier_prior_macro"] == pytest.approx(m["brier_prior"])
|
||||
|
||||
|
||||
def test_logloss_no_outcome_branch():
|
||||
m = compute_calibration([_row(est=0.2, prior=0.7, outcome=0.0)])
|
||||
assert m["logloss_model"] == pytest.approx(-math.log(0.8))
|
||||
assert m["logloss_prior"] == pytest.approx(-math.log(0.3))
|
||||
|
||||
|
||||
def test_logloss_clipping_keeps_hard_miss_finite():
|
||||
# A hard 1.0 estimate on a NO outcome must not produce inf.
|
||||
m = compute_calibration([_row(est=1.0, prior=0.5, outcome=0.0)])
|
||||
assert math.isfinite(m["logloss_model"])
|
||||
assert m["logloss_model"] == pytest.approx(-math.log(LOGLOSS_EPS))
|
||||
|
||||
|
||||
def test_micro_weights_evaluations_macro_weights_markets():
|
||||
# Market a: 3 evaluations, model error 0.1; market b: 1 evaluation, error 0.5.
|
||||
rows = [
|
||||
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
|
||||
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
|
||||
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
|
||||
_row(market_id="b", est=0.5, prior=0.6, outcome=1.0),
|
||||
]
|
||||
m = compute_calibration(rows)
|
||||
assert m["n_evaluations"] == 4
|
||||
assert m["n_markets"] == 2
|
||||
# micro: (3*0.01 + 0.25) / 4 ; macro: (0.01 + 0.25) / 2
|
||||
assert m["brier_model"] == pytest.approx((3 * 0.01 + 0.25) / 4)
|
||||
assert m["brier_model_macro"] == pytest.approx((0.01 + 0.25) / 2)
|
||||
assert m["brier_prior"] == pytest.approx((3 * 0.04 + 0.16) / 4)
|
||||
assert m["brier_prior_macro"] == pytest.approx((0.04 + 0.16) / 2)
|
||||
|
||||
|
||||
def test_model_beating_market_gives_negative_delta():
|
||||
# est closer to the outcome than the price on every row
|
||||
rows = [
|
||||
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
|
||||
_row(market_id="b", est=0.3, prior=0.45, outcome=0.0),
|
||||
]
|
||||
m = compute_calibration(rows)
|
||||
assert m["brier_model"] < m["brier_prior"]
|
||||
assert m["logloss_model"] < m["logloss_prior"]
|
||||
|
||||
|
||||
def test_per_category_grouping_and_unknown():
|
||||
rows = [
|
||||
_row(market_id="a", category="politics", est=0.8, prior=0.6, outcome=1.0),
|
||||
_row(market_id="b", category="politics", est=0.7, prior=0.6, outcome=1.0),
|
||||
_row(market_id="c", category=None, est=0.4, prior=0.5, outcome=0.0),
|
||||
]
|
||||
m = compute_calibration(rows)
|
||||
assert set(m["per_category"]) == {"politics", "unknown"}
|
||||
pol = m["per_category"]["politics"]
|
||||
assert pol["n"] == 2 and pol["markets"] == 2
|
||||
assert pol["brier_model"] == pytest.approx((0.04 + 0.09) / 2)
|
||||
unk = m["per_category"]["unknown"]
|
||||
assert unk["n"] == 1 and unk["markets"] == 1
|
||||
assert unk["brier_model"] == pytest.approx(0.16)
|
||||
|
||||
|
||||
def test_repeated_market_counts_once_in_markets():
|
||||
rows = [
|
||||
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
|
||||
_row(market_id="a", est=0.7, prior=0.55, outcome=1.0),
|
||||
]
|
||||
m = compute_calibration(rows)
|
||||
assert m["n_markets"] == 1
|
||||
assert m["per_category"]["politics"]["markets"] == 1
|
||||
|
||||
|
||||
# ── print_report ─────────────────────────────────────────────────────────────
|
||||
|
||||
def test_report_handles_no_metrics(capsys):
|
||||
print_report(None, "R0 archive")
|
||||
assert "no scorable rows yet" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_report_prints_all_metric_lines(capsys):
|
||||
m = compute_calibration([
|
||||
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
|
||||
_row(market_id="b", category=None, est=0.4, prior=0.5, outcome=0.0),
|
||||
])
|
||||
print_report(m, "R0 archive")
|
||||
out = capsys.readouterr().out
|
||||
assert "2 evaluations, 2 markets" in out
|
||||
for label in ("Brier micro", "Brier macro", "logloss micro",
|
||||
"politics", "unknown"):
|
||||
assert label in out
|
||||
@@ -0,0 +1,367 @@
|
||||
"""
|
||||
Tests for the Replay R1 replay core (bot/replay.py) and the as_of clock
|
||||
injection in BayesianStrategy.evaluate().
|
||||
|
||||
The central contract is round-trip fidelity: a decision recorded by R0 and
|
||||
replayed through replay_cycle() with the same strategy constants must match
|
||||
field-for-field (matched=True, mismatch_field=None). Each round-trip test
|
||||
produces the "archive" by running the real evaluate() with FakeNews, then
|
||||
replays the drained record as if it had been read back from the signals table.
|
||||
"""
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
import bot.strategy.bayesian as bayesian
|
||||
from bot.data.polymarket import Market, market_family_key
|
||||
from bot.strategy.bayesian import BayesianStrategy, _days_to_resolution
|
||||
from bot.replay import (
|
||||
ReplayNews,
|
||||
build_ext,
|
||||
build_market,
|
||||
replay_cycle,
|
||||
strategy_config_hash,
|
||||
)
|
||||
|
||||
from tests.test_news_guardrail import FakeNews, _sentiment_for
|
||||
|
||||
|
||||
def _end_date(days_ahead: int = 20) -> str:
|
||||
dt = datetime.now(timezone.utc) + timedelta(days=days_ahead)
|
||||
return dt.strftime("%Y-%m-%dT00:00:00Z")
|
||||
|
||||
|
||||
def _make_market(
|
||||
yes_price: float,
|
||||
question: str = "Will John Smith win the election?",
|
||||
category: str = "politics",
|
||||
market_id: str = "mkt-replay-1",
|
||||
end_date: str = None,
|
||||
) -> Market:
|
||||
return Market(
|
||||
id=market_id,
|
||||
condition_id="cond-replay-1",
|
||||
question=question,
|
||||
yes_token_id="yes-tok",
|
||||
no_token_id="no-tok",
|
||||
yes_price=yes_price,
|
||||
no_price=1.0 - yes_price,
|
||||
volume_24h=50_000.0,
|
||||
end_date=end_date if end_date is not None else _end_date(),
|
||||
active=True,
|
||||
category=category,
|
||||
)
|
||||
|
||||
|
||||
def _snapshot(valid: bool = True) -> dict:
|
||||
"""An ext_snapshots row as read back from the DB."""
|
||||
return {
|
||||
"btc_price": 100_000.0,
|
||||
"btc_change_24h": 0.0,
|
||||
"eth_price": 4_000.0,
|
||||
"eth_change_24h": 0.0,
|
||||
"btc_dominance": 50.0,
|
||||
"fear_greed_index": 50,
|
||||
"fear_greed_label": "neutral",
|
||||
"total_market_cap_change": 0.0,
|
||||
"valid": valid,
|
||||
}
|
||||
|
||||
|
||||
def _market_row(market: Market) -> dict:
|
||||
"""A markets-table row for the given Market."""
|
||||
return {
|
||||
"id": market.id,
|
||||
"condition_id": market.condition_id,
|
||||
"question": market.question,
|
||||
"category": market.category,
|
||||
"end_date": market.end_date,
|
||||
}
|
||||
|
||||
|
||||
def _record_with_live_evaluate(
|
||||
market: Market,
|
||||
news=None,
|
||||
families: set = frozenset(),
|
||||
) -> dict:
|
||||
"""Run the real evaluate() and return the R0 record it produced —
|
||||
the same dict save_signal_records() would have archived."""
|
||||
strategy = BayesianStrategy(news=news, manifold=None, db=None)
|
||||
asyncio.run(strategy.evaluate(market, build_ext(_snapshot()), set(families)))
|
||||
return strategy.drain_cycle_records()[0]
|
||||
|
||||
|
||||
def _replay_one(record: dict, market: Market, snapshot: dict = None) -> dict:
|
||||
cycle_ts = datetime.now(timezone.utc)
|
||||
decisions = asyncio.run(replay_cycle(
|
||||
cycle_ts,
|
||||
snapshot or _snapshot(),
|
||||
[record],
|
||||
{market.id: _market_row(market)},
|
||||
))
|
||||
assert len(decisions) == 1
|
||||
return decisions[0]
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Clock injection
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_days_to_resolution_uses_injected_clock():
|
||||
end = "2026-08-01T00:00:00Z"
|
||||
as_of = datetime(2026, 7, 2, 12, 0, tzinfo=timezone.utc)
|
||||
assert _days_to_resolution(end, as_of) == 29
|
||||
assert _days_to_resolution(end, as_of - timedelta(days=60)) == 89
|
||||
|
||||
|
||||
def test_default_clock_is_wall_clock():
|
||||
end = _end_date(days_ahead=40)
|
||||
assert _days_to_resolution(end) == _days_to_resolution(
|
||||
end, datetime.now(timezone.utc)
|
||||
)
|
||||
|
||||
|
||||
def test_as_of_changes_regime_threshold():
|
||||
"""Same politics market: <30 d out → regime 0.08; replayed from 60 d
|
||||
earlier → regime 0.12. The clock, not the wall time, must decide."""
|
||||
market = _make_market(0.470)
|
||||
sentiment = _sentiment_for(0.470, 0.601)
|
||||
|
||||
def _regime(as_of):
|
||||
strategy = BayesianStrategy(news=FakeNews(sentiment), manifold=None, db=None)
|
||||
asyncio.run(strategy.evaluate(
|
||||
market, build_ext(_snapshot()), set(), as_of=as_of,
|
||||
))
|
||||
return strategy.drain_cycle_records()[0]["regime_min_edge"]
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
assert _regime(now) == pytest.approx(0.08)
|
||||
assert _regime(now - timedelta(days=60)) == pytest.approx(0.12)
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Round-trip fidelity: record with live evaluate(), replay, expect match
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_roundtrip_confidence_skip():
|
||||
"""Georgia signature: edge passes, confidence blocks — full-field match."""
|
||||
sentiment = _sentiment_for(0.470, 0.601)
|
||||
market = _make_market(0.470)
|
||||
record = _record_with_live_evaluate(market, news=FakeNews(sentiment))
|
||||
assert record["skip_reason"] == "confidence"
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["mismatch_field"] is None
|
||||
assert decision["skip_reason"] == "confidence"
|
||||
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
|
||||
assert decision["edge_net"] == pytest.approx(record["edge_net"])
|
||||
assert decision["confidence"] == pytest.approx(record["confidence"])
|
||||
assert decision["direction"] == record["direction"]
|
||||
assert decision["would_trade"] is False
|
||||
|
||||
|
||||
def test_roundtrip_edge_net_skip():
|
||||
market = _make_market(0.50)
|
||||
record = _record_with_live_evaluate(market)
|
||||
assert record["skip_reason"] == "edge_net"
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["would_trade"] is False
|
||||
|
||||
|
||||
def test_roundtrip_guardrail_clamp():
|
||||
"""Clamped posterior must reproduce exactly (raw != final in archive)."""
|
||||
market = _make_market(0.845)
|
||||
record = _record_with_live_evaluate(
|
||||
market, news=FakeNews(_sentiment_for(0.845, 0.431))
|
||||
)
|
||||
assert record["guardrail_applied"] is True
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["raw_final_prob"] == pytest.approx(record["raw_final_prob"])
|
||||
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
|
||||
|
||||
|
||||
def test_roundtrip_prior_extreme():
|
||||
market = _make_market(0.03)
|
||||
record = _record_with_live_evaluate(market)
|
||||
assert record["skip_reason"] == "prior_extreme"
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["skip_reason"] == "prior_extreme"
|
||||
|
||||
|
||||
def test_roundtrip_family_skip():
|
||||
"""Family-skipped rows replay with their own family injected as occupied."""
|
||||
market = _make_market(0.50)
|
||||
record = _record_with_live_evaluate(
|
||||
market, families={market_family_key(market)}
|
||||
)
|
||||
assert record["skip_reason"] == "family"
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["skip_reason"] == "family"
|
||||
|
||||
|
||||
def test_roundtrip_unsupported():
|
||||
market = _make_market(0.50, question="Will it rain tomorrow?", category="")
|
||||
record = _record_with_live_evaluate(market)
|
||||
assert record["skip_reason"] == "unsupported"
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
|
||||
|
||||
def test_roundtrip_no_signals():
|
||||
"""ext.valid=False archived → replay rebuilds the invalid snapshot."""
|
||||
market = _make_market(0.50)
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
asyncio.run(strategy.evaluate(market, build_ext(_snapshot(valid=False)), set()))
|
||||
record = strategy.drain_cycle_records()[0]
|
||||
assert record["skip_reason"] == "no_signals"
|
||||
|
||||
decision = _replay_one(record, market, snapshot=_snapshot(valid=False))
|
||||
assert decision["matched"] is True
|
||||
|
||||
|
||||
def test_roundtrip_trade_path(monkeypatch):
|
||||
"""skip_reason=None (tradeable) round-trips with would_trade=True.
|
||||
Politics can't clear MIN_CONFIDENCE=0.55 (the known ceiling), so the
|
||||
gate is lowered for this test only — both record and replay see the
|
||||
same constant, which is exactly the config_hash contract."""
|
||||
monkeypatch.setattr(bayesian, "MIN_CONFIDENCE", 0.45)
|
||||
sentiment = _sentiment_for(0.470, 0.601)
|
||||
market = _make_market(0.470)
|
||||
record = _record_with_live_evaluate(market, news=FakeNews(sentiment))
|
||||
assert record["skip_reason"] is None
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["skip_reason"] is None
|
||||
assert decision["would_trade"] is True
|
||||
assert decision["direction"] == "BUY_YES"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Replay-specific semantics
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_budget_skipped_row_replays_without_news():
|
||||
"""A budget-skipped archive row (sentiment 0.0) must replay to the same
|
||||
no-news decision — and never consume a replay-side budget."""
|
||||
market = _make_market(0.50)
|
||||
strategy = BayesianStrategy(news=FakeNews(0.9), manifold=None, db=None)
|
||||
strategy._news_queries_this_cycle = bayesian.MAX_NEWS_QUERIES_PER_CYCLE
|
||||
asyncio.run(strategy.evaluate(market, build_ext(_snapshot()), set()))
|
||||
record = strategy.drain_cycle_records()[0]
|
||||
assert record["news_budget_skipped"] is True
|
||||
assert record["news_sentiment"] == 0.0
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is True
|
||||
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
|
||||
|
||||
|
||||
def test_reentry_guard_row_is_recalibrated_not_compared():
|
||||
"""record_skip() rows carry no decision fields; the replay re-evaluates
|
||||
them (calibration data) but marks them non-comparable."""
|
||||
market = _make_market(0.50)
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
strategy.record_skip(market, "reentry_guard")
|
||||
record = strategy.drain_cycle_records()[0]
|
||||
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is None
|
||||
assert decision["recorded_skip_reason"] == "reentry_guard"
|
||||
# Re-evaluated on its merits: a full decision despite the recorded skip
|
||||
assert decision["estimated_prob"] is not None
|
||||
assert decision["skip_reason"] == "edge_net"
|
||||
|
||||
|
||||
def test_missing_market_row_flagged_not_crashed():
|
||||
market = _make_market(0.50)
|
||||
record = _record_with_live_evaluate(market)
|
||||
|
||||
decisions = asyncio.run(replay_cycle(
|
||||
datetime.now(timezone.utc), _snapshot(), [record], {},
|
||||
))
|
||||
assert decisions[0]["matched"] is False
|
||||
assert decisions[0]["mismatch_field"] == "market_missing"
|
||||
|
||||
|
||||
def test_mismatch_detected_when_config_differs(monkeypatch):
|
||||
"""Counterfactual sanity: replaying under a different guardrail band
|
||||
must produce matched=False with the diverging field named."""
|
||||
market = _make_market(0.845)
|
||||
record = _record_with_live_evaluate(
|
||||
market, news=FakeNews(_sentiment_for(0.845, 0.431))
|
||||
)
|
||||
assert record["guardrail_applied"] is True
|
||||
|
||||
monkeypatch.setattr(bayesian, "MAX_NEWS_ONLY_PROB_SHIFT", 0.10)
|
||||
decision = _replay_one(record, market)
|
||||
assert decision["matched"] is False
|
||||
# Tighter clamp (prior 0.845 ± 0.10 → est 0.745): edge_net drops from
|
||||
# 0.21 to 0.06 < regime 0.08, so the skip flips confidence → edge_net
|
||||
# and skip_reason is the first field _compare() sees diverge.
|
||||
assert decision["mismatch_field"] == "skip_reason"
|
||||
assert decision["skip_reason"] == "edge_net"
|
||||
|
||||
|
||||
def test_multi_row_cycle_preserves_order_and_isolation():
|
||||
"""Rows replay independently within a cycle: a family skip and a full
|
||||
evaluation with different sentiments don't bleed into each other."""
|
||||
m1 = _make_market(0.470, market_id="m1")
|
||||
m2 = _make_market(
|
||||
0.50, market_id="m2",
|
||||
question="Will Jane Doe win the Georgia Senate race?",
|
||||
)
|
||||
r1 = _record_with_live_evaluate(m1, news=FakeNews(_sentiment_for(0.470, 0.601)))
|
||||
r2 = _record_with_live_evaluate(m2) # no news → edge_net skip
|
||||
|
||||
decisions = asyncio.run(replay_cycle(
|
||||
datetime.now(timezone.utc),
|
||||
_snapshot(),
|
||||
[r1, r2],
|
||||
{"m1": _market_row(m1), "m2": _market_row(m2)},
|
||||
))
|
||||
assert [d["market_id"] for d in decisions] == ["m1", "m2"]
|
||||
assert all(d["matched"] is True for d in decisions)
|
||||
assert decisions[0]["skip_reason"] == "confidence"
|
||||
assert decisions[1]["skip_reason"] == "edge_net"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Run tagging
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_config_hash_stable_and_sensitive(monkeypatch):
|
||||
h1 = strategy_config_hash()
|
||||
assert strategy_config_hash() == h1
|
||||
monkeypatch.setattr(bayesian, "MAX_NEWS_ONLY_PROB_SHIFT", 0.10)
|
||||
assert strategy_config_hash() != h1
|
||||
|
||||
|
||||
def test_replay_news_returns_current_sentiment():
|
||||
news = ReplayNews()
|
||||
assert asyncio.run(news.get_sentiment("q")) == 0.0
|
||||
news.sentiment = -0.42
|
||||
assert asyncio.run(news.get_sentiment("q")) == -0.42
|
||||
|
||||
|
||||
def test_build_market_reconstruction():
|
||||
market = _make_market(0.37)
|
||||
record = _record_with_live_evaluate(market)
|
||||
rebuilt = build_market(_market_row(market), record)
|
||||
assert rebuilt.id == market.id
|
||||
assert rebuilt.yes_price == pytest.approx(0.37)
|
||||
assert rebuilt.volume_24h == pytest.approx(market.volume_24h)
|
||||
assert rebuilt.end_date == market.end_date
|
||||
assert rebuilt.category == "politics"
|
||||
assert market_family_key(rebuilt) == market_family_key(market)
|
||||
@@ -0,0 +1,224 @@
|
||||
"""
|
||||
Tests for the Replay R0 snapshot recorder (strategy-side record accumulation).
|
||||
|
||||
Every evaluate() call must leave exactly one record in _cycle_records, whatever
|
||||
exit path it takes, so the signals archive is a complete account of each cycle.
|
||||
DB persistence itself (save_signal_records) is exercised in prod; these tests
|
||||
cover the record-building contract the replay engine will rely on:
|
||||
|
||||
- one record per market per evaluate() call, drained per cycle
|
||||
- skip_reason granularity (prior_extreme / family / edge_net / confidence /
|
||||
unsupported / reentry_guard via record_skip)
|
||||
- full input/output fields on records that reached edge computation
|
||||
- news_budget_skipped distinguishes "not asked" from "no news"
|
||||
"""
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
import bot.strategy.bayesian as bayesian
|
||||
from bot.data.external import ExternalSignals
|
||||
from bot.data.polymarket import Market
|
||||
from bot.strategy.bayesian import (
|
||||
MAX_NEWS_QUERIES_PER_CYCLE,
|
||||
BayesianStrategy,
|
||||
)
|
||||
|
||||
from tests.test_news_guardrail import FakeNews, _sentiment_for
|
||||
|
||||
|
||||
def _end_date(days_ahead: int = 20) -> str:
|
||||
dt = datetime.now(timezone.utc) + timedelta(days=days_ahead)
|
||||
return dt.strftime("%Y-%m-%dT00:00:00Z")
|
||||
|
||||
|
||||
def _make_market(
|
||||
yes_price: float,
|
||||
question: str = "Will John Smith win the election?",
|
||||
category: str = "politics",
|
||||
market_id: str = "mkt-recorder-1",
|
||||
) -> Market:
|
||||
return Market(
|
||||
id=market_id,
|
||||
condition_id="cond-recorder-1",
|
||||
question=question,
|
||||
yes_token_id="yes-tok",
|
||||
no_token_id="no-tok",
|
||||
yes_price=yes_price,
|
||||
no_price=1.0 - yes_price,
|
||||
volume_24h=50_000.0,
|
||||
end_date=_end_date(), # ~20 d → politics regime_min 0.08
|
||||
active=True,
|
||||
category=category,
|
||||
)
|
||||
|
||||
|
||||
def _make_signals() -> ExternalSignals:
|
||||
return ExternalSignals(
|
||||
btc_price=100_000.0,
|
||||
btc_change_24h=0.0,
|
||||
eth_price=4_000.0,
|
||||
eth_change_24h=0.0,
|
||||
btc_dominance=50.0,
|
||||
fear_greed_index=50,
|
||||
fear_greed_label="neutral",
|
||||
total_market_cap_change=0.0,
|
||||
valid=True,
|
||||
)
|
||||
|
||||
|
||||
def _evaluate(strategy: BayesianStrategy, market: Market, families=None) -> None:
|
||||
asyncio.run(strategy.evaluate(market, _make_signals(), families or set()))
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Full-evaluation records: every input/output field the replay needs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_confidence_skip_record_has_full_fields():
|
||||
"""Politics market whose edge passes but confidence blocks (the known
|
||||
politics ceiling): record must carry the complete decision context."""
|
||||
sentiment = _sentiment_for(0.470, 0.601) # Georgia signature: edge_net 0.091
|
||||
strategy = BayesianStrategy(news=FakeNews(sentiment), manifold=None, db=None)
|
||||
market = _make_market(0.470)
|
||||
_evaluate(strategy, market)
|
||||
|
||||
records = strategy.drain_cycle_records()
|
||||
assert len(records) == 1
|
||||
rec = records[0]
|
||||
assert rec["market_id"] == "mkt-recorder-1"
|
||||
assert rec["skip_reason"] == "confidence"
|
||||
assert rec["category"] == "politics"
|
||||
assert rec["polymarket_price"] == pytest.approx(0.470)
|
||||
assert rec["prior_prob"] == pytest.approx(0.470)
|
||||
assert rec["estimated_prob"] == pytest.approx(0.601, abs=1e-3)
|
||||
assert rec["raw_final_prob"] == pytest.approx(0.601, abs=1e-3)
|
||||
assert rec["edge_net"] == pytest.approx(0.091, abs=1e-3)
|
||||
assert rec["regime_min_edge"] == pytest.approx(0.08)
|
||||
assert rec["passed_net"] is True
|
||||
assert rec["confidence"] == pytest.approx(0.50)
|
||||
assert rec["direction"] == "BUY_YES"
|
||||
assert rec["news_sentiment"] == pytest.approx(sentiment, abs=1e-6)
|
||||
assert rec["feat_news_lo"] != 0.0
|
||||
assert rec["news_budget_skipped"] is False
|
||||
assert rec["guardrail_applied"] is False
|
||||
assert rec["guardrail_changed_decision"] is False
|
||||
assert rec["days_to_resolution"] is not None
|
||||
assert rec["acted_on"] is False
|
||||
|
||||
|
||||
def test_edge_net_skip_record():
|
||||
"""No news, no edge → skip_reason=edge_net with passed_net False."""
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
market = _make_market(0.50)
|
||||
_evaluate(strategy, market)
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["skip_reason"] == "edge_net"
|
||||
assert rec["passed_net"] is False
|
||||
assert rec["estimated_prob"] == pytest.approx(0.50, abs=1e-3)
|
||||
assert rec["feat_news_lo"] == 0.0
|
||||
|
||||
|
||||
def test_guardrail_fields_recorded_when_clamped():
|
||||
"""Guardrail clamp shows up in the record (applied=True, raw != final)."""
|
||||
strategy = BayesianStrategy(
|
||||
news=FakeNews(_sentiment_for(0.845, 0.431)), manifold=None, db=None
|
||||
)
|
||||
market = _make_market(0.845)
|
||||
_evaluate(strategy, market)
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["guardrail_applied"] is True
|
||||
assert rec["raw_final_prob"] == pytest.approx(0.431, abs=1e-3)
|
||||
assert rec["estimated_prob"] == pytest.approx(
|
||||
0.845 - bayesian.MAX_NEWS_ONLY_PROB_SHIFT, abs=1e-3
|
||||
)
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Early-skip records: minimal but present
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_prior_extreme_record():
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
_evaluate(strategy, _make_market(0.03))
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["skip_reason"] == "prior_extreme"
|
||||
assert rec["polymarket_price"] == pytest.approx(0.03)
|
||||
assert rec["prior_prob"] == pytest.approx(0.05) # clamped prior
|
||||
assert rec["estimated_prob"] is None
|
||||
assert rec["edge_net"] is None
|
||||
|
||||
|
||||
def test_family_skip_record():
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
market = _make_market(0.50)
|
||||
from bot.data.polymarket import market_family_key
|
||||
_evaluate(strategy, market, families={market_family_key(market)})
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["skip_reason"] == "family"
|
||||
assert rec["family_key"] is not None
|
||||
|
||||
|
||||
def test_unsupported_record():
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
market = _make_market(0.50, question="Will it rain tomorrow?", category="")
|
||||
_evaluate(strategy, market)
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["skip_reason"] == "unsupported"
|
||||
|
||||
|
||||
def test_record_skip_external_reason():
|
||||
"""main.py records reentry-guard skips through record_skip()."""
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
strategy.record_skip(_make_market(0.50), "reentry_guard")
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["skip_reason"] == "reentry_guard"
|
||||
assert rec["estimated_prob"] is None
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Budget flag + cycle lifecycle
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_news_budget_skipped_flag():
|
||||
"""With the cycle budget exhausted, the record must say GNews was never
|
||||
asked — feat_news_lo=0.0 alone would be indistinguishable from no-news."""
|
||||
strategy = BayesianStrategy(news=FakeNews(0.9), manifold=None, db=None)
|
||||
strategy._news_queries_this_cycle = MAX_NEWS_QUERIES_PER_CYCLE
|
||||
_evaluate(strategy, _make_market(0.50))
|
||||
|
||||
rec = strategy.drain_cycle_records()[0]
|
||||
assert rec["news_budget_skipped"] is True
|
||||
assert rec["news_sentiment"] == 0.0
|
||||
assert rec["feat_news_lo"] == 0.0
|
||||
|
||||
|
||||
def test_drain_empties_and_reset_clears():
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
_evaluate(strategy, _make_market(0.50))
|
||||
assert len(strategy.drain_cycle_records()) == 1
|
||||
assert strategy.drain_cycle_records() == []
|
||||
|
||||
_evaluate(strategy, _make_market(0.50))
|
||||
strategy.reset_cycle()
|
||||
assert strategy.drain_cycle_records() == []
|
||||
|
||||
|
||||
def test_one_record_per_market_accumulates_in_order():
|
||||
strategy = BayesianStrategy(news=None, manifold=None, db=None)
|
||||
_evaluate(strategy, _make_market(0.03, market_id="m1")) # prior_extreme
|
||||
_evaluate(strategy, _make_market(0.50, market_id="m2")) # edge_net
|
||||
_evaluate(strategy, _make_market(0.97, market_id="m3")) # prior_extreme
|
||||
|
||||
records = strategy.drain_cycle_records()
|
||||
assert [r["market_id"] for r in records] == ["m1", "m2", "m3"]
|
||||
assert [r["skip_reason"] for r in records] == [
|
||||
"prior_extreme", "edge_net", "prior_extreme",
|
||||
]
|
||||
Reference in New Issue
Block a user