Compare commits

..
Author SHA1 Message Date
Renovate Bot edfbaa0f55 chore(deps): update dependency uvicorn to v0.50.2 2026-07-06 12:01:40 +00:00
chemavx dc3b17e10c docs: captura de Grafana (90d) + nota sobre el panel Win Rate vacío
El panel Win Rate estuvo vacío los 81 días por diseño, no por bug: la API
emite win_rate: null hasta >=10 mercados resueltos (hubo 2) y Grafana no
renderiza null. Documentado en el informe final.
2026-07-06 11:45:16 +00:00
chemavxandClaude Fable 5 eef721a9ed docs: README de portfolio — arquitectura, decisiones, resultados y archivado
Reescritura completa como pieza de portfolio: problema/solución, diagrama
Mermaid de arquitectura, decisiones de diseño, stack y sección de estado
del proyecto (archivado por el bloqueo regulatorio de la DGOJ a Polymarket
en España, mayo 2026, con enlaces a las fuentes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 07:40:31 +00:00
chemavxandClaude Fable 5 cef8a7f2e1 docs: informe final de paper trading + evidencia visual
Generado desde la BD de producción (snapshot 2026-07-06) con el bot aún
vivo, Fase 1 del plan de decomisión. Realized +$247.78 (n=2, anecdótico
y señalado como tal), MTM de las 5 posiciones abiertas a precios reales
de la Gamma API, y el foco en el activo real: pipeline de señales
(~223k evaluaciones), replay determinista y observabilidad.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 07:40:31 +00:00
chemavxandClaude Fable 5 1dde4d27eb ci: switch trigger to manual workflow_dispatch for decommission docs phase [skip ci]
Phase 1 of the decommission plan produces documentation-only commits;
push:main would rebuild and redeploy all three images on each one.
The workflow is kept (not deleted) and can still be run manually from
the Gitea Actions UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 07:24:20 +00:00
chemavx 7804d25c51 Merge pull request 'ci: bump outcomes-joiner CronJob image tag alongside deployment-bot' (#18) from ci/bump-outcomes-cronjob-image into main
CI/CD / build-and-push (push) Successful in 15s
2026-07-02 20:21:58 +00:00
chemavxandClaude Fable 5 0816e19740 ci: bump outcomes-joiner CronJob image tag alongside deployment-bot
The outcomes-joiner CronJob (k8s-manifests, Replay R2) runs the same bot
image; without this its tag would freeze at the sha it was created with
while the deployment moves on. Same sed, one more file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:21:39 +00:00
chemavx 2b326ad54f Merge pull request 'feat(replay): R2 outcomes + calibration metrics' (#17) from feat/replay-r2-outcomes into main
CI/CD / build-and-push (push) Successful in 7s
2026-07-02 20:12:21 +00:00
chemavxandClaude Fable 5 124b6d8558 feat(replay): R2 outcomes + calibration metrics
Scores every archived estimate against reality — the sample multiplier
the phase plan calls for: Brier/log-loss of estimated_prob benchmarked
against the market price (prior_prob) on the same rows, over ALL
evaluations with a resolved outcome, not just executed trades.

- schema.sql: market_outcomes (one row per resolved market; outcome =
  final YES price 1.0/0.0, UMA-final only)
- bot/outcomes.py: CLI (python -m bot.outcomes) with two phases —
  fetch resolutions for archived markets via the existing
  get_market_resolution() (open/disputed/ambiguous markets simply retry
  next invocation; no data-loss urgency, Gamma reports past resolutions
  at any time), then compute calibration: Brier micro (per evaluation) /
  macro (per market — the honest sample size given ~1 eval/min
  autocorrelation), log-loss with 1e-9 clipping, per-category breakdown.
  --run-id scores a replay run's re-estimates instead of the archive
  (counterfactual calibration).
- db.py: 4 accessors (pending markets, outcome upsert, coverage,
  calibration rows for archive or run)
- tests: 12 new (116 total green); compute_calibration is a pure
  function driven by literals

No prod behavior change: the bot never imports bot.outcomes; the only
shared surface is the idempotent schema migration (dry-run BEGIN/ROLLBACK
clean against prod).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:09:53 +00:00
chemavx 6c544e46e2 Merge pull request 'feat(replay): R1 replay core — clock injection + replay of archived cycles' (#16) from feat/replay-r1-core into main
CI/CD / build-and-push (push) Successful in 8s
2026-07-02 19:57:40 +00:00
chemavxandClaude Fable 5 0ac48ba7f8 feat(replay): R1 replay core — clock injection + replay of archived cycles
Re-executes BayesianStrategy.evaluate() over the R0 archive and stores
results in replay_runs/replay_decisions, tagged with git sha + a hash of
the strategy constants (same hash vs archive = determinism check,
different hash = counterfactual run).

- bayesian.py: optional as_of param on evaluate()/_days_to_resolution()
  (clock injection; default None = wall clock, prod behavior unchanged —
  the only touch to frozen code, purely additive)
- bot/replay.py: replay engine + CLI (python -m bot.replay --from --to);
  ReplayNews feeds archived sentiment back (GNews never called, per-cycle
  budget bypassed — archived sentiment already encodes it); manifold/db
  not wired (observational-only in prod); recorded-vs-replayed compare
  at 1e-9 tolerance
- schema.sql: replay_runs + replay_decisions (+ indexes), idempotent
- db.py: 6 replay accessors/writers
- tests: 19 new round-trip fidelity tests (104 total green)

Validated against a real prod cycle (2026-07-02T14:03:15Z, 46 markets,
4 skip paths incl. the Georgia confidence record): 46/46 matched,
max float delta 0.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:05:25 +00:00
chemavx eb4f67414a Merge pull request 'feat(replay): R0 snapshot recorder — archivo por ciclo en signals' (#15) from feat/replay-r0-recorder into main
CI/CD / build-and-push (push) Successful in 9s
2026-07-02 12:16:12 +00:00
14 changed files with 1706 additions and 12 deletions
+6 -4
View File
@@ -1,9 +1,10 @@
name: CI/CD
# Decommission (2026-07): trigger switched from push:main to manual only, so
# documentation commits don't rebuild/redeploy images. Run from the Gitea UI
# (Actions → Run workflow) if a rebuild is ever needed again.
on:
push:
branches:
- main
workflow_dispatch:
env:
REGISTRY: gitea.gitea.svc.cluster.local:3000
@@ -178,7 +179,8 @@ jobs:
# keep their current (still existing) tag in the registry.
if [ "${{ steps.changes.outputs.build_bot }}" = "true" ]; then
sed -i "s|image: .*polymarket-bot[^-].*|image: git.chemavx.xyz/chemavx/polymarket-bot:${TAG}|g" \
polymarket-bot/deployment-bot.yaml
polymarket-bot/deployment-bot.yaml \
polymarket-bot/cronjob-outcomes.yaml
fi
if [ "${{ steps.changes.outputs.build_api }}" = "true" ]; then
sed -i "s|image: .*polymarket-bot-api.*|image: git.chemavx.xyz/chemavx/polymarket-bot-api:${TAG}|g" \
+137 -3
View File
@@ -1,7 +1,118 @@
# polymarket-bot
Bot de paper-trading para Polymarket con estrategia bayesiana, API FastAPI y
dashboard React. Corre en k3s vía GitOps (Gitea Actions → registry → ArgoCD).
Bot de **paper trading** para mercados de predicción de Polymarket: estrategia
bayesiana sobre el precio de mercado como prior, señales externas en log-odds,
gates de riesgo en cascada y un motor de replay determinista para calibración
offline. API FastAPI + dashboard React. Desplegado en k3s vía GitOps
(Gitea Actions → registry → ArgoCD), con PostgreSQL, secrets por Infisical y
observabilidad completa (Grafana, uptime-kuma, Telegram).
> **📦 Estado del proyecto: ARCHIVADO (julio 2026).**
> El 26 de mayo de 2026 la DGOJ [ordenó el bloqueo cautelar de Polymarket y
> Kalshi en España](https://www.ordenacionjuego.es/novedades/dgoj-abre-expediente-sancionador-plataformas-polymarket-kalshi-ordena-bloqueo-sus-webs)
> por operar sin licencia de juego ([CoinDesk](https://www.coindesk.com/policy/2026/05/26/spain-joins-growing-list-of-countries-shutting-out-polymarket-and-kalshi)).
> Con el bloqueo a nivel de ISP la fuente de datos primaria desaparece, así que
> el proyecto se jubila de forma ordenada: datos respaldados y verificados,
> resultados documentados en el [informe final](docs/informe-final.md), e
> infraestructura apagada vía GitOps. El cierre es parte de la historia del
> proyecto, no un abandono — los módulos centrales son agnósticos de la fuente
> y el pivote natural es Manifold Markets (API abierta, play-money, sin
> fricción regulatoria).
## El problema y la solución
Los mercados de predicción publican probabilidades implícitas en el precio.
La hipótesis: en mercados de bajo volumen hay ineficiencias detectables
combinando el precio con señales externas (noticias, sentimiento cripto,
mercados espejo en otras plataformas). El bot la pone a prueba **sin arriesgar
dinero**, con la disciplina de un sistema real:
1. **Prior desde el precio** de Polymarket (mid del order book CLOB).
2. **Ajuste bayesiano en log-odds** con señales independientes: GNews (con
presupuesto de 5 consultas/día y *guardrail* que limita el ajuste a
prior±0,25), Fear&Greed, momentum BTC, dominancia BTC, y Manifold Markets
como señal cruzada (en modo observacional tras detectarse un bug de
inversión — auditado en 165k filas).
3. **Gates en cascada** antes de operar: prior extremo (<0,08 / >0,92), edge
bruto mínimo por régimen de volatilidad, edge neto positivo tras comisión
(2%) y spread, confianza ≥0,55, conflicto de familias, guard de reentrada.
4. **Agrupación por familias de mercados**: mercados hermanos del mismo evento
(candidatos de una misma elección, umbrales de un mismo precio) comparten
`family_key`; solo se mantiene la posición de mayor edge para no apilar la
misma apuesta con dos nombres.
5. **Sizing por Kelly fraccionado** con techo por posición sobre un bankroll
simulado de $10.000.
Resultado honesto en 81 días: 12 trades (n estadísticamente irrelevante, y el
informe lo subraya), realized **+$247,78**, y — más interesante — **~223.000
evaluaciones archivadas con su decisión reproducible**, porque el sistema
prefirió no operar antes que operar sin ventaja medible.
Detalle completo: [docs/informe-final.md](docs/informe-final.md).
## Arquitectura
```mermaid
flowchart LR
subgraph ext["Fuentes externas"]
PM["Polymarket<br/>CLOB + Gamma API"]
GN["GNews"]
MF["Manifold Markets"]
CG["CoinGecko /<br/>alternative.me"]
end
subgraph k3s["k3s · namespace polymarket-bot"]
BOT["bot (ciclo ~64s)<br/>estrategia bayesiana + gates"]
PG[("PostgreSQL 16<br/>trades · signals · replay")]
API["api (FastAPI :8000)"]
DASH["dashboard (React/nginx)"]
CJ1["CronJob metrics-retention 00:10"]
CJ2["CronJob outcomes-joiner 00:30"]
end
subgraph obs["Observabilidad"]
GRAF["Grafana"]
UK["uptime-kuma"]
TG["Telegram"]
end
PM --> BOT
GN --> BOT
MF --> BOT
CG --> BOT
BOT --> PG
PG --> API
API --> DASH
API --> GRAF
UK --> DASH
BOT --> TG
PM --> CJ2
CJ2 --> PG
CJ1 --> PG
subgraph ci["GitOps"]
GA["Gitea Actions"] --> REG["registry"] --> ARGO["ArgoCD<br/>prune + selfHeal"]
end
ARGO -.-> k3s
```
## Decisiones de diseño
- **Paper trading por diseño, no como demo**: el bot exigía ≥10 mercados
resueltos y ≥30 días antes de considerarse promocionable a dinero real, y
reportaba `n/a (insufficient_sample)` en vez de métricas infladas.
- **Todo skip es un dato**: cada evaluación archiva prior, estimación, edge
bruto/neto, descomposición por feature en log-odds y el gate exacto que la
bloqueó (~55k filas/día en `signals`).
- **Replay determinista** (julio 2026): inyección de reloj + re-ejecución de
ciclos archivados con 22.697/22.697 decisiones idénticas; joiner diario de
resoluciones UMA para calibrar sobre *todas* las evaluaciones, no solo los
trades.
- **Los errores se cierran y se documentan**: el bug de inversión de Manifold
y los conflictos de familia cerraron 5 posiciones en abril; la señal pasó a
observacional-only con auditoría completa en vez de eliminarse.
- **GitOps estricto**: nunca `kubectl` para cambios persistentes (ArgoCD con
`selfHeal` revierte cualquier parche manual); secrets fuera de git vía
Infisical operator; smoke test PostSync con notificación a Telegram.
## Componentes
@@ -11,10 +122,19 @@ dashboard React. Corre en k3s vía GitOps (Gitea Actions → registry → ArgoCD
| api | `api/` (+ `bot/` como librería) | `polymarket-bot-api` | `uvicorn api.main:app` |
| dashboard | `dashboard/` | `polymarket-bot-dashboard` | nginx estático |
Dashboard: https://polymarket.chemavx.xyz
Dashboard (hasta el apagado): https://polymarket.chemavx.xyz
## Stack
Python 3 (asyncio) · FastAPI · React + Vite · PostgreSQL 16 · k3s · ArgoCD ·
Gitea Actions + BuildKit · Infisical (secrets) · Grafana + Prometheus ·
uptime-kuma · Telegram Bot API.
## CI/CD
> ⚠️ **Decomisión**: el trigger automático `push:main` se desactivó en la Fase 1
> del archivado; el workflow queda solo bajo `workflow_dispatch` manual.
`.gitea/workflows/ci.yml` construye **solo las imágenes cuyos ficheros
cambiaron** en el push (diff contra `github.event.before`):
@@ -33,3 +153,17 @@ sincroniza vía webhook en segundos.
```bash
python3 -m pytest tests/ -q
```
## Datos y backup
Base de datos respaldada y verificada (2026-07-06): `pg_dump -Fc` + exports
CSV.gz de las tablas de señales y auditoría, con checksums y restore de prueba
(recuentos idénticos 11/11 tablas). Ubicación: almacenamiento de backups del
clúster con sync offsite. El dump final se toma en la fase de apagado.
## Documentación
- [Informe final de paper trading](docs/informe-final.md) — resultados,
pipeline de datos y estado de la BD al cierre.
- `docs/pivot-manifold.md` — notas de diseño del posible pivote (Fase 5,
pendiente).
+148
View File
@@ -747,6 +747,154 @@ class Database:
except (ValueError, IndexError):
return 0
# ── Replay R1: replay core ───────────────────────────────────────────────
async def get_replay_cycles(self, from_ts, to_ts) -> list:
"""Return the cycle_ts values with archived decisions in [from_ts, to_ts)."""
async with self._pool.acquire() as conn:
rows = await conn.fetch("""
SELECT DISTINCT cycle_ts FROM signals
WHERE cycle_ts >= $1 AND cycle_ts < $2
ORDER BY cycle_ts
""", from_ts, to_ts)
return [r["cycle_ts"] for r in rows]
async def get_ext_snapshot(self, cycle_ts) -> Optional[dict]:
"""Return one cycle's ExternalSignals snapshot, or None if missing."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT * FROM ext_snapshots WHERE cycle_ts = $1", cycle_ts
)
return dict(row) if row else None
async def get_cycle_signal_rows(self, cycle_ts) -> list[dict]:
"""Return one cycle's archived decision rows in original evaluation
order (id = insertion order = the order main.py evaluated them)."""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT * FROM signals WHERE cycle_ts = $1 ORDER BY id", cycle_ts
)
return [dict(r) for r in rows]
async def get_markets_by_ids(self, market_ids: list[str]) -> dict[str, dict]:
"""Return market metadata rows keyed by id (for Market reconstruction)."""
if not market_ids:
return {}
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT * FROM markets WHERE id = ANY($1::text[])", market_ids
)
return {r["id"]: dict(r) for r in rows}
async def save_replay_run(self, run: dict) -> None:
async with self._pool.acquire() as conn:
await conn.execute("""
INSERT INTO replay_runs (
run_id, git_sha, config_hash, config_json,
from_ts, to_ts, cycles, decisions, matched, mismatched, note
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11)
""",
run["run_id"], run["git_sha"], run["config_hash"],
run["config_json"], run["from_ts"], run["to_ts"],
run["cycles"], run["decisions"], run["matched"],
run["mismatched"], run["note"],
)
async def save_replay_decisions(self, run_id: str, decisions: list[dict]) -> None:
if not decisions:
return
rows = [
(
run_id, d["cycle_ts"], d["market_id"],
d["skip_reason"], d["prior_prob"], d["estimated_prob"],
d["raw_final_prob"], d["edge_gross"], d["edge_net"],
d["regime_min_edge"], d["days_to_resolution"],
d["confidence"], d["direction"], d["would_trade"],
d["recorded_skip_reason"], d["matched"], d["mismatch_field"],
)
for d in decisions
]
async with self._pool.acquire() as conn:
await conn.executemany("""
INSERT INTO replay_decisions (
run_id, cycle_ts, market_id,
skip_reason, prior_prob, estimated_prob,
raw_final_prob, edge_gross, edge_net,
regime_min_edge, days_to_resolution,
confidence, direction, would_trade,
recorded_skip_reason, matched, mismatch_field
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17)
""", rows)
# ── Replay R2: outcomes + calibration metrics ────────────────────────────
async def get_unresolved_archived_market_ids(self) -> list[str]:
"""Archived markets (present in signals) with no stored outcome yet."""
async with self._pool.acquire() as conn:
rows = await conn.fetch("""
SELECT DISTINCT s.market_id FROM signals s
LEFT JOIN market_outcomes o ON o.market_id = s.market_id
WHERE o.market_id IS NULL
ORDER BY s.market_id
""")
return [r["market_id"] for r in rows]
async def upsert_market_outcome(
self, market_id: str, outcome: float, resolved_at
) -> None:
async with self._pool.acquire() as conn:
await conn.execute("""
INSERT INTO market_outcomes (market_id, outcome, resolved_at)
VALUES ($1, $2, $3)
ON CONFLICT (market_id) DO UPDATE
SET outcome = EXCLUDED.outcome,
resolved_at = EXCLUDED.resolved_at,
fetched_at = NOW()
""", market_id, outcome, resolved_at)
async def get_outcome_coverage(self) -> dict:
"""How much of the archive is scorable: resolved vs archived markets."""
async with self._pool.acquire() as conn:
row = await conn.fetchrow("""
SELECT
(SELECT COUNT(DISTINCT market_id) FROM signals) AS archived,
(SELECT COUNT(*) FROM market_outcomes
WHERE market_id IN (SELECT DISTINCT market_id FROM signals)
) AS resolved
""")
return dict(row)
async def get_calibration_rows(self, run_id: Optional[str] = None) -> list[dict]:
"""Every archived evaluation with a full estimate AND a known outcome.
run_id None scores the R0 archive (signals); a run_id scores that
replay run's re-estimates instead (counterfactual calibration).
Rows without estimated_prob (skipped before estimation: prior_extreme,
unsupported, family, no_signals) carry no model prediction to score.
"""
async with self._pool.acquire() as conn:
if run_id is None:
rows = await conn.fetch("""
SELECT s.market_id, s.category,
s.estimated_prob, s.prior_prob, o.outcome
FROM signals s
JOIN market_outcomes o ON o.market_id = s.market_id
WHERE s.estimated_prob IS NOT NULL
AND s.prior_prob IS NOT NULL
""")
else:
rows = await conn.fetch("""
SELECT d.market_id, m.category,
d.estimated_prob, d.prior_prob, o.outcome
FROM replay_decisions d
JOIN market_outcomes o ON o.market_id = d.market_id
LEFT JOIN markets m ON m.id = d.market_id
WHERE d.run_id = $1
AND d.estimated_prob IS NOT NULL
AND d.prior_prob IS NOT NULL
""", run_id)
return [dict(r) for r in rows]
async def mark_manifold_audit_used(self, audit_id: str) -> None:
async with self._pool.acquire() as conn:
await conn.execute(
+82
View File
@@ -370,3 +370,85 @@ CREATE TABLE IF NOT EXISTS ext_snapshots (
total_market_cap_change DOUBLE PRECISION,
valid BOOLEAN
);
-- ─────────────────────────────────────────────────────────────────────────────
-- Replay R1: replay core — re-execute evaluate() over the R0 archive
--
-- A replay run reads cycles from signals + ext_snapshots + markets, rebuilds
-- the exact inputs (including archived news_sentiment — GNews is never called),
-- re-runs BayesianStrategy.evaluate() with the archived cycle_ts as clock, and
-- writes one replay_decisions row per (cycle, market).
--
-- replay_runs tags every run with the code (git_sha) and strategy constants
-- (config_hash) that produced it: two runs over the same window with different
-- config_hash values are a counterfactual comparison; same config_hash against
-- the recorded rows is a determinism check (mismatches should be 0, modulo
-- day-boundary crossings between cycle_ts and the original wall-clock).
--
-- matched: replayed decision equals the recorded one (skip_reason, probs,
-- confidence, direction). NULL when not comparable — e.g. reentry_guard
-- rows, recorded outside evaluate() with no decision fields to compare;
-- the replay still re-evaluates them, which is extra calibration data.
-- mismatch_field: first field that differed, for triage.
-- ─────────────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS replay_runs (
run_id TEXT PRIMARY KEY,
created_at TIMESTAMPTZ DEFAULT NOW(),
git_sha TEXT,
config_hash TEXT,
config_json TEXT,
from_ts TIMESTAMPTZ,
to_ts TIMESTAMPTZ,
cycles INTEGER,
decisions INTEGER,
matched INTEGER,
mismatched INTEGER,
note TEXT
);
CREATE TABLE IF NOT EXISTS replay_decisions (
id SERIAL PRIMARY KEY,
run_id TEXT NOT NULL,
cycle_ts TIMESTAMPTZ NOT NULL,
market_id TEXT NOT NULL,
-- replayed outputs (same semantics as the signals columns)
skip_reason TEXT,
prior_prob DOUBLE PRECISION,
estimated_prob DOUBLE PRECISION,
raw_final_prob DOUBLE PRECISION,
edge_gross DOUBLE PRECISION,
edge_net DOUBLE PRECISION,
regime_min_edge DOUBLE PRECISION,
days_to_resolution INTEGER,
confidence DOUBLE PRECISION,
direction TEXT,
would_trade BOOLEAN,
-- fidelity vs the recorded signals row
recorded_skip_reason TEXT,
matched BOOLEAN,
mismatch_field TEXT
);
CREATE INDEX IF NOT EXISTS idx_replay_decisions_run ON replay_decisions(run_id);
CREATE INDEX IF NOT EXISTS idx_replay_decisions_mkt ON replay_decisions(market_id);
-- ─────────────────────────────────────────────────────────────────────────────
-- Replay R2: outcomes + calibration metrics
--
-- One row per resolved market, fetched from the Gamma API via
-- get_market_resolution() (UMA-final only: a market closed but still in
-- proposal/dispute is not stored). outcome is the final YES price:
-- 1.0 = YES won, 0.0 = NO won.
--
-- Joining signals (or replay_decisions) to market_outcomes scores every
-- archived estimate against reality — Brier / log-loss of estimated_prob
-- benchmarked against the market price (prior_prob) on the same rows,
-- answering "does the model add value over the market?" across ALL
-- evaluations, not just executed trades.
-- ─────────────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS market_outcomes (
market_id TEXT PRIMARY KEY,
outcome DOUBLE PRECISION NOT NULL,
resolved_at TIMESTAMPTZ,
fetched_at TIMESTAMPTZ DEFAULT NOW()
);
+208
View File
@@ -0,0 +1,208 @@
"""
Replay R2 outcomes + calibration metrics.
Two phases, one CLI:
1. Fetch: for every archived market (present in `signals`) without a stored
outcome, ask the Gamma API via PolymarketClient.get_market_resolution()
the same UMA-finality gate the trading loop uses to settle positions.
Definitive resolutions are upserted into `market_outcomes`; open, disputed
or ambiguous markets are simply retried on the next invocation. There is
no data-loss urgency here (unlike the R0 recorder): Gamma reports past
resolutions at any time, so running this lazily loses nothing.
2. Score: join archived estimates to outcomes and compute Brier / log-loss of
estimated_prob, benchmarked against the market price (prior_prob) on the
same rows. This scores ALL evaluations with a full estimate the sample
multiplier the phase plan calls for not just executed trades. With
--run-id it scores a replay run's re-estimates instead (counterfactual
calibration: did config X predict better than the market?).
Reading the numbers: lower is better for both metrics; model < prior means
the model added information over the market price. Micro averages weight
every evaluation equally, so long-lived markets (~1 evaluation/min while in
the universe) dominate; macro averages score each market once (mean of its
evaluations) and answer the same question per market. Evaluations of one
market minutes apart are highly autocorrelated n_evaluations overstates
the effective sample size, n_markets is the honest one.
CLI:
python -m bot.outcomes # fetch new outcomes, then score archive
python -m bot.outcomes --fetch-only
python -m bot.outcomes --metrics-only
python -m bot.outcomes --run-id UUID # score a replay run (implies no fetch)
"""
import argparse
import asyncio
import logging
import math
from collections import defaultdict
from typing import Optional
from bot.data.db import Database
from bot.data.polymarket import PolymarketClient
log = logging.getLogger(__name__)
# Clip probabilities before log() so a (theoretical) hard 0/1 estimate on a
# wrong outcome scores ~20.7 nats instead of infinity poisoning the mean.
LOGLOSS_EPS = 1e-9
async def fetch_outcomes(poly, market_ids: list[str]) -> list[dict]:
"""Resolve archived markets against Gamma; returns only definitive ones.
Sequential on purpose: ~50 markets per invocation, and the Gamma API has
no bulk endpoint. get_market_resolution() already returns None on API
errors and resolved=False on open/disputed/ambiguous markets.
"""
resolved = []
for market_id in market_ids:
res = await poly.get_market_resolution(market_id)
if res is None or not res.resolved or res.resolution is None:
continue
resolved.append({
"market_id": market_id,
"outcome": res.resolution,
"resolved_at": res.resolved_at,
})
return resolved
def _logloss(p: float, outcome: float) -> float:
p = min(max(p, LOGLOSS_EPS), 1.0 - LOGLOSS_EPS)
return -math.log(p) if outcome == 1.0 else -math.log(1.0 - p)
def compute_calibration(rows: list[dict]) -> Optional[dict]:
"""Score estimated_prob vs prior_prob against outcomes; None if no rows.
rows: dicts with market_id, category, estimated_prob, prior_prob, outcome.
Pure function the CLI feeds it DB rows, tests feed it literals.
"""
if not rows:
return None
n = len(rows)
brier_model = sum((r["estimated_prob"] - r["outcome"]) ** 2 for r in rows) / n
brier_prior = sum((r["prior_prob"] - r["outcome"]) ** 2 for r in rows) / n
logloss_model = sum(_logloss(r["estimated_prob"], r["outcome"]) for r in rows) / n
logloss_prior = sum(_logloss(r["prior_prob"], r["outcome"]) for r in rows) / n
by_market: dict[str, list[dict]] = defaultdict(list)
for r in rows:
by_market[r["market_id"]].append(r)
market_briers = [
(
sum((r["estimated_prob"] - r["outcome"]) ** 2 for r in mrows) / len(mrows),
sum((r["prior_prob"] - r["outcome"]) ** 2 for r in mrows) / len(mrows),
)
for mrows in by_market.values()
]
brier_model_macro = sum(b[0] for b in market_briers) / len(market_briers)
brier_prior_macro = sum(b[1] for b in market_briers) / len(market_briers)
by_category: dict[str, list[dict]] = defaultdict(list)
for r in rows:
by_category[r["category"] or "unknown"].append(r)
per_category = {
cat: {
"n": len(crows),
"markets": len({r["market_id"] for r in crows}),
"brier_model": sum((r["estimated_prob"] - r["outcome"]) ** 2
for r in crows) / len(crows),
"brier_prior": sum((r["prior_prob"] - r["outcome"]) ** 2
for r in crows) / len(crows),
}
for cat, crows in sorted(by_category.items())
}
return {
"n_evaluations": n,
"n_markets": len(by_market),
"brier_model": brier_model,
"brier_prior": brier_prior,
"brier_model_macro": brier_model_macro,
"brier_prior_macro": brier_prior_macro,
"logloss_model": logloss_model,
"logloss_prior": logloss_prior,
"per_category": per_category,
}
def print_report(metrics: Optional[dict], source: str) -> None:
if metrics is None:
print(f"calibration : no scorable rows yet for {source} "
"(no archived estimate has a resolved outcome)")
return
print(f"calibration : {source}{metrics['n_evaluations']} evaluations, "
f"{metrics['n_markets']} markets")
print(f"{'':14s}{'model':>10s}{'market':>10s}{'delta':>10s}")
for label, m_key, p_key in (
("Brier micro", "brier_model", "brier_prior"),
("Brier macro", "brier_model_macro", "brier_prior_macro"),
("logloss micro", "logloss_model", "logloss_prior"),
):
m, p = metrics[m_key], metrics[p_key]
print(f" {label:12s}{m:>10.4f}{p:>10.4f}{m - p:>+10.4f}")
print(" (delta < 0 = model beats the market price)")
for cat, c in metrics["per_category"].items():
print(f" {cat:12s}n={c['n']:<6d} markets={c['markets']:<3d} "
f"brier model {c['brier_model']:.4f} vs market {c['brier_prior']:.4f}")
async def _amain(args: argparse.Namespace) -> None:
db = Database()
await db.connect()
try:
if not args.metrics_only and args.run_id is None:
pending = await db.get_unresolved_archived_market_ids()
poly = PolymarketClient()
try:
resolved = await fetch_outcomes(poly, pending)
finally:
await poly.close()
for out in resolved:
await db.upsert_market_outcome(
out["market_id"], out["outcome"], out["resolved_at"]
)
print(f"outcomes : {len(resolved)} newly resolved "
f"(of {len(pending)} pending markets checked)")
coverage = await db.get_outcome_coverage()
print(f"coverage : {coverage['resolved']}/{coverage['archived']} "
"archived markets resolved")
if args.fetch_only:
return
rows = await db.get_calibration_rows(run_id=args.run_id)
source = f"replay run {args.run_id}" if args.run_id else "R0 archive"
print_report(compute_calibration(rows), source)
finally:
await db.disconnect()
def main() -> None:
parser = argparse.ArgumentParser(
prog="python -m bot.outcomes",
description="Fetch market resolutions and score archived estimates.",
)
parser.add_argument("--fetch-only", action="store_true",
help="only fetch/store outcomes, skip metrics")
parser.add_argument("--metrics-only", action="store_true",
help="skip the Gamma fetch, score what is stored")
parser.add_argument("--run-id", default=None,
help="score a replay run's re-estimates instead of "
"the R0 archive (implies --metrics-only)")
args = parser.parse_args()
if args.fetch_only and args.metrics_only:
parser.error("--fetch-only and --metrics-only are mutually exclusive")
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
asyncio.run(_amain(args))
if __name__ == "__main__":
main()
+394
View File
@@ -0,0 +1,394 @@
"""
Replay R1 replay core.
Re-executes BayesianStrategy.evaluate() over the R0 archive (signals +
ext_snapshots + markets) and stores the outcome in replay_runs /
replay_decisions.
Determinism contract: evaluate() is a pure function of
(market, ext, occupied_families, as_of) plus the news client, so a replay
rebuilds exactly those four inputs from the archive:
market metadata from `markets`, per-cycle price/volume from `signals`
ext the cycle's `ext_snapshots` row
families a family-skipped row replays with its own family_key occupied;
every other row replays with no occupancy (the recorded
skip_reason already reflects the original portfolio state)
as_of the archived cycle_ts (clock injection, Replay R1)
GNews is never called: ReplayNews feeds back the archived news_sentiment.
The per-cycle query budget is bypassed (reset before every market) because
the archived sentiment already encodes the budget's effect — a
budget-skipped market was recorded with sentiment 0.0.
Manifold and the DB are not wired into the replayed strategy (manifold=None,
db=None): the signal is observational-only in production (feat_mfld_lo is
always 0.0 in the archive), so the replay reproduces decisions without
touching cooldowns or audit tables. If MANIFOLD_SIGNAL_ENABLED is ever
turned on, replayed decisions will diverge from recorded ones and the
matched/mismatch_field columns will say so.
Run tagging: every run stores the git sha and a hash of the strategy
constants. Same config_hash vs the archive = determinism check (expect 0
mismatches, modulo UTC-day-boundary crossings between cycle_ts and the
original wall-clock). Different config_hash = counterfactual run.
CLI:
python -m bot.replay --from 2026-07-02T00:00:00Z --to 2026-07-03 --note "..."
"""
import argparse
import asyncio
import hashlib
import json
import logging
import os
import subprocess
import uuid
from collections import Counter
from datetime import datetime, timedelta, timezone
from typing import Optional
import bot.strategy.bayesian as bayesian
from bot.data.db import Database
from bot.data.external import ExternalSignals
from bot.data.polymarket import Market
from bot.strategy.bayesian import BayesianStrategy
log = logging.getLogger(__name__)
# Absolute float tolerance for recorded-vs-replayed comparison. Archived
# values are float8 (exact IEEE-754 round-trip of Python floats), so any real
# divergence is far larger than this.
FLOAT_TOL = 1e-9
# Strategy constants that define a replay configuration. Hashed into
# replay_runs.config_hash; read from the module at call time so a
# counterfactual run can monkeypatch them and be tagged distinctly.
CONFIG_KEYS = (
"SPREAD_ESTIMATE",
"COMMISSION_RATE",
"MIN_CONFIDENCE",
"NEWS_LOGODDS_WEIGHT",
"MANIFOLD_LOGODDS_WEIGHT",
"MANIFOLD_SIGNAL_ENABLED",
"NEWS_GUARDRAIL_ENABLED",
"MAX_NEWS_ONLY_PROB_SHIFT",
"NEWS_MATERIAL_LOGODDS_THRESHOLD",
"MAX_NEWS_QUERIES_PER_CYCLE",
)
# Rows recorded outside evaluate() (via record_skip) carry no decision fields;
# the replay still re-evaluates them for calibration but cannot compare.
NON_COMPARABLE_SKIPS = {"reentry_guard"}
def strategy_config() -> dict:
return {k: getattr(bayesian, k) for k in CONFIG_KEYS}
def strategy_config_hash() -> str:
blob = json.dumps(strategy_config(), sort_keys=True)
return hashlib.sha256(blob.encode()).hexdigest()[:12]
def _git_sha() -> str:
sha = os.getenv("GIT_SHA", "")
if sha:
return sha
try:
return subprocess.run(
["git", "rev-parse", "--short", "HEAD"],
capture_output=True, text=True, timeout=5,
).stdout.strip() or "unknown"
except (OSError, subprocess.SubprocessError):
return "unknown"
class ReplayNews:
"""NewsClient stand-in that feeds archived sentiment back into evaluate().
No HTTP, no cache: the engine sets `sentiment` to the archived value
before each evaluate() call. Values below evaluate()'s 0.05 materiality
threshold were archived as 0.0, so the round-trip is exact.
"""
enabled = True
def __init__(self) -> None:
self.sentiment: float = 0.0
async def get_sentiment(self, question: str) -> float:
return self.sentiment
def get_freshness(self, question: str) -> float:
return 1.0 # only used by gnews_priority(), which replay never calls
def build_ext(snapshot: dict) -> ExternalSignals:
"""Rebuild the ExternalSignals a cycle was evaluated against."""
return ExternalSignals(
btc_price=snapshot["btc_price"],
btc_change_24h=snapshot["btc_change_24h"],
eth_price=snapshot["eth_price"],
eth_change_24h=snapshot["eth_change_24h"],
btc_dominance=snapshot["btc_dominance"],
fear_greed_index=snapshot["fear_greed_index"],
fear_greed_label=snapshot["fear_greed_label"],
total_market_cap_change=snapshot["total_market_cap_change"],
valid=snapshot["valid"],
)
def build_market(market_row: dict, signal_row: dict) -> Market:
"""Rebuild a Market: metadata from `markets`, per-cycle state from `signals`.
Token ids are irrelevant to evaluate() and left empty; no_price is the
YES complement (evaluate() never reads it either).
"""
yes_price = signal_row["polymarket_price"]
return Market(
id=market_row["id"],
condition_id=market_row["condition_id"] or "",
question=market_row["question"],
yes_token_id="",
no_token_id="",
yes_price=yes_price,
no_price=1.0 - yes_price,
volume_24h=signal_row["volume_24h"] or 0.0,
end_date=market_row["end_date"] or "",
active=True,
category=signal_row["category"] or (market_row["category"] or ""),
)
def _compare(recorded: dict, replayed: dict) -> Optional[str]:
"""Return the first field where replayed diverges from recorded, or None."""
if recorded["skip_reason"] != replayed["skip_reason"]:
return "skip_reason"
for field in ("prior_prob", "estimated_prob", "raw_final_prob",
"edge_net", "confidence"):
a, b = recorded[field], replayed[field]
if a is None and b is None:
continue
if a is None or b is None or abs(a - b) > FLOAT_TOL:
return field
if recorded["direction"] != replayed["direction"]:
return "direction"
return None
async def replay_cycle(
cycle_ts: datetime,
snapshot: dict,
signal_rows: list[dict],
market_rows: dict[str, dict],
) -> list[dict]:
"""Re-evaluate one archived cycle; returns one decision dict per row.
Pure with respect to the DB everything it needs is passed in, so tests
can drive it with synthetic rows.
"""
news = ReplayNews()
strategy = BayesianStrategy(news=news, manifold=None, db=None)
ext = build_ext(snapshot)
decisions: list[dict] = []
for row in signal_rows:
recorded_skip = row["skip_reason"]
decision = {
"cycle_ts": cycle_ts,
"market_id": row["market_id"],
"skip_reason": None,
"prior_prob": None,
"estimated_prob": None,
"raw_final_prob": None,
"edge_gross": None,
"edge_net": None,
"regime_min_edge": None,
"days_to_resolution": None,
"confidence": None,
"direction": None,
"would_trade": None,
"recorded_skip_reason": recorded_skip,
"matched": None,
"mismatch_field": None,
}
market_row = market_rows.get(row["market_id"])
if market_row is None:
# Should not happen (R0 upserts markets every cycle) — record the
# gap instead of crashing the run.
decision["matched"] = False
decision["mismatch_field"] = "market_missing"
decisions.append(decision)
continue
market = build_market(market_row, row)
# A family-skipped row replays against its own occupied family; all
# other rows replay unoccupied — their recorded skip_reason already
# reflects whatever portfolio state existed, and evaluate() checks
# the family gate before anything portfolio-dependent.
families = (
{row["family_key"]}
if recorded_skip == "family" and row["family_key"]
else set()
)
news.sentiment = row["news_sentiment"] or 0.0
# Bypass the per-cycle GNews budget: archived sentiment already
# encodes it (budget-skipped markets were recorded with 0.0).
strategy._news_queries_this_cycle = 0
signal = await strategy.evaluate(market, ext, families, as_of=cycle_ts)
rec = strategy.drain_cycle_records()[-1]
decision.update(
skip_reason=rec["skip_reason"],
prior_prob=rec["prior_prob"],
estimated_prob=rec["estimated_prob"],
raw_final_prob=rec["raw_final_prob"],
edge_gross=rec["edge_gross"],
edge_net=rec["edge_net"],
regime_min_edge=rec["regime_min_edge"],
days_to_resolution=rec["days_to_resolution"],
confidence=rec["confidence"],
direction=rec["direction"],
would_trade=signal is not None,
)
if recorded_skip in NON_COMPARABLE_SKIPS:
decision["matched"] = None # re-evaluated for calibration only
else:
mismatch = _compare(row, rec)
decision["matched"] = mismatch is None
decision["mismatch_field"] = mismatch
decisions.append(decision)
return decisions
async def run_replay(
db: Database,
from_ts: datetime,
to_ts: datetime,
note: str = "",
limit_cycles: Optional[int] = None,
) -> dict:
"""Replay every archived cycle in [from_ts, to_ts) and persist the run.
Returns the replay_runs row (plus a mismatch_fields Counter) for reporting.
"""
run_id = str(uuid.uuid4())
cycles = await db.get_replay_cycles(from_ts, to_ts)
if limit_cycles:
cycles = cycles[:limit_cycles]
decisions_total = 0
matched = 0
mismatched = 0
mismatch_fields: Counter = Counter()
skipped_cycles = 0
for cycle_ts in cycles:
snapshot = await db.get_ext_snapshot(cycle_ts)
if snapshot is None:
skipped_cycles += 1
log.warning("Replay: no ext_snapshot for cycle %s — skipped", cycle_ts)
continue
signal_rows = await db.get_cycle_signal_rows(cycle_ts)
market_rows = await db.get_markets_by_ids(
[r["market_id"] for r in signal_rows]
)
decisions = await replay_cycle(cycle_ts, snapshot, signal_rows, market_rows)
await db.save_replay_decisions(run_id, decisions)
decisions_total += len(decisions)
for d in decisions:
if d["matched"] is True:
matched += 1
elif d["matched"] is False:
mismatched += 1
mismatch_fields[d["mismatch_field"]] += 1
run = {
"run_id": run_id,
"git_sha": _git_sha(),
"config_hash": strategy_config_hash(),
"config_json": json.dumps(strategy_config(), sort_keys=True),
"from_ts": from_ts,
"to_ts": to_ts,
"cycles": len(cycles) - skipped_cycles,
"decisions": decisions_total,
"matched": matched,
"mismatched": mismatched,
"note": note,
}
await db.save_replay_run(run)
run["mismatch_fields"] = dict(mismatch_fields)
run["skipped_cycles"] = skipped_cycles
return run
def _parse_ts(value: str) -> datetime:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
async def _amain(args: argparse.Namespace) -> None:
db = Database()
await db.connect()
try:
run = await run_replay(
db,
from_ts=args.from_ts,
to_ts=args.to_ts,
note=args.note,
limit_cycles=args.limit_cycles,
)
finally:
await db.disconnect()
comparable = run["matched"] + run["mismatched"]
print(f"run_id : {run['run_id']}")
print(f"git_sha : {run['git_sha']} config_hash: {run['config_hash']}")
print(f"window : {run['from_ts'].isoformat()}{run['to_ts'].isoformat()}")
print(f"cycles : {run['cycles']} (skipped: {run['skipped_cycles']})")
print(f"decisions : {run['decisions']} ({comparable} comparable)")
print(f"matched : {run['matched']}")
print(f"mismatched : {run['mismatched']}")
if run["mismatch_fields"]:
for field, count in sorted(run["mismatch_fields"].items(), key=lambda x: -x[1]):
print(f" {field}: {count}")
def main() -> None:
parser = argparse.ArgumentParser(
prog="python -m bot.replay",
description="Replay archived trading cycles through the current strategy.",
)
now = datetime.now(timezone.utc)
parser.add_argument(
"--from", dest="from_ts", type=_parse_ts,
default=now - timedelta(hours=24),
help="window start, ISO-8601 (default: 24h ago)",
)
parser.add_argument(
"--to", dest="to_ts", type=_parse_ts, default=now,
help="window end, ISO-8601, exclusive (default: now)",
)
parser.add_argument("--note", default="", help="free-text tag for replay_runs")
parser.add_argument(
"--limit-cycles", type=int, default=None,
help="replay at most N cycles (smoke runs)",
)
args = parser.parse_args()
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
# evaluate() logs one INFO line per market — thousands per replay window.
logging.getLogger("bot.strategy.bayesian").setLevel(logging.WARNING)
asyncio.run(_amain(args))
if __name__ == "__main__":
main()
+18 -4
View File
@@ -167,15 +167,22 @@ def _regime_min_edge(category: str, days_to_resolution: int) -> float:
return 0.10 # tech, crypto/finance, events, default
def _days_to_resolution(end_date: str) -> int:
"""Return calendar days until market resolution, or 30 if unknown."""
def _days_to_resolution(end_date: str, as_of: Optional[datetime] = None) -> int:
"""Return calendar days until market resolution, or 30 if unknown.
as_of (Replay R1): reference clock for the computation. None (production)
means wall-clock now; a replay run passes the archived cycle_ts so
days-to-resolution and therefore the regime edge threshold is computed
against the moment the decision was originally made.
"""
if not end_date:
return 30 # conservative: treat as medium-term
try:
dt = datetime.fromisoformat(end_date.replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
days = (dt - datetime.now(timezone.utc)).days
now = as_of if as_of is not None else datetime.now(timezone.utc)
days = (dt - now).days
return max(0, days)
except (ValueError, TypeError):
return 30
@@ -457,10 +464,17 @@ class BayesianStrategy:
market: Market,
ext: ExternalSignals,
occupied_families: set[str],
as_of: Optional[datetime] = None,
) -> Optional[TradingSignal]:
"""
Evaluate a market and return a TradingSignal if actionable.
as_of (Replay R1): clock injection None in production (wall-clock
now); a replay passes the archived cycle_ts so the regime threshold
matches the original decision moment. Only days-to-resolution
depends on the clock; everything else is a pure function of
(market, ext, occupied_families) and the news/manifold clients.
Returns None with a structured log line in all skip cases.
Skip reasons (Phase 5 observability):
SKIP_UNSUPPORTED category not supported
@@ -558,7 +572,7 @@ class BayesianStrategy:
return None
# ── Phase 4: regime min-edge ─────────────────────────────────────────
days = _days_to_resolution(market.end_date)
days = _days_to_resolution(market.end_date, as_of)
regime_min = _regime_min_edge(category, days)
# ── Bayesian probability estimation ──────────────────────────────────
Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

+171
View File
@@ -0,0 +1,171 @@
# Informe final de paper trading — polymarket-bot
**Snapshot: 2026-07-06** · Generado desde la base de datos de producción con el bot
aún operativo, como parte de la Fase 1 del plan de decomisión (bloqueo ISP de
Polymarket en España, mayo 2026).
---
## ⚠️ Advertencia de tamaño muestral (léase primero)
**12 trades ejecutados, de los cuales solo 2 llegaron a resolución, no constituyen
evidencia estadística de nada.** Ni el P&L positivo demuestra que la estrategia
funcione, ni su ausencia lo habría refutado. El propio bot lo sabía: sus criterios
de promoción a dinero real exigían ≥10 mercados resueltos y ≥30 días de
observación (`promotion_ready: false`, win rate y Sharpe deliberadamente
reportados como `n/a (insufficient_sample)` en toda la instrumentación).
El valor demostrable de este proyecto no está en el P&L: está en la
infraestructura de evaluación, el pipeline de datos y la disciplina de gates que
impidió que el bot tradease sin ventaja medible. Este informe presenta el P&L
como lo que es — **una anécdota** — y dedica el grueso al sistema.
---
## 1. Resultados de trading (anecdóticos)
Bankroll simulado: **$10.000** (USDC virtual) · Capital desplegado: **$2.447** ·
Periodo de actividad: **1422 de abril de 2026** (los 12 trades se abrieron en 9
días; después, el endurecimiento de los gates de edge y confianza dejó el bot en
0 trades — funcionando, evaluando ~1.350 ciclos/día, sin encontrar ventaja neta
que superara sus propios umbrales).
### P&L realizado: **+$247,78** (2 mercados resueltos)
| Mercado | Dirección | Entrada | Resultado | P&L neto |
|---|---|---|---|---|
| Will Ken Paxton win the 2026 Texas Republican Primary? | BUY_YES | 0,618 | ✅ YES (2026-06-11) | **+$299,06** |
| Will NVIDIA be the largest company in the world…? | BUY_NO | 0,156 | ❌ YES (2026-06-30) | **$51,28** |
P&L neto de comisiones (2% simulado por lado). 1 acierto / 1 fallo: n=2.
### P&L no realizado (mark-to-market): **+$809,59** (5 posiciones abiertas)
Marcado a los precios reales de Polymarket (Gamma API) el 2026-07-06. Estas
posiciones **no están cerradas**: el número de abajo cambiaría cada día que los
mercados sigan moviéndose, y varios no resuelven hasta noviembre de 2026.
| Mercado | Dirección | Entrada | Precio actual (lado) | Coste neto | MTM P&L |
|---|---|---|---|---|---|
| Karen Bass gana la alcaldía de LA 2026 | BUY_YES | 0,268 | 0,595 | $510,00 | **+$600,07** |
| OpenAI IPO antes del 31-dic-2026 | BUY_NO | 0,618 | 0,785 | $510,00 | **+$125,11** |
| Demócratas ganan el Senado de Texas 2026 | BUY_YES | 0,418 | 0,435 | $509,49 | **+$10,32** |
| Demócratas ganan gobernación de Ohio 2026 | BUY_NO | 0,458 | 0,520 | $407,74 | **+$46,12** |
| Republicanos ganan gobernación de Nebraska 2026 | BUY_NO | 0,158 | 0,170 | $510,00 | **+$27,97** |
> Nota metodológica: el tracker interno reportaba `unrealized_pnl_est = +$522,73`,
> pero esa cifra es la **expectativa del propio modelo a fecha de entrada**
> (edge_net × coste), no un marcado a mercado. La tabla de arriba usa precios
> reales de mercado del día del snapshot, que es el criterio honesto. Ambas
> cifras coinciden en el signo; ninguna de las dos es un resultado realizado.
### Los otros 5 cierres (sin P&L): la historia real de abril
De los 7 trades cerrados, solo 2 cerraron por resolución. Los otros 5 los cerró
el propio bot al detectar defectos en su lógica — y son la parte más
instructiva del histórico:
- **3 por conflicto de familia**: dos posiciones sobre mercados hermanos del
mismo evento (p. ej. YES-demócratas y YES-republicanos en la misma
gobernación) — el agrupador de familias los detectó y cerró el lado débil.
- **2 por el bug de inversión de Manifold**: la señal cruzada de Manifold se
aplicaba invertida en mercados espejo. Detectado por instrumentación, las
posiciones afectadas se cerraron y el matching se movió a modo
observacional-only con auditoría completa (165.538 filas en
`manifold_match_audit`).
- **1 excluido de métricas** (`invalid_manifold_match_legacy`, P&L $0,00).
Después de esa semana el bot no volvió a operar: los gates endurecidos
(edge neto > mínimo por régimen, confianza ≥ 0,55, guardrail de noticias
prior±0,25) filtraron el 100% de las ~223.000 evaluaciones posteriores.
**Un sistema de paper trading cuyo resultado principal es "no tradees sin
ventaja" es un resultado**, y está instrumentado hasta el último skip.
---
## 2. El sistema (el activo real)
### Pipeline de señales
| Métrica | Valor (snapshot 2026-07-06) |
|---|---|
| Ciclos de evaluación archivados | **5.356** (cadencia ~64 s) |
| Evaluaciones mercado-ciclo archivadas | **222.738** |
| Mercados distintos evaluados | 87 (events 114,9k · politics 96,0k · crypto/finance 7,7k · tech 4,2k evaluaciones) |
| Archivo `signals` operativo desde | 2026-07-02 (replay R0) — crece ~55k filas/día |
| Histórico de métricas diarias | **81 días** (2026-04-14 → 2026-07-06, `metrics_daily`) |
| Snapshots de contexto externo | 5.356 (BTC, dominancia, Fear&Greed por ciclo) |
Cada evaluación archiva el prior, la probabilidad estimada, el edge bruto/neto,
la descomposición por feature (Fear&Greed, momentum, noticias, Manifold,
dominancia BTC en log-odds), el gate que la bloqueó y el contexto de mercado —
suficiente para reproducir la decisión completa offline.
### Motor de replay (R0R2, julio 2026)
- Inyección de reloj + replay determinista de ciclos archivados:
**22.697/22.697 decisiones idénticas** al reproducir el histórico.
- Joiner diario de resoluciones (CronJob 00:30 UTC) contra la Gamma API:
cobertura de outcomes 9/87 mercados al snapshot, creciendo sola.
- Diseñado para calibrar sobre **todas** las evaluaciones (no solo trades),
esquivando el problema del n=12.
### Estrategia (congelada desde 2026-07-03)
Prior desde el precio de Polymarket → ajuste bayesiano en log-odds con señales
externas (GNews con presupuesto de 5 consultas/día y guardrail prior±0,25;
Fear&Greed; momentum BTC; Manifold observacional) → gates en cascada: prior
extremo, edge bruto por régimen de volatilidad, edge neto tras comisión 2% y
spread, confianza mínima 0,55, conflictos de familia, guard de reentrada.
Sizing por Kelly fraccionado con techo por posición.
### Infraestructura
3 imágenes (bot / API FastAPI / dashboard) construidas por Gitea Actions solo
cuando cambian sus fuentes, desplegadas por ArgoCD (prune + selfHeal) en k3s;
PostgreSQL 16 en StatefulSet; secrets vía Infisical operator; smoke test
PostSync contra la API; backups nocturnos con sync offsite (rclone → MEGA);
observabilidad en Grafana + uptime-kuma + notificaciones Telegram (deploys,
checkpoints del bot, fallos de sync). Retención de métricas y join de outcomes
como CronJobs. **88 días de uptime del StatefulSet al snapshot.**
Detalle honesto de observabilidad: el panel *Win Rate* del dashboard de Grafana
(`docs/dashboard-grafana-2026-07-06.png`) pasó vacío los 81 días. No es una
query rota: la API emite `win_rate: null` hasta tener ≥10 mercados resueltos
(hubo 2), y Grafana no pinta nada con un null. El panel vacío es el criterio
de promoción del bot aplicándose a su propia monitorización.
---
## 3. Estado de la base de datos al snapshot
| Tabla | Filas | Contenido |
|---|---|---|
| `signals` | 222.549* | Archivo de evaluaciones por ciclo (desde 2026-07-02) |
| `manifold_match_audit` | 165.538 | Auditoría completa del matching Polymarket↔Manifold |
| `replay_decisions` | 22.697 | Decisiones reproducidas por el motor de replay |
| `ext_snapshots` | 5.353* | Contexto externo por ciclo |
| `metrics_daily` | 516 | Cierres diarios (81 días) + snapshots intradía de hoy |
| `markets` | 87 | Universo evaluado |
| `trades` | 12 | Ledger completo de paper trades |
| `market_outcomes` | 9 | Resoluciones UMA-final joineadas |
| `replay_runs` / `manifold_eval_cooldown` / `checkpoint_alerts` | 1 / 76 / 3 | Metadatos |
\* La BD sigue viva; las tablas por ciclo crecen ~55k y ~1,3k filas/día
respectivamente. Backup verificado del 2026-07-06 en
`/data/backups/backups/polymarket-decommission/` (pg_dump -Fc + CSV.gz +
checksums; restore de prueba con recuentos idénticos 11/11 tablas). El dump
final se hará en la Fase 3, justo antes del apagado.
---
## 4. Conclusión
El proyecto se archiva por una causa externa (bloqueo regulatorio del origen de
datos), no por fracaso técnico. Lo que queda: un motor de evaluación bayesiano
determinista y auditable, un pipeline de datos que archivó cada decisión con su
contexto completo, y la evidencia — anecdótica en P&L, sólida en ingeniería —
de que el sistema prefería no operar antes que operar sin ventaja. Todos los
módulos centrales (edge, familias, replay, observabilidad) son agnósticos de la
fuente y quedan listos para un eventual pivote a Manifold
(`docs/pivot-manifold.md`, Fase 5).
Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

+1 -1
View File
@@ -2,7 +2,7 @@
asyncpg==0.29.0
httpx==0.27.0
fastapi==0.111.0
uvicorn[standard]==0.29.0
uvicorn[standard]==0.50.2
pydantic==2.7.0
# Polymarket (install from PyPI when ready for real trading)
+174
View File
@@ -0,0 +1,174 @@
"""Replay R2 tests — outcome fetching and calibration scoring."""
import asyncio
import math
import pytest
from bot.data.polymarket import MarketResolution
from bot.outcomes import (
LOGLOSS_EPS,
compute_calibration,
fetch_outcomes,
print_report,
)
from datetime import datetime, timezone
class FakePoly:
"""get_market_resolution stand-in driven by a dict of canned responses."""
def __init__(self, responses: dict):
self.responses = responses
self.calls: list[str] = []
async def get_market_resolution(self, market_id: str):
self.calls.append(market_id)
return self.responses.get(market_id)
RESOLVED_AT = datetime(2026, 7, 1, 12, 0, tzinfo=timezone.utc)
def _row(market_id="m1", category="politics", est=0.6, prior=0.5, outcome=1.0):
return {
"market_id": market_id,
"category": category,
"estimated_prob": est,
"prior_prob": prior,
"outcome": outcome,
}
# ── fetch_outcomes ───────────────────────────────────────────────────────────
def test_fetch_keeps_only_definitive_resolutions():
poly = FakePoly({
"yes": MarketResolution(resolved=True, resolution=1.0,
resolved_at=RESOLVED_AT),
"no": MarketResolution(resolved=True, resolution=0.0,
resolved_at=None),
"open": MarketResolution(resolved=False),
"disputed": MarketResolution(resolved=False),
"apierror": None, # get_market_resolution returns None on HTTP errors
})
out = asyncio.run(
fetch_outcomes(poly, ["yes", "no", "open", "disputed", "apierror"])
)
assert poly.calls == ["yes", "no", "open", "disputed", "apierror"]
assert out == [
{"market_id": "yes", "outcome": 1.0, "resolved_at": RESOLVED_AT},
{"market_id": "no", "outcome": 0.0, "resolved_at": None},
]
def test_fetch_empty_list_is_noop():
poly = FakePoly({})
assert asyncio.run(fetch_outcomes(poly, [])) == []
assert poly.calls == []
# ── compute_calibration ──────────────────────────────────────────────────────
def test_no_rows_returns_none():
assert compute_calibration([]) is None
def test_single_row_known_values():
m = compute_calibration([_row(est=0.8, prior=0.6, outcome=1.0)])
assert m["n_evaluations"] == 1
assert m["n_markets"] == 1
assert m["brier_model"] == pytest.approx((0.8 - 1.0) ** 2)
assert m["brier_prior"] == pytest.approx((0.6 - 1.0) ** 2)
assert m["logloss_model"] == pytest.approx(-math.log(0.8))
assert m["logloss_prior"] == pytest.approx(-math.log(0.6))
# one market: macro == micro
assert m["brier_model_macro"] == pytest.approx(m["brier_model"])
assert m["brier_prior_macro"] == pytest.approx(m["brier_prior"])
def test_logloss_no_outcome_branch():
m = compute_calibration([_row(est=0.2, prior=0.7, outcome=0.0)])
assert m["logloss_model"] == pytest.approx(-math.log(0.8))
assert m["logloss_prior"] == pytest.approx(-math.log(0.3))
def test_logloss_clipping_keeps_hard_miss_finite():
# A hard 1.0 estimate on a NO outcome must not produce inf.
m = compute_calibration([_row(est=1.0, prior=0.5, outcome=0.0)])
assert math.isfinite(m["logloss_model"])
assert m["logloss_model"] == pytest.approx(-math.log(LOGLOSS_EPS))
def test_micro_weights_evaluations_macro_weights_markets():
# Market a: 3 evaluations, model error 0.1; market b: 1 evaluation, error 0.5.
rows = [
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
_row(market_id="a", est=0.9, prior=0.8, outcome=1.0),
_row(market_id="b", est=0.5, prior=0.6, outcome=1.0),
]
m = compute_calibration(rows)
assert m["n_evaluations"] == 4
assert m["n_markets"] == 2
# micro: (3*0.01 + 0.25) / 4 ; macro: (0.01 + 0.25) / 2
assert m["brier_model"] == pytest.approx((3 * 0.01 + 0.25) / 4)
assert m["brier_model_macro"] == pytest.approx((0.01 + 0.25) / 2)
assert m["brier_prior"] == pytest.approx((3 * 0.04 + 0.16) / 4)
assert m["brier_prior_macro"] == pytest.approx((0.04 + 0.16) / 2)
def test_model_beating_market_gives_negative_delta():
# est closer to the outcome than the price on every row
rows = [
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
_row(market_id="b", est=0.3, prior=0.45, outcome=0.0),
]
m = compute_calibration(rows)
assert m["brier_model"] < m["brier_prior"]
assert m["logloss_model"] < m["logloss_prior"]
def test_per_category_grouping_and_unknown():
rows = [
_row(market_id="a", category="politics", est=0.8, prior=0.6, outcome=1.0),
_row(market_id="b", category="politics", est=0.7, prior=0.6, outcome=1.0),
_row(market_id="c", category=None, est=0.4, prior=0.5, outcome=0.0),
]
m = compute_calibration(rows)
assert set(m["per_category"]) == {"politics", "unknown"}
pol = m["per_category"]["politics"]
assert pol["n"] == 2 and pol["markets"] == 2
assert pol["brier_model"] == pytest.approx((0.04 + 0.09) / 2)
unk = m["per_category"]["unknown"]
assert unk["n"] == 1 and unk["markets"] == 1
assert unk["brier_model"] == pytest.approx(0.16)
def test_repeated_market_counts_once_in_markets():
rows = [
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
_row(market_id="a", est=0.7, prior=0.55, outcome=1.0),
]
m = compute_calibration(rows)
assert m["n_markets"] == 1
assert m["per_category"]["politics"]["markets"] == 1
# ── print_report ─────────────────────────────────────────────────────────────
def test_report_handles_no_metrics(capsys):
print_report(None, "R0 archive")
assert "no scorable rows yet" in capsys.readouterr().out
def test_report_prints_all_metric_lines(capsys):
m = compute_calibration([
_row(market_id="a", est=0.8, prior=0.6, outcome=1.0),
_row(market_id="b", category=None, est=0.4, prior=0.5, outcome=0.0),
])
print_report(m, "R0 archive")
out = capsys.readouterr().out
assert "2 evaluations, 2 markets" in out
for label in ("Brier micro", "Brier macro", "logloss micro",
"politics", "unknown"):
assert label in out
+367
View File
@@ -0,0 +1,367 @@
"""
Tests for the Replay R1 replay core (bot/replay.py) and the as_of clock
injection in BayesianStrategy.evaluate().
The central contract is round-trip fidelity: a decision recorded by R0 and
replayed through replay_cycle() with the same strategy constants must match
field-for-field (matched=True, mismatch_field=None). Each round-trip test
produces the "archive" by running the real evaluate() with FakeNews, then
replays the drained record as if it had been read back from the signals table.
"""
import asyncio
from datetime import datetime, timedelta, timezone
import pytest
import bot.strategy.bayesian as bayesian
from bot.data.polymarket import Market, market_family_key
from bot.strategy.bayesian import BayesianStrategy, _days_to_resolution
from bot.replay import (
ReplayNews,
build_ext,
build_market,
replay_cycle,
strategy_config_hash,
)
from tests.test_news_guardrail import FakeNews, _sentiment_for
def _end_date(days_ahead: int = 20) -> str:
dt = datetime.now(timezone.utc) + timedelta(days=days_ahead)
return dt.strftime("%Y-%m-%dT00:00:00Z")
def _make_market(
yes_price: float,
question: str = "Will John Smith win the election?",
category: str = "politics",
market_id: str = "mkt-replay-1",
end_date: str = None,
) -> Market:
return Market(
id=market_id,
condition_id="cond-replay-1",
question=question,
yes_token_id="yes-tok",
no_token_id="no-tok",
yes_price=yes_price,
no_price=1.0 - yes_price,
volume_24h=50_000.0,
end_date=end_date if end_date is not None else _end_date(),
active=True,
category=category,
)
def _snapshot(valid: bool = True) -> dict:
"""An ext_snapshots row as read back from the DB."""
return {
"btc_price": 100_000.0,
"btc_change_24h": 0.0,
"eth_price": 4_000.0,
"eth_change_24h": 0.0,
"btc_dominance": 50.0,
"fear_greed_index": 50,
"fear_greed_label": "neutral",
"total_market_cap_change": 0.0,
"valid": valid,
}
def _market_row(market: Market) -> dict:
"""A markets-table row for the given Market."""
return {
"id": market.id,
"condition_id": market.condition_id,
"question": market.question,
"category": market.category,
"end_date": market.end_date,
}
def _record_with_live_evaluate(
market: Market,
news=None,
families: set = frozenset(),
) -> dict:
"""Run the real evaluate() and return the R0 record it produced —
the same dict save_signal_records() would have archived."""
strategy = BayesianStrategy(news=news, manifold=None, db=None)
asyncio.run(strategy.evaluate(market, build_ext(_snapshot()), set(families)))
return strategy.drain_cycle_records()[0]
def _replay_one(record: dict, market: Market, snapshot: dict = None) -> dict:
cycle_ts = datetime.now(timezone.utc)
decisions = asyncio.run(replay_cycle(
cycle_ts,
snapshot or _snapshot(),
[record],
{market.id: _market_row(market)},
))
assert len(decisions) == 1
return decisions[0]
# ─────────────────────────────────────────────────────────────────────────────
# Clock injection
# ─────────────────────────────────────────────────────────────────────────────
def test_days_to_resolution_uses_injected_clock():
end = "2026-08-01T00:00:00Z"
as_of = datetime(2026, 7, 2, 12, 0, tzinfo=timezone.utc)
assert _days_to_resolution(end, as_of) == 29
assert _days_to_resolution(end, as_of - timedelta(days=60)) == 89
def test_default_clock_is_wall_clock():
end = _end_date(days_ahead=40)
assert _days_to_resolution(end) == _days_to_resolution(
end, datetime.now(timezone.utc)
)
def test_as_of_changes_regime_threshold():
"""Same politics market: <30 d out → regime 0.08; replayed from 60 d
earlier regime 0.12. The clock, not the wall time, must decide."""
market = _make_market(0.470)
sentiment = _sentiment_for(0.470, 0.601)
def _regime(as_of):
strategy = BayesianStrategy(news=FakeNews(sentiment), manifold=None, db=None)
asyncio.run(strategy.evaluate(
market, build_ext(_snapshot()), set(), as_of=as_of,
))
return strategy.drain_cycle_records()[0]["regime_min_edge"]
now = datetime.now(timezone.utc)
assert _regime(now) == pytest.approx(0.08)
assert _regime(now - timedelta(days=60)) == pytest.approx(0.12)
# ─────────────────────────────────────────────────────────────────────────────
# Round-trip fidelity: record with live evaluate(), replay, expect match
# ─────────────────────────────────────────────────────────────────────────────
def test_roundtrip_confidence_skip():
"""Georgia signature: edge passes, confidence blocks — full-field match."""
sentiment = _sentiment_for(0.470, 0.601)
market = _make_market(0.470)
record = _record_with_live_evaluate(market, news=FakeNews(sentiment))
assert record["skip_reason"] == "confidence"
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["mismatch_field"] is None
assert decision["skip_reason"] == "confidence"
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
assert decision["edge_net"] == pytest.approx(record["edge_net"])
assert decision["confidence"] == pytest.approx(record["confidence"])
assert decision["direction"] == record["direction"]
assert decision["would_trade"] is False
def test_roundtrip_edge_net_skip():
market = _make_market(0.50)
record = _record_with_live_evaluate(market)
assert record["skip_reason"] == "edge_net"
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["would_trade"] is False
def test_roundtrip_guardrail_clamp():
"""Clamped posterior must reproduce exactly (raw != final in archive)."""
market = _make_market(0.845)
record = _record_with_live_evaluate(
market, news=FakeNews(_sentiment_for(0.845, 0.431))
)
assert record["guardrail_applied"] is True
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["raw_final_prob"] == pytest.approx(record["raw_final_prob"])
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
def test_roundtrip_prior_extreme():
market = _make_market(0.03)
record = _record_with_live_evaluate(market)
assert record["skip_reason"] == "prior_extreme"
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["skip_reason"] == "prior_extreme"
def test_roundtrip_family_skip():
"""Family-skipped rows replay with their own family injected as occupied."""
market = _make_market(0.50)
record = _record_with_live_evaluate(
market, families={market_family_key(market)}
)
assert record["skip_reason"] == "family"
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["skip_reason"] == "family"
def test_roundtrip_unsupported():
market = _make_market(0.50, question="Will it rain tomorrow?", category="")
record = _record_with_live_evaluate(market)
assert record["skip_reason"] == "unsupported"
decision = _replay_one(record, market)
assert decision["matched"] is True
def test_roundtrip_no_signals():
"""ext.valid=False archived → replay rebuilds the invalid snapshot."""
market = _make_market(0.50)
strategy = BayesianStrategy(news=None, manifold=None, db=None)
asyncio.run(strategy.evaluate(market, build_ext(_snapshot(valid=False)), set()))
record = strategy.drain_cycle_records()[0]
assert record["skip_reason"] == "no_signals"
decision = _replay_one(record, market, snapshot=_snapshot(valid=False))
assert decision["matched"] is True
def test_roundtrip_trade_path(monkeypatch):
"""skip_reason=None (tradeable) round-trips with would_trade=True.
Politics can't clear MIN_CONFIDENCE=0.55 (the known ceiling), so the
gate is lowered for this test only both record and replay see the
same constant, which is exactly the config_hash contract."""
monkeypatch.setattr(bayesian, "MIN_CONFIDENCE", 0.45)
sentiment = _sentiment_for(0.470, 0.601)
market = _make_market(0.470)
record = _record_with_live_evaluate(market, news=FakeNews(sentiment))
assert record["skip_reason"] is None
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["skip_reason"] is None
assert decision["would_trade"] is True
assert decision["direction"] == "BUY_YES"
# ─────────────────────────────────────────────────────────────────────────────
# Replay-specific semantics
# ─────────────────────────────────────────────────────────────────────────────
def test_budget_skipped_row_replays_without_news():
"""A budget-skipped archive row (sentiment 0.0) must replay to the same
no-news decision and never consume a replay-side budget."""
market = _make_market(0.50)
strategy = BayesianStrategy(news=FakeNews(0.9), manifold=None, db=None)
strategy._news_queries_this_cycle = bayesian.MAX_NEWS_QUERIES_PER_CYCLE
asyncio.run(strategy.evaluate(market, build_ext(_snapshot()), set()))
record = strategy.drain_cycle_records()[0]
assert record["news_budget_skipped"] is True
assert record["news_sentiment"] == 0.0
decision = _replay_one(record, market)
assert decision["matched"] is True
assert decision["estimated_prob"] == pytest.approx(record["estimated_prob"])
def test_reentry_guard_row_is_recalibrated_not_compared():
"""record_skip() rows carry no decision fields; the replay re-evaluates
them (calibration data) but marks them non-comparable."""
market = _make_market(0.50)
strategy = BayesianStrategy(news=None, manifold=None, db=None)
strategy.record_skip(market, "reentry_guard")
record = strategy.drain_cycle_records()[0]
decision = _replay_one(record, market)
assert decision["matched"] is None
assert decision["recorded_skip_reason"] == "reentry_guard"
# Re-evaluated on its merits: a full decision despite the recorded skip
assert decision["estimated_prob"] is not None
assert decision["skip_reason"] == "edge_net"
def test_missing_market_row_flagged_not_crashed():
market = _make_market(0.50)
record = _record_with_live_evaluate(market)
decisions = asyncio.run(replay_cycle(
datetime.now(timezone.utc), _snapshot(), [record], {},
))
assert decisions[0]["matched"] is False
assert decisions[0]["mismatch_field"] == "market_missing"
def test_mismatch_detected_when_config_differs(monkeypatch):
"""Counterfactual sanity: replaying under a different guardrail band
must produce matched=False with the diverging field named."""
market = _make_market(0.845)
record = _record_with_live_evaluate(
market, news=FakeNews(_sentiment_for(0.845, 0.431))
)
assert record["guardrail_applied"] is True
monkeypatch.setattr(bayesian, "MAX_NEWS_ONLY_PROB_SHIFT", 0.10)
decision = _replay_one(record, market)
assert decision["matched"] is False
# Tighter clamp (prior 0.845 ± 0.10 → est 0.745): edge_net drops from
# 0.21 to 0.06 < regime 0.08, so the skip flips confidence → edge_net
# and skip_reason is the first field _compare() sees diverge.
assert decision["mismatch_field"] == "skip_reason"
assert decision["skip_reason"] == "edge_net"
def test_multi_row_cycle_preserves_order_and_isolation():
"""Rows replay independently within a cycle: a family skip and a full
evaluation with different sentiments don't bleed into each other."""
m1 = _make_market(0.470, market_id="m1")
m2 = _make_market(
0.50, market_id="m2",
question="Will Jane Doe win the Georgia Senate race?",
)
r1 = _record_with_live_evaluate(m1, news=FakeNews(_sentiment_for(0.470, 0.601)))
r2 = _record_with_live_evaluate(m2) # no news → edge_net skip
decisions = asyncio.run(replay_cycle(
datetime.now(timezone.utc),
_snapshot(),
[r1, r2],
{"m1": _market_row(m1), "m2": _market_row(m2)},
))
assert [d["market_id"] for d in decisions] == ["m1", "m2"]
assert all(d["matched"] is True for d in decisions)
assert decisions[0]["skip_reason"] == "confidence"
assert decisions[1]["skip_reason"] == "edge_net"
# ─────────────────────────────────────────────────────────────────────────────
# Run tagging
# ─────────────────────────────────────────────────────────────────────────────
def test_config_hash_stable_and_sensitive(monkeypatch):
h1 = strategy_config_hash()
assert strategy_config_hash() == h1
monkeypatch.setattr(bayesian, "MAX_NEWS_ONLY_PROB_SHIFT", 0.10)
assert strategy_config_hash() != h1
def test_replay_news_returns_current_sentiment():
news = ReplayNews()
assert asyncio.run(news.get_sentiment("q")) == 0.0
news.sentiment = -0.42
assert asyncio.run(news.get_sentiment("q")) == -0.42
def test_build_market_reconstruction():
market = _make_market(0.37)
record = _record_with_live_evaluate(market)
rebuilt = build_market(_market_row(market), record)
assert rebuilt.id == market.id
assert rebuilt.yes_price == pytest.approx(0.37)
assert rebuilt.volume_24h == pytest.approx(market.volume_24h)
assert rebuilt.end_date == market.end_date
assert rebuilt.category == "politics"
assert market_family_key(rebuilt) == market_family_key(market)