Scores every archived estimate against reality — the sample multiplier
the phase plan calls for: Brier/log-loss of estimated_prob benchmarked
against the market price (prior_prob) on the same rows, over ALL
evaluations with a resolved outcome, not just executed trades.
- schema.sql: market_outcomes (one row per resolved market; outcome =
final YES price 1.0/0.0, UMA-final only)
- bot/outcomes.py: CLI (python -m bot.outcomes) with two phases —
fetch resolutions for archived markets via the existing
get_market_resolution() (open/disputed/ambiguous markets simply retry
next invocation; no data-loss urgency, Gamma reports past resolutions
at any time), then compute calibration: Brier micro (per evaluation) /
macro (per market — the honest sample size given ~1 eval/min
autocorrelation), log-loss with 1e-9 clipping, per-category breakdown.
--run-id scores a replay run's re-estimates instead of the archive
(counterfactual calibration).
- db.py: 4 accessors (pending markets, outcome upsert, coverage,
calibration rows for archive or run)
- tests: 12 new (116 total green); compute_calibration is a pure
function driven by literals
No prod behavior change: the bot never imports bot.outcomes; the only
shared surface is the idempotent schema migration (dry-run BEGIN/ROLLBACK
clean against prod).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
