The outcomes-joiner CronJob (k8s-manifests, Replay R2) runs the same bot
image; without this its tag would freeze at the sha it was created with
while the deployment moves on. Same sed, one more file.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scores every archived estimate against reality — the sample multiplier
the phase plan calls for: Brier/log-loss of estimated_prob benchmarked
against the market price (prior_prob) on the same rows, over ALL
evaluations with a resolved outcome, not just executed trades.
- schema.sql: market_outcomes (one row per resolved market; outcome =
final YES price 1.0/0.0, UMA-final only)
- bot/outcomes.py: CLI (python -m bot.outcomes) with two phases —
fetch resolutions for archived markets via the existing
get_market_resolution() (open/disputed/ambiguous markets simply retry
next invocation; no data-loss urgency, Gamma reports past resolutions
at any time), then compute calibration: Brier micro (per evaluation) /
macro (per market — the honest sample size given ~1 eval/min
autocorrelation), log-loss with 1e-9 clipping, per-category breakdown.
--run-id scores a replay run's re-estimates instead of the archive
(counterfactual calibration).
- db.py: 4 accessors (pending markets, outcome upsert, coverage,
calibration rows for archive or run)
- tests: 12 new (116 total green); compute_calibration is a pure
function driven by literals
No prod behavior change: the bot never imports bot.outcomes; the only
shared surface is the idempotent schema migration (dry-run BEGIN/ROLLBACK
clean against prod).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The signals and markets tables existed since Phase 2/5 but never had a
writer; the replay engine (phase plan line 2.1) needs a per-(market, cycle)
archive of what the strategy saw and decided. This wires them up:
- signals: one row per evaluated market per cycle, now carrying INPUTS
(news_sentiment, feat_*_lo, volume_24h, days_to_resolution) plus the
existing outputs (probs, edges, gates, skip_reason). skip_reason is
granular: unsupported/no_signals/prior_extreme/family/edge_net/
confidence/reentry_guard. news_budget_skipped distinguishes "GNews not
asked" (5-query budget) from "no news".
- ext_snapshots: one row per cycle with the ExternalSignals snapshot;
signals rows join on cycle_ts.
- markets: metadata upserted each cycle (replay rebuilds Market from it).
- Retention: prune > SIGNALS_RETENTION_DAYS (default 90) once a day.
- SIGNAL_RECORDER_ENABLED (default true) gates all DB writes; every write
is try/except — the recorder can never break trading.
Strategy changes are purely additive (record accumulation at each exit
path of evaluate()); no weights, thresholds, gates or sizing touched,
per the freeze in the current phase plan.
Tests: 10 new deterministic tests (85 total passing). Schema migration
dry-run validated against prod postgres inside a rolled-back transaction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-mortem NVIDIA 631181: one uncorroborated high-weight signal (legacy
Manifold 0.13 at weight 0.6) flipped a 0.845 market to 0.431 and lost.
With Manifold observational-only and macro signals gated behind
is_non_price, GNews (weight 1.5) is the only live signal able to move
politics markets 20-30 pp against the order-book consensus. This adds a
catastrophic fuse, not a fine calibration:
- apply_news_guardrail(): when |news_lo| >= NEWS_MATERIAL_LOGODDS_THRESHOLD
(0.10) and every other signal (fg, mom, btc_dom, mfld) is below it,
clamp the posterior to prior ± MAX_NEWS_ONLY_PROB_SHIFT (0.25). Any
corroborating material signal disables the clamp. Config via env
(NEWS_GUARDRAIL_ENABLED=true by default).
- edge_gross/edge_net computed from the clamped posterior; raw_final_prob
preserved in reasoning (persisted via trades.reasoning — no schema
migration) and in the NEWS_MATERIAL log line.
- guardrail_changed_trade_decision: raw edge crossed the regime gate but
the clamped edge no longer does (fuse prevented a trade). Note: with
the default 0.25 band the clamped edge_net is 0.21, above every regime
minimum, so the flag only fires with a tighter configured band.
- Observability gated on materiality: NEWS_MATERIAL per-market line and a
compact NEWS SUMMARY cycle line, only when with_news > 0 — no flood
from the ~145 news-less markets per cycle.
- 9 deterministic tests (extreme clamp, in-band passthrough, corroboration,
inclusive threshold, disabled, changed_decision).
No changes to NEWS_LOGODDS_WEIGHT, Manifold flags, edge thresholds,
sizing, payout, resolution, or historical trades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
httpx logs every request URL at INFO level, and the GNews search URL
carries the API key as a `?token=` query param, so GNEWS_API_KEY was
written in plaintext into the pod logs on every news query. Raise the
httpx/httpcore loggers to WARNING so request URLs never reach INFO.
The bot's own GNews log lines only print the sanitised keyword query
(NewsClient._build_query), never the token, so they are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two minor faults found during the GNews capture/prioritisation diagnostic:
1. Hyphens/dashes reached the GNews query verbatim. '-' is GNews's exclusion
operator, so a token like "El-Sayed" returned HTTP 400 and wasted a query.
_PUNCT_RE now strips '-', en dash and em dash to spaces.
2. The per-cycle GNews budget counter incremented in evaluate() before
get_sentiment() checked the API key, so with no key configured the
[CYCLE SUMMARY] reported a phantom "gnews_queries_used: 5/5" with zero real
requests. Added NewsClient.enabled and gated the GNews block on it; with no
key the counter stays 0/5 and no spurious SKIP_GNEWS_PRIORITY is logged.
No behaviour change when a key is present.
Prioritisation itself was confirmed correct and is left untouched: politics
markets are sorted by gnews_priority DESC and prior-extreme markets return
before the budget is consumed, so no query is ever spent on a market that
cannot trade.
Tests: tests/test_news_query.py (4 new); full suite 66 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Path-based build selection diffed against github.event.before:
- bot/, api/, requirements.txt -> bot + api images (both COPY the same
python sources; only the CMD differs)
- Dockerfile -> bot only; Dockerfile.api -> api only
- dashboard/ -> dashboard only
- .gitea/workflows/ci.yml, first push or force push -> all (safe fallback)
- anything else (tests/, docs) -> no builds, no manifest update
The k8s-manifests sed only bumps tags of rebuilt images, so unchanged
deployments keep their current tag and don't restart. Registry login,
buildx, verification and manifest update are all skipped when nothing
needs building. Telegram message now lists what was built.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sharpe_ratio was hardcoded to 0.0 in MetricsTracker and exposed as
'or 0' in /api/summary. With only 1 resolved trade (~40 flat days plus
one +299 jump) any computed Sharpe is statistically meaningless, so:
- bot/metrics/sharpe.py: annualized Sharpe (sqrt(365)) from daily
total_pnl closes, normalized by bankroll; sharpe_with_gate() returns
None + status until >=30 days observed AND >=10 resolved trades.
- Database.get_daily_pnl_closes(): last metrics_daily snapshot per UTC
day, oldest first — the return series input.
- MetricsTracker: stores the real (gated) Sharpe in the snapshot, NULL
below the gate; log line now includes sharpe.
- /api/summary: live Sharpe + sharpe_status/days_observed/min_* fields
explaining why it is null; resolved_count now live from COUNT(*).
- promotion_ready: requires resolved>=10, days>=30, and non-null
win_rate/calibration/sharpe plus existing thresholds — a single lucky
resolved trade can no longer promote.
- Dashboard Sharpe card shows the insufficient-sample explanation when
null instead of a bare em dash.
Tests: 13 new in tests/test_sharpe_gate.py (formula, gate, API contract,
tracker snapshot); verified failing pre-fix. Suite: 62 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shows /api/summary cash_available (now consistent with the executor's cash
model) next to Capital Deployed, with its share of bankroll as subtitle and
progress bar.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bug #5: metrics.record_trade() only delegated to save_trade(), which
executor.execute() already calls — every trade was written twice (deduped
only by ON CONFLICT DO NOTHING). Remove the redundant call and the now-dead
method. RealExecutor.execute() raises NotImplementedError, so real mode is
unaffected.
Bug #6 (CYCLE SUMMARY): manifold accepted/rejected counters only increment
on the active-signal path, so with MANIFOLD_SIGNAL_ENABLED=false they always
printed 0/0 — print 'manifold_signal: disabled' instead.
family_conflicts_prevented duplicated blocked_by_family (same counter
printed twice); removed. gnews_cap was a dead variable with a misleading
comment; removed.
Bug #7 (/api/summary): total_trades was len() over a LIMIT-500 query —
capped once history grows; counts now come from COUNT(*) via
compute_metrics_from_db. cash_available was reimplemented in the API;
extract cash_available() in paper.py (same formula, unchanged) and feed it
from get_open_position_data() — the exact source/helper
PaperExecutor.initialize() uses. Test asserts API and executor report
identical cash for the same DB state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
asyncio.create_task() results were discarded, and the event loop only holds
a weak reference to running tasks — a pending notification could be
garbage-collected before executing, silently dropping Telegram messages
(documented asyncio pitfall).
Route the three notification call sites (trade_opened, trade_legacy_closed,
trade_closed) through _notify_in_background(), which stores the task in a
module-level set and discards it on completion. Notifications stay
fire-and-forget; no business logic changed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 3 excluded momentum and Fear & Greed from politics/tech/events
markets; Phase 4 fixed ticker detection. But the BTC-dominance signal was
still applied to non-price markets that legitimately mention a ticker
('Will the ETH ETF be approved?'), despite having no demonstrated causality
for non-price outcomes. Reuse the existing is_non_price gate so the
contribution stays 0.0 -> feat_btc_dom_lo = 0.0 for those markets.
Price-market behavior unchanged: ETH/altcoin/general-crypto markets keep
the +/-0.03 dominance adjustment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Substring matching over question_lower flagged non-crypto markets as crypto:
'dissolved' matched 'sol', 'Canada' matched 'ada', 'Seth' matched 'eth'.
Those false flags armed the BTC-dominance signal (btc_dom_lo=+0.06 observed
on politics markets in production).
Short tickers (btc, eth, sol, xrp, doge, ltc, bnb, ada, avax) now go through
has_token(), which requires non-alphanumeric boundaries so 'ETH', '$ETH' and
'ETH/USD' still match. Long unambiguous names (bitcoin, ethereum, solana,
cardano, ...) remain substring checks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
For politics/tech/events markets there is no above/below price notion, so
is_price_above defaulted to False (or flipped on accidental wording like
"reach") and sign-inverted the macro adjustments: BTC +5% or high Fear&Greed
subtracted probability from YES on "Will X win the election?" markets.
Skip both signals entirely for non-price markets: contributions stay 0.0,
feat_mom_lo / feat_fg_lo persist as 0.0. Price markets (BTC/ETH/crypto)
keep the exact current behavior, including the below-market sign flip.
Removes the now-dead BTC(sentiment) momentum branch and its 0.5 attenuator.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
resolved_count shared the final_prob IS NOT NULL filter with
calibration_score, so the resolved legacy Paxton trade (no signal data)
didn't count: realized_pnl=+309.06 and wins_realized=1 but resolved_count=0.
resolved_count now only requires resolution + not excluded; calibration
keeps the final_prob requirement since it scores against the estimate.
Validated against prod DB: new filter returns 1, old returned 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cycle-10 resolution check found market 562186 resolved (Paxton YES) but the
close failed with asyncpg AmbiguousParameterError: Postgres cannot infer the
type of a bare '$3 IS NOT NULL' in the close_pnl CASE. Reproduced via PREPARE
in the postgres pod; fixed by casting every $3 use to double precision.
The failed DB write also left memory/DB diverged: close_position() popped the
position and credited cash before persisting, so the retry at cycle 20 skipped
the market (pnl=n/a) while the DB row stayed open. Now the DB write happens
first and memory mutates only on success; check_resolutions() also isolates
per-market close failures so one error doesn't abort the cycle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- PolymarketClient.get_market_resolution(): query Gamma API by market id;
resolved only when closed AND uma status final AND outcome prices binary
(never settle on disputed/ambiguous outcomes)
- bot/main.py: check_resolutions() runs every 10 cycles (~10 min) in paper
mode, settles open positions via PaperExecutor.close_position()
- close_reason now persisted as 'resolved' (resolution has its own column)
- tests/test_resolution_detector.py: 10 tests covering API parsing shapes
and the BUY_NO settlement flow; 27/27 suite green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Legacy scan called ManifoldClient.get_probability(), removed in the v3
matcher migration, causing AttributeError when positions had changed
family keys. The block used Manifold to escalate positions to
CLOSE_RECOMMENDED (inversion detection) — a trading decision forbidden
under MANIFOLD_SIGNAL_ENABLED=false — so the dependency is removed
entirely; the scan keeps family re-keying and sibling-conflict logic.
- PaperExecutor.close_position() computed cash += position_cost * resolution,
ignoring direction: a winning BUY_NO (resolution=0.0) paid out $0 and
reported a loss. Now settles per trade:
BUY_YES: payout = shares * resolution
BUY_NO: payout = shares * (1 - resolution)
with pnl = payout - net_cost; Telegram win/loss keys off pnl > 0.
Adds read-only Database.get_open_trades_for_market().
- tests/test_paper_close.py covers the 4 deterministic payout cases;
tests/conftest.py shims datetime.UTC for local Python 3.10 (prod is 3.11).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-category coverage audit showed coverage_rate=0.0 across every category in the
bot's current universe, so any edge Manifold produced was false edge. Retire it as
an ACTIVE trading signal while keeping the full audit/coverage/cooldown trail for a
future reactivation decision.
Two module-level flags in bayesian.py (read from env):
- MANIFOLD_SIGNAL_ENABLED (default False): when False, Manifold never touches the
edge model — manifold_log_adj stays 0.0 (no posterior shift), no confidence bump,
feat_mfld_lo=0.0 (so it can never be the dominant feature), no trade contribution,
and mfld_audit_id is not propagated so the audit's used_in_trade stays FALSE.
- MANIFOLD_AUDIT_ENABLED (default True): matcher still runs; audit/coverage rows and
cooldowns are still written. The matcher is only called when a flag is on.
When signal is disabled, logs and reasoning carry "Manifold: observational_only".
Endpoints /api/metrics/manifold-matches and /api/metrics/manifold-coverage,
cooldowns, audit tables and existing trades are unchanged. No code or tables removed.
Other signals, thresholds, exposure, risk manager and the executor are untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measure real Manifold coverage per semantic market category counted by UNIQUE
market (not audit rows, which are inflated by repeated re-evaluation). Base table
is manifold_match_audit filtered to v3_outcome_guard; each poly_market_id is
collapsed to one row, LEFT JOIN trades for family_key, category inferred from
family_key when present else from poly_question (gubernatorial / mayoral / senate
/ primary-republican / primary-democrat / big-tech / geopolitics / other).
Per category: unique_evaluated / unique_accepted / unique_rejected /
unique_no_results / coverage_rate, ordered by unique_evaluated DESC. Summary adds
total_unique_evaluated, total_unique_accepted, overall_coverage_rate,
categories_with_coverage. Read-only: new db method + endpoint only; matcher,
scheduler, cooldowns, thresholds and exposure untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trading loop re-evaluated the same ~22 politics/tech markets every ~60s,
flooding manifold_match_audit with ~76k rows (~3,500 attempts/market) of which
none carried new information, making the metrics uninterpretable.
Add per-market persistent cooldowns:
- New table manifold_eval_cooldown (poly_market_id PK, last_evaluated_at,
last_status, retry_after, cooldown_reason) created via run_migrations.
- bayesian.evaluate() consults the cooldown BEFORE calling the matcher and skips
the call entirely while now() < retry_after — no matcher call, no audit row,
no signal (equivalent to no_results). After a real evaluation it upserts the
cooldown with a verdict-dependent backoff: no_results/low_score/outcome_mismatch/
ambiguous_inversion -> 24h, conditional_market -> 7d, accepted -> 1h.
- manifold.py stays a pure client/matcher with no DB access.
- New db methods get_manifold_cooldown / upsert_manifold_cooldown.
- /api/metrics/manifold-matches summary gains unique_markets
{evaluated, accepted, coverage_rate} for per-market (not per-attempt) coverage.
Matching logic, MANIFOLD_MATCHER_VERSION, MANIFOLD_LOGODDS_WEIGHT, edge/exposure
thresholds and existing audit rows are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trades_dominated_by_mfld counter omitted the excluded_from_metrics
filter, so the admin-closed Maine governor trade inflated it to 1 while
attribution/features (which exclude such trades) were empty.
Add excluded_from_metrics IS NOT TRUE and mfld_match_status = 'accepted'
to the query so the counter is consistent with the attribution and
feature-metrics endpoints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add MANIFOLD_MATCHER_VERSION="v3_outcome_guard" tag persisted to
manifold_match_audit.matcher_version so metrics can isolate current-matcher
stats from pre-versioning records, whose accepted matches the outcome
guard would now reject.
- schema: add matcher_version column + index; idempotent startup backfill
tagging NULL rows as legacy_pre_outcome_guard (no outcome types) or
v2_outcome_guard_no_version (has outcome type, version not persisted)
- save_manifold_audit: write matcher_version on every new record
- get_manifold_matches: split summary into current_version / all_time /
legacy; recent_matches now carry matcher_version
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reject false-positive matches where Jaccard overlap is high but the outcome is
not equivalent (e.g. Poly nomination vs Manifold "If X is nominee, will he win").
- _is_conditional(): detect conditional Manifold markets (If/Conditional on/
Assuming/Given that prefixes + mid-sentence " if ...," clauses) -> reject with
reason "conditional_market".
- _classify_outcome(): classify into nomination|primary_win|general_win|
conditional|other; reject when poly/mfld types differ or either is conditional
-> reason "outcome_mismatch: poly=... manifold=...".
- Persist poly_outcome_type/mfld_outcome_type on ManifoldMatchResult, in
manifold_match_audit (CREATE + idempotent ALTER), save_manifold_audit() and
the bayesian call site.
- Tests covering classification, conditional detection and the Graham Platner
regression (now rejected); valid nomination<->nomination still accepted.
Untouched: _MATCH_THRESHOLD (0.40), MANIFOLD_LOGODDS_WEIGHT, edge thresholds,
exposure, trading logic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New module bot/notify/telegram.py — httpx async, fire-and-forget via
asyncio.create_task, swallows all errors so notifications never affect
trade execution.
Three alert types:
📈/📉 TRADE ABIERTO — direction, size, edge_net (in execute())
✅/❌ GANADO/PERDIDO — approx PnL (in close_position())
🔒 LEGACY CLOSE — recovered capital + reason (in close_legacy_position())
close_position() and close_legacy_position() gain an optional question=""
param so the message shows the market name instead of market_id.
bot/main.py updated to pass question= to close_legacy_position().
Credentials (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID) read from env vars
injected via bot-secrets k8s secret.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Verify all 3 images exist in Gitea registry via Docker API before updating manifests
- Validate YAML of modified manifests after sed (python3 yaml.safe_load)
- Notify Telegram on success/failure with job status (if: always())
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Legacy builder (DOCKER_BUILDKIT=0) cannot handle OCI image indexes from
registry-cache, causing fallback to Docker Hub which is unreachable.
BuildKit sends proper OCI Accept headers and reads buildkitd.toml to use
HTTP for both the registry-cache mirror and internal Gitea registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Docker in DinD cannot reach git.chemavx.xyz (Cloudflare) from within
the cluster — TCP :443 times out. Switch docker login/build/push to
gitea.gitea.svc.cluster.local:3000 (insecure, same backend storage).
k8s manifest updates still reference git.chemavx.xyz for node pulls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Docker 24's embedded BuildKit ignores the http:// prefix in registry-mirrors
and always attempts HTTPS, breaking the local pull-through cache.
DOCKER_BUILDKIT=0 uses the legacy builder which respects the daemon mirror
config correctly. Cache still works via --cache-from + buildcache tag.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The docker-container buildkitd driver creates an isolated process that
cannot use DinD's registry-mirror config, and the cluster's registry-cache
returns 500 on BuildKit's ?ns=docker.io mirror protocol.
Plain docker build routes through the DinD daemon directly, which already
has registry-mirrors configured for docker.io pull-through cache.
Uses BUILDKIT_INLINE_CACHE=1 for layer caching between builds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BuildKit docker-container driver runs isolated from DinD daemon config,
so it needs its own mirror declaration to route docker.io pulls through
the cluster-local pull-through cache instead of Cloudflare CDN directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds alpha attribution by dominant signal feature — which feat_*_lo had
the largest absolute log-odds value on each trade.
Changes:
- _dominant_feature() helper in api/main.py: picks the winning feature
from signal_components (threshold 0.0001, same as "triggered" in
/api/metrics/features)
- _enrich_trade() refactored to single exit point; adds dominant_feature
field to every open trade in /api/trades
- compute_attribution_from_db() in db.py: VALUES subquery finds dominant
feature per trade in SQL, then aggregates trade_count/avg_edge_net/
unrealized_pnl_est/realized_pnl/resolved_count/win_rate per group
- /api/metrics/attribution endpoint: returns attribution dict + total_attributed_trades
No schema changes, no strategy changes. Pure observability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eliminates the phantom 0.4% exposure overage after pod restarts.
During live trading execute() stores size_usdc in portfolio.positions and
deducts net_cost from cash — so total_value = bankroll − fees and
exposure_pct = sum(size_usdc) / (bankroll − fees).
Old initialize() stored net_cost in positions, making total_value = bankroll
and inflating exposure_pct (observed: 30.085% vs runtime 29.670%).
Fix: new get_open_position_data() returns both {market_id: size_usdc} and
total_net_cost in one query; initialize() uses size_usdc for positions and
total_net_cost for cash — identical model to execute().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds feat_fg_lo / feat_mom_lo / feat_news_lo / feat_mfld_lo / feat_btc_dom_lo
to every trade, all normalized to log-odds contribution for direct comparability.
- fg / mom / btc_dom: raw probability-delta × 2 → log-odds
- news / mfld: already log-odds (LOGODDS_WEIGHT already applied), no scaling
- btc_dom tracked separately in bayesian.py instead of bundled in total_adj
- reasoning string updated to fg_lo= / mom_lo= notation for self-documentation
Schema: 5 new DOUBLE PRECISION columns + 2 partial indexes
Stack: TradingSignal → Order → Trade → save_trade all carry feat fields
Startup: backfill_feature_columns() recovers fg/mom/news/mfld from old
reasoning strings (×2 applied to fg/mom); btc_dom_lo stays NULL for legacy
API: /api/metrics/features — triggered/material split per feature with
two-level thresholds (0.05 for fg/mom/btc_dom, 0.10 for news/mfld)
API: /api/trades/legacy — exposes pre-Phase-1 trades (edge_net IS NULL)
API: _enrich_trade backward-compat: reads DB columns first, falls back to
reasoning regex with unit conversion for pre-Phase-6 trades
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
schema.sql
trades: + close_pnl, resolution (market outcome storage)
metrics_daily: + unrealized_pnl_est, realized_pnl, open/closed/resolved_count
db.py
close_paper_position(): accepts resolution; computes close_pnl in SQL
BUY_YES: (resolution − entry_price) × shares
BUY_NO: ((1 − resolution) − entry_price) × shares
save_daily_metrics(): persists new columns
compute_metrics_from_db(): single DB query for all metrics; no in-memory state
tracker.py — complete rewrite (stateless)
Removed self._trades, self._daily_returns, compute_metrics(), _compute_sharpe(),
check_promotion_thresholds(), _empty_metrics()
update_daily_summary() now reads compute_metrics_from_db() every cycle
Safe across pod restarts: always reflects full DB history
paper.py
close_position(): passes resolution to close_paper_position()
api/main.py /api/summary
Added unrealized_pnl_est (estimated, open trades) and realized_pnl (exact,
closed+resolved) as separate fields alongside total_pnl
win_rate: null if < 5 resolved trades (was proxy on entry_price < 0.5)
calibration_score: Brier-based, null if < 10 resolved trades
resolved_count exposed as field
Each field annotated with: exact/estimated, source, null conditions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kaniko has its own HTTP client and ignores the dind --registry-mirror flag.
Passing --registry-mirror=mirror.gcr.io directly to kaniko lets it pull
python:3.11-slim (and other Docker Hub base images) via Google's mirror
instead of the Cloudflare CDN endpoint (172.64.66.1) which is unreachable
from the runner's dind network.
Also adds mirror to dind deployment args (--registry-mirror=https://mirror.gcr.io)
so node:16-bullseye (the job runner container) is also fetched via mirror.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
total_pnl now uses edge_net × net_cost instead of (0.5 - entry_price) × shares.
The old formula overestimated BUY_NO trades at low entry prices by 3–10× because
buying at price 0.158 yields 3164 shares — any exit-at-0.5 assumption produced
$1072 PnL on $500 deployed. edge_net × net_cost is bounded by net_cost per trade
and uses the model's own signal, giving $122 for the same position.
calibration_score is now None (null in API) instead of 1 - 2×|avg_edge|. That
formula was not a real calibration: it requires knowing market resolutions
(YES=1/NO=0) which we do not store yet. Returning null is more honest than
returning 0.0 or a meaningless proxy. Fix 3 will compute it from closed trades.
check_promotion_thresholds updated to handle None calibration (null → not ready).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- db: update_family_key() persists corrected family slugs for open trades
- db: get_recently_closed_inverted() returns markets closed for inversion
within N hours; used as reentry guard in the trading loop
- db: get_recent_trades() accepts status=open|closed|None and adds a
computed "status" field to every row
- bot/main.py: legacy scan now computes family_key from stored question
alone (dummy Market) when a position's market is no longer active —
fixes NULL family_key on legacy trades like Ken Paxton (562186)
- bot/main.py: legacy scan (Step 2.5) persists corrected family_keys in
DB so family conflict guards work correctly on next restart
- bot/main.py: positions with NULL edge_net and no live market are tagged
legacy_incomplete instead of OK; counted separately in scan summary
- bot/main.py: reentry_guard blocks re-entering any market closed for
inversion bug within 24h; logs reentry_guard_triggered per skip
- api/main.py: /api/trades now accepts ?status=open|closed|all (default
open) and includes status_filter in response
DB fix (applied directly): 629558 family_key politics-2026 →
ohio-gubernatorial-2026; 562186 family_key NULL → texas-republican-2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds run_legacy_scan() that executes once at startup before the trading loop:
1. Re-keys every open DB position using the current market_family_key()
2. Groups by new family key; KEEP = highest edge_net, CLOSE_RECOMMENDED = sibling
3. Manifold re-query for positions whose family key changed; if corrected
probability contradicts the trade direction → CLOSE_RECOMMENDED
4. Logs full report (KEEP / REVIEW / CLOSE_RECOMMENDED) before any closures
5. In paper mode: auto-closes all CLOSE_RECOMMENDED positions
For the existing Ohio bug:
- Democrats win Ohio governor (629557): CLOSE_RECOMMENDED
family changed ohio-democrat-2026 → ohio-gubernatorial-2026
Manifold re-query confirms prob=0.05 contradicts BUY_YES (inversion bug)
$X returned to cash at break-even
- Republicans win Ohio governor (629558): KEEP
higher edge_net (0.349 > 0.247)
Infrastructure:
- schema.sql: closed_at TIMESTAMPTZ, close_reason TEXT on trades
- db.py: all open-position queries filter WHERE closed_at IS NULL
+ close_paper_position(market_id, reason)
- paper.py: close_legacy_position(market_id, reason) → float
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Republicans (plural) previously didn't match _PARTY_RE because the pattern
was r"\bRepublican\b" (no optional s). Added Republicans? for symmetry with
Democrats?. The general-election family fix already handles this case via
etype_m, but the plural match is needed for the party-only fallback branch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FASE 1 — market_family_key() general election fix
General elections now group by office, not by party, so complementary
markets ("Republicans win Ohio governor" / "Democrats win Ohio governor")
share the same family key (ohio-gubernatorial-2026). The second market
is blocked by the occupied_families check rather than traded as independent.
Primaries still keep the party (texas-republican-2026) because each party
runs its own separate primary race.
FASE 2 — Manifold party inversion guard
_detect_party() identifies the winning side in both the Polymarket question
and the matched Manifold title. If they are confirmed opposites (republican
vs democrat), the probability is inverted (1 - prob) before use.
Full audit log per query:
poly_question / manifold_title / manifold_url / match_score /
prob_raw / inverted / prob_final
Root cause of Ohio Manifold:0.95 on both sides: both queries matched the
same Manifold market ("Republicans win Ohio governor" prob=0.95). For the
"Democrats win" query the inversion now produces prob_final=0.05 instead of
blindly applying 0.95 to the wrong direction.
FASE 4 — startup contradiction scan
get_open_position_details() added to db.py. main.py checks all open
positions at startup, warns on any family with >1 position, and recommends
keeping the one with the highest edge_net. No auto-close.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signal 5: ManifoldClient queries Manifold Markets API for a matching binary
market by keyword overlap (threshold 0.25) and applies a log-odds adjustment
proportional to the divergence from the Polymarket prior.
manifold_log_adj = (log_odds(manifold_prob) - log_odds(prior)) × 0.6
A 30pp divergence (Manifold 0.75 vs Poly 0.45) produces edge_gross ≈ 0.19,
clearing the politics far-horizon regime_min=0.12 after costs. Confidence
boosted +0.08 when Manifold match found.
Per-feature observability: every SKIP_EDGE_NET and TRADE log line now includes
fg=±X.XXX mom=±X.XXX mfld=±X.XXXX news=±X.XXXX
so the contribution of each signal to edge is auditable per market.
Files: bot/data/manifold.py (new), bot/strategy/bayesian.py, bot/main.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 1 — Edge neto real (paper.py, bayesian.py, risk/manager.py, db.py):
- Trade records now store edge_gross, edge_net, prior_prob, final_prob,
mid_price, spread_estimate, commission, family_key
- edge_net = edge_gross - SPREAD_ESTIMATE(0.02) - COMMISSION_RATE(0.02)
NOTE: both constants are heuristics, not exact Polymarket exchange costs
- Execution gate changed from edge_gross > MIN_EDGE to edge_net > regime_min_edge
Phase 2 — Market families (polymarket.py):
- market_family_key(market) groups related markets:
texas-republican-2026, fed-april-2026, openai-2026, etc.
- At most 1 trade per family per cycle; occupied_families propagated via main.py
- Family key logged on every TRADE and SKIP line
Phase 3 — GNews priority (news.py, bayesian.py, main.py):
- NewsClient.get_freshness() returns 1.0/0.75/0.40/0.10 by cache age
- gnews_priority(market, news) = uncertainty × volume_score × freshness
- Politics markets sorted by priority DESC before eval so best markets get
the 5-query/cycle GNews budget first
Phase 4 — Regime min-edge by category/horizon (bayesian.py):
- politics >60d → 0.12, 30-60d → 0.10, <30d → 0.08
- tech / crypto/finance → 0.10
- All thresholds applied to edge_net (not edge_gross)
Phase 5 — Observability (bayesian.py, main.py):
- Structured skip labels: SKIP_UNSUPPORTED, SKIP_NO_SIGNALS,
SKIP_PRIOR_EXTREME, SKIP_FAMILY, SKIP_GNEWS_PRIORITY, SKIP_EDGE_NET
- TRADE lines now include family_key, edge_gross, edge_net, regime_min, days
- schema.sql: 8 new cols on trades, 7 new cols on signals (via ALTER TABLE IF NOT EXISTS)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add European football leagues (La Liga, Premier League, Bundesliga, etc.)
to _SPORTS_EXCLUSIONS so those markets are filtered before category detection
- Reorder _detect_category: check tech before crypto/finance so company-specific
markets (OpenAI IPO, NVIDIA, Apple) resolve to "tech" instead of "crypto/finance"
- Widen resolution horizon default from 60 to 90 days to surface more
markets in the 0.08–0.92 uncertainty zone
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reconstruct dashboard from compiled container: App.jsx, main.jsx, index.css
- nginx.conf with SPA routing and /api proxy to api:8000
- Multi-stage Dockerfile: node:20-alpine build + nginx:alpine serve
- Add third kaniko build step in ci.yml for chemavx/polymarket-bot-dashboard
- Update k8s manifest sed to patch deployment-dashboard.yaml image on each push
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add _SPORTS_EXCLUSIONS list checked first in _detect_category so NBA/NFL/
MLB/NHL/tennis/golf/UFC/boxing/wrestling/tournament markets never bleed into
politics or events categories. Also removes 'super bowl' from _EVENTS_KEYWORDS
since it's now covered by the sports exclusion.
Keywords excluded: nba, nfl, mlb, nhl, basketball, football, baseball, hockey,
soccer, mvp, rookie of the, championship, super bowl, world series, playoffs,
playoff, tournament, tennis, golf, ufc, boxing, wrestler, wrestling,
slam dunk, home run, touchdown.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Markets where Polymarket consensus is near-certain leave no room for our
signals to generate MIN_EDGE=0.10 — evaluating them wastes GNews quota and
produces noise. Filter them out early with a clear log reason.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
vaderSentiment==3.3.2 added to requirements.txt.
_score_headlines now:
- scores each article (title + description) with VADER compound ∈ [-1, +1]
- filters out articles with |compound| ≤ 0.05 (no clear signal)
- weights remaining articles by recency (GNews newest-first, rank 0 → highest weight)
- returns weighted mean clamped to [-1, +1]
Removes the custom keyword sets (_POSITIVE/_NEGATIVE) and the set-based
bag-of-words algorithm that capped scores at ~±0.5 in practice.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CACHE_TTL: 4h → 6h (≤36 req/day with ≤9 politics markets)
- GNews only called for is_politics markets (BTC/F&G cover crypto/macro)
- MAX_NEWS_QUERIES_PER_CYCLE=5: BayesianStrategy.reset_cycle() called each
iteration; counter increments only on actual API call (cache hits free)
- 2s asyncio.sleep in news.py finally block after each real HTTP request
- main.py sorts markets: politics first by end_date ascending, so soonest-
resolving markets consume the 5-query budget before others
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Drop the 'from' date filter — it's a paid GNews feature, causes 403 on free tier
- Add User-Agent header to httpx client; urllib default passes, httpx default blocked
- Log actual HTTP status code for every request (INFO) and response body on non-200
- Cache neutral result on 400/401/403/429 to avoid hammering the quota
- Remove unused _iso_days_ago() helper and 'days' param from get_sentiment()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bot/data/news.py (new):
- NewsClient with in-memory cache (TTL=4h) to stay within 100 req/day limit
- _build_query(): strips dates, punctuation and stopwords from market question
- _score_headlines(): keyword-based pos/neg vote per article, averaged ∈ [-1, +1]
- Degrades to 0.0 on missing key, 403 quota, or network error
bot/strategy/bayesian.py:
- BayesianStrategy(news=NewsClient) — optional, backwards compatible
- Signal 4: GNews sentiment applied as direct log-odds shift (weight=1.5)
so a ±1.0 sentiment score moves a 50% prior to 82%/18%
- +0.10 confidence boost when news signal is present
- NEWS_LOGODDS_WEIGHT constant documented at module level
bot/main.py:
- Instantiate NewsClient, pass to BayesianStrategy, close in finally block
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Every market now emits an INFO line:
TRADE/SKIP <question> | cat=... | prior=... | est=... | edge=... | conf=... | dir=... | signals=... [| reason=...]
Unsupported-category and no-external-signals early exits also log at INFO
so the full evaluation funnel is visible without changing log level.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- polymarket.py: add keyword lists for politics (election, trump, ukraine…),
tech (AI, OpenAI, Apple, nvidia…), and events (super bowl, oscar, spacex…);
introduce _detect_category() so all four categories flow through a single
code path; filter already-expired markets (end_dt < now) in addition to
the existing future-cutoff filter; log per-category counts at startup
- bayesian.py: extend is_any_supported to include is_politics / is_tech /
is_events; use BTC as a risk-sentiment proxy for non-crypto categories
(halved weight to reflect weaker correlation); cap confidence_cap at 0.65
for macro/politics/tech/events; MIN_EDGE stays at 0.10
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
