The signals and markets tables existed since Phase 2/5 but never had a
writer; the replay engine (phase plan line 2.1) needs a per-(market, cycle)
archive of what the strategy saw and decided. This wires them up:
- signals: one row per evaluated market per cycle, now carrying INPUTS
(news_sentiment, feat_*_lo, volume_24h, days_to_resolution) plus the
existing outputs (probs, edges, gates, skip_reason). skip_reason is
granular: unsupported/no_signals/prior_extreme/family/edge_net/
confidence/reentry_guard. news_budget_skipped distinguishes "GNews not
asked" (5-query budget) from "no news".
- ext_snapshots: one row per cycle with the ExternalSignals snapshot;
signals rows join on cycle_ts.
- markets: metadata upserted each cycle (replay rebuilds Market from it).
- Retention: prune > SIGNALS_RETENTION_DAYS (default 90) once a day.
- SIGNAL_RECORDER_ENABLED (default true) gates all DB writes; every write
is try/except — the recorder can never break trading.
Strategy changes are purely additive (record accumulation at each exit
path of evaluate()); no weights, thresholds, gates or sizing touched,
per the freeze in the current phase plan.
Tests: 10 new deterministic tests (85 total passing). Schema migration
dry-run validated against prod postgres inside a rolled-back transaction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-mortem NVIDIA 631181: one uncorroborated high-weight signal (legacy
Manifold 0.13 at weight 0.6) flipped a 0.845 market to 0.431 and lost.
With Manifold observational-only and macro signals gated behind
is_non_price, GNews (weight 1.5) is the only live signal able to move
politics markets 20-30 pp against the order-book consensus. This adds a
catastrophic fuse, not a fine calibration:
- apply_news_guardrail(): when |news_lo| >= NEWS_MATERIAL_LOGODDS_THRESHOLD
(0.10) and every other signal (fg, mom, btc_dom, mfld) is below it,
clamp the posterior to prior ± MAX_NEWS_ONLY_PROB_SHIFT (0.25). Any
corroborating material signal disables the clamp. Config via env
(NEWS_GUARDRAIL_ENABLED=true by default).
- edge_gross/edge_net computed from the clamped posterior; raw_final_prob
preserved in reasoning (persisted via trades.reasoning — no schema
migration) and in the NEWS_MATERIAL log line.
- guardrail_changed_trade_decision: raw edge crossed the regime gate but
the clamped edge no longer does (fuse prevented a trade). Note: with
the default 0.25 band the clamped edge_net is 0.21, above every regime
minimum, so the flag only fires with a tighter configured band.
- Observability gated on materiality: NEWS_MATERIAL per-market line and a
compact NEWS SUMMARY cycle line, only when with_news > 0 — no flood
from the ~145 news-less markets per cycle.
- 9 deterministic tests (extreme clamp, in-band passthrough, corroboration,
inclusive threshold, disabled, changed_decision).
No changes to NEWS_LOGODDS_WEIGHT, Manifold flags, edge thresholds,
sizing, payout, resolution, or historical trades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
httpx logs every request URL at INFO level, and the GNews search URL
carries the API key as a `?token=` query param, so GNEWS_API_KEY was
written in plaintext into the pod logs on every news query. Raise the
httpx/httpcore loggers to WARNING so request URLs never reach INFO.
The bot's own GNews log lines only print the sanitised keyword query
(NewsClient._build_query), never the token, so they are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bug #5: metrics.record_trade() only delegated to save_trade(), which
executor.execute() already calls — every trade was written twice (deduped
only by ON CONFLICT DO NOTHING). Remove the redundant call and the now-dead
method. RealExecutor.execute() raises NotImplementedError, so real mode is
unaffected.
Bug #6 (CYCLE SUMMARY): manifold accepted/rejected counters only increment
on the active-signal path, so with MANIFOLD_SIGNAL_ENABLED=false they always
printed 0/0 — print 'manifold_signal: disabled' instead.
family_conflicts_prevented duplicated blocked_by_family (same counter
printed twice); removed. gnews_cap was a dead variable with a misleading
comment; removed.
Bug #7 (/api/summary): total_trades was len() over a LIMIT-500 query —
capped once history grows; counts now come from COUNT(*) via
compute_metrics_from_db. cash_available was reimplemented in the API;
extract cash_available() in paper.py (same formula, unchanged) and feed it
from get_open_position_data() — the exact source/helper
PaperExecutor.initialize() uses. Test asserts API and executor report
identical cash for the same DB state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cycle-10 resolution check found market 562186 resolved (Paxton YES) but the
close failed with asyncpg AmbiguousParameterError: Postgres cannot infer the
type of a bare '$3 IS NOT NULL' in the close_pnl CASE. Reproduced via PREPARE
in the postgres pod; fixed by casting every $3 use to double precision.
The failed DB write also left memory/DB diverged: close_position() popped the
position and credited cash before persisting, so the retry at cycle 20 skipped
the market (pnl=n/a) while the DB row stayed open. Now the DB write happens
first and memory mutates only on success; check_resolutions() also isolates
per-market close failures so one error doesn't abort the cycle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- PolymarketClient.get_market_resolution(): query Gamma API by market id;
resolved only when closed AND uma status final AND outcome prices binary
(never settle on disputed/ambiguous outcomes)
- bot/main.py: check_resolutions() runs every 10 cycles (~10 min) in paper
mode, settles open positions via PaperExecutor.close_position()
- close_reason now persisted as 'resolved' (resolution has its own column)
- tests/test_resolution_detector.py: 10 tests covering API parsing shapes
and the BUY_NO settlement flow; 27/27 suite green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Legacy scan called ManifoldClient.get_probability(), removed in the v3
matcher migration, causing AttributeError when positions had changed
family keys. The block used Manifold to escalate positions to
CLOSE_RECOMMENDED (inversion detection) — a trading decision forbidden
under MANIFOLD_SIGNAL_ENABLED=false — so the dependency is removed
entirely; the scan keeps family re-keying and sibling-conflict logic.
- PaperExecutor.close_position() computed cash += position_cost * resolution,
ignoring direction: a winning BUY_NO (resolution=0.0) paid out $0 and
reported a loss. Now settles per trade:
BUY_YES: payout = shares * resolution
BUY_NO: payout = shares * (1 - resolution)
with pnl = payout - net_cost; Telegram win/loss keys off pnl > 0.
Adds read-only Database.get_open_trades_for_market().
- tests/test_paper_close.py covers the 4 deterministic payout cases;
tests/conftest.py shims datetime.UTC for local Python 3.10 (prod is 3.11).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New module bot/notify/telegram.py — httpx async, fire-and-forget via
asyncio.create_task, swallows all errors so notifications never affect
trade execution.
Three alert types:
📈/📉 TRADE ABIERTO — direction, size, edge_net (in execute())
✅/❌ GANADO/PERDIDO — approx PnL (in close_position())
🔒 LEGACY CLOSE — recovered capital + reason (in close_legacy_position())
close_position() and close_legacy_position() gain an optional question=""
param so the message shows the market name instead of market_id.
bot/main.py updated to pass question= to close_legacy_position().
Credentials (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID) read from env vars
injected via bot-secrets k8s secret.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds feat_fg_lo / feat_mom_lo / feat_news_lo / feat_mfld_lo / feat_btc_dom_lo
to every trade, all normalized to log-odds contribution for direct comparability.
- fg / mom / btc_dom: raw probability-delta × 2 → log-odds
- news / mfld: already log-odds (LOGODDS_WEIGHT already applied), no scaling
- btc_dom tracked separately in bayesian.py instead of bundled in total_adj
- reasoning string updated to fg_lo= / mom_lo= notation for self-documentation
Schema: 5 new DOUBLE PRECISION columns + 2 partial indexes
Stack: TradingSignal → Order → Trade → save_trade all carry feat fields
Startup: backfill_feature_columns() recovers fg/mom/news/mfld from old
reasoning strings (×2 applied to fg/mom); btc_dom_lo stays NULL for legacy
API: /api/metrics/features — triggered/material split per feature with
two-level thresholds (0.05 for fg/mom/btc_dom, 0.10 for news/mfld)
API: /api/trades/legacy — exposes pre-Phase-1 trades (edge_net IS NULL)
API: _enrich_trade backward-compat: reads DB columns first, falls back to
reasoning regex with unit conversion for pre-Phase-6 trades
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- db: update_family_key() persists corrected family slugs for open trades
- db: get_recently_closed_inverted() returns markets closed for inversion
within N hours; used as reentry guard in the trading loop
- db: get_recent_trades() accepts status=open|closed|None and adds a
computed "status" field to every row
- bot/main.py: legacy scan now computes family_key from stored question
alone (dummy Market) when a position's market is no longer active —
fixes NULL family_key on legacy trades like Ken Paxton (562186)
- bot/main.py: legacy scan (Step 2.5) persists corrected family_keys in
DB so family conflict guards work correctly on next restart
- bot/main.py: positions with NULL edge_net and no live market are tagged
legacy_incomplete instead of OK; counted separately in scan summary
- bot/main.py: reentry_guard blocks re-entering any market closed for
inversion bug within 24h; logs reentry_guard_triggered per skip
- api/main.py: /api/trades now accepts ?status=open|closed|all (default
open) and includes status_filter in response
DB fix (applied directly): 629558 family_key politics-2026 →
ohio-gubernatorial-2026; 562186 family_key NULL → texas-republican-2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds run_legacy_scan() that executes once at startup before the trading loop:
1. Re-keys every open DB position using the current market_family_key()
2. Groups by new family key; KEEP = highest edge_net, CLOSE_RECOMMENDED = sibling
3. Manifold re-query for positions whose family key changed; if corrected
probability contradicts the trade direction → CLOSE_RECOMMENDED
4. Logs full report (KEEP / REVIEW / CLOSE_RECOMMENDED) before any closures
5. In paper mode: auto-closes all CLOSE_RECOMMENDED positions
For the existing Ohio bug:
- Democrats win Ohio governor (629557): CLOSE_RECOMMENDED
family changed ohio-democrat-2026 → ohio-gubernatorial-2026
Manifold re-query confirms prob=0.05 contradicts BUY_YES (inversion bug)
$X returned to cash at break-even
- Republicans win Ohio governor (629558): KEEP
higher edge_net (0.349 > 0.247)
Infrastructure:
- schema.sql: closed_at TIMESTAMPTZ, close_reason TEXT on trades
- db.py: all open-position queries filter WHERE closed_at IS NULL
+ close_paper_position(market_id, reason)
- paper.py: close_legacy_position(market_id, reason) → float
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FASE 1 — market_family_key() general election fix
General elections now group by office, not by party, so complementary
markets ("Republicans win Ohio governor" / "Democrats win Ohio governor")
share the same family key (ohio-gubernatorial-2026). The second market
is blocked by the occupied_families check rather than traded as independent.
Primaries still keep the party (texas-republican-2026) because each party
runs its own separate primary race.
FASE 2 — Manifold party inversion guard
_detect_party() identifies the winning side in both the Polymarket question
and the matched Manifold title. If they are confirmed opposites (republican
vs democrat), the probability is inverted (1 - prob) before use.
Full audit log per query:
poly_question / manifold_title / manifold_url / match_score /
prob_raw / inverted / prob_final
Root cause of Ohio Manifold:0.95 on both sides: both queries matched the
same Manifold market ("Republicans win Ohio governor" prob=0.95). For the
"Democrats win" query the inversion now produces prob_final=0.05 instead of
blindly applying 0.95 to the wrong direction.
FASE 4 — startup contradiction scan
get_open_position_details() added to db.py. main.py checks all open
positions at startup, warns on any family with >1 position, and recommends
keeping the one with the highest edge_net. No auto-close.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signal 5: ManifoldClient queries Manifold Markets API for a matching binary
market by keyword overlap (threshold 0.25) and applies a log-odds adjustment
proportional to the divergence from the Polymarket prior.
manifold_log_adj = (log_odds(manifold_prob) - log_odds(prior)) × 0.6
A 30pp divergence (Manifold 0.75 vs Poly 0.45) produces edge_gross ≈ 0.19,
clearing the politics far-horizon regime_min=0.12 after costs. Confidence
boosted +0.08 when Manifold match found.
Per-feature observability: every SKIP_EDGE_NET and TRADE log line now includes
fg=±X.XXX mom=±X.XXX mfld=±X.XXXX news=±X.XXXX
so the contribution of each signal to edge is auditable per market.
Files: bot/data/manifold.py (new), bot/strategy/bayesian.py, bot/main.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 1 — Edge neto real (paper.py, bayesian.py, risk/manager.py, db.py):
- Trade records now store edge_gross, edge_net, prior_prob, final_prob,
mid_price, spread_estimate, commission, family_key
- edge_net = edge_gross - SPREAD_ESTIMATE(0.02) - COMMISSION_RATE(0.02)
NOTE: both constants are heuristics, not exact Polymarket exchange costs
- Execution gate changed from edge_gross > MIN_EDGE to edge_net > regime_min_edge
Phase 2 — Market families (polymarket.py):
- market_family_key(market) groups related markets:
texas-republican-2026, fed-april-2026, openai-2026, etc.
- At most 1 trade per family per cycle; occupied_families propagated via main.py
- Family key logged on every TRADE and SKIP line
Phase 3 — GNews priority (news.py, bayesian.py, main.py):
- NewsClient.get_freshness() returns 1.0/0.75/0.40/0.10 by cache age
- gnews_priority(market, news) = uncertainty × volume_score × freshness
- Politics markets sorted by priority DESC before eval so best markets get
the 5-query/cycle GNews budget first
Phase 4 — Regime min-edge by category/horizon (bayesian.py):
- politics >60d → 0.12, 30-60d → 0.10, <30d → 0.08
- tech / crypto/finance → 0.10
- All thresholds applied to edge_net (not edge_gross)
Phase 5 — Observability (bayesian.py, main.py):
- Structured skip labels: SKIP_UNSUPPORTED, SKIP_NO_SIGNALS,
SKIP_PRIOR_EXTREME, SKIP_FAMILY, SKIP_GNEWS_PRIORITY, SKIP_EDGE_NET
- TRADE lines now include family_key, edge_gross, edge_net, regime_min, days
- schema.sql: 8 new cols on trades, 7 new cols on signals (via ALTER TABLE IF NOT EXISTS)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CACHE_TTL: 4h → 6h (≤36 req/day with ≤9 politics markets)
- GNews only called for is_politics markets (BTC/F&G cover crypto/macro)
- MAX_NEWS_QUERIES_PER_CYCLE=5: BayesianStrategy.reset_cycle() called each
iteration; counter increments only on actual API call (cache hits free)
- 2s asyncio.sleep in news.py finally block after each real HTTP request
- main.py sorts markets: politics first by end_date ascending, so soonest-
resolving markets consume the 5-query budget before others
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bot/data/news.py (new):
- NewsClient with in-memory cache (TTL=4h) to stay within 100 req/day limit
- _build_query(): strips dates, punctuation and stopwords from market question
- _score_headlines(): keyword-based pos/neg vote per article, averaged ∈ [-1, +1]
- Degrades to 0.0 on missing key, 403 quota, or network error
bot/strategy/bayesian.py:
- BayesianStrategy(news=NewsClient) — optional, backwards compatible
- Signal 4: GNews sentiment applied as direct log-odds shift (weight=1.5)
so a ±1.0 sentiment score moves a 50% prior to 82%/18%
- +0.10 confidence boost when news signal is present
- NEWS_LOGODDS_WEIGHT constant documented at module level
bot/main.py:
- Instantiate NewsClient, pass to BayesianStrategy, close in finally block
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chemavx