The signals and markets tables existed since Phase 2/5 but never had a
writer; the replay engine (phase plan line 2.1) needs a per-(market, cycle)
archive of what the strategy saw and decided. This wires them up:
- signals: one row per evaluated market per cycle, now carrying INPUTS
(news_sentiment, feat_*_lo, volume_24h, days_to_resolution) plus the
existing outputs (probs, edges, gates, skip_reason). skip_reason is
granular: unsupported/no_signals/prior_extreme/family/edge_net/
confidence/reentry_guard. news_budget_skipped distinguishes "GNews not
asked" (5-query budget) from "no news".
- ext_snapshots: one row per cycle with the ExternalSignals snapshot;
signals rows join on cycle_ts.
- markets: metadata upserted each cycle (replay rebuilds Market from it).
- Retention: prune > SIGNALS_RETENTION_DAYS (default 90) once a day.
- SIGNAL_RECORDER_ENABLED (default true) gates all DB writes; every write
is try/except — the recorder can never break trading.
Strategy changes are purely additive (record accumulation at each exit
path of evaluate()); no weights, thresholds, gates or sizing touched,
per the freeze in the current phase plan.
Tests: 10 new deterministic tests (85 total passing). Schema migration
dry-run validated against prod postgres inside a rolled-back transaction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-mortem NVIDIA 631181: one uncorroborated high-weight signal (legacy
Manifold 0.13 at weight 0.6) flipped a 0.845 market to 0.431 and lost.
With Manifold observational-only and macro signals gated behind
is_non_price, GNews (weight 1.5) is the only live signal able to move
politics markets 20-30 pp against the order-book consensus. This adds a
catastrophic fuse, not a fine calibration:
- apply_news_guardrail(): when |news_lo| >= NEWS_MATERIAL_LOGODDS_THRESHOLD
(0.10) and every other signal (fg, mom, btc_dom, mfld) is below it,
clamp the posterior to prior ± MAX_NEWS_ONLY_PROB_SHIFT (0.25). Any
corroborating material signal disables the clamp. Config via env
(NEWS_GUARDRAIL_ENABLED=true by default).
- edge_gross/edge_net computed from the clamped posterior; raw_final_prob
preserved in reasoning (persisted via trades.reasoning — no schema
migration) and in the NEWS_MATERIAL log line.
- guardrail_changed_trade_decision: raw edge crossed the regime gate but
the clamped edge no longer does (fuse prevented a trade). Note: with
the default 0.25 band the clamped edge_net is 0.21, above every regime
minimum, so the flag only fires with a tighter configured band.
- Observability gated on materiality: NEWS_MATERIAL per-market line and a
compact NEWS SUMMARY cycle line, only when with_news > 0 — no flood
from the ~145 news-less markets per cycle.
- 9 deterministic tests (extreme clamp, in-band passthrough, corroboration,
inclusive threshold, disabled, changed_decision).
No changes to NEWS_LOGODDS_WEIGHT, Manifold flags, edge thresholds,
sizing, payout, resolution, or historical trades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two minor faults found during the GNews capture/prioritisation diagnostic:
1. Hyphens/dashes reached the GNews query verbatim. '-' is GNews's exclusion
operator, so a token like "El-Sayed" returned HTTP 400 and wasted a query.
_PUNCT_RE now strips '-', en dash and em dash to spaces.
2. The per-cycle GNews budget counter incremented in evaluate() before
get_sentiment() checked the API key, so with no key configured the
[CYCLE SUMMARY] reported a phantom "gnews_queries_used: 5/5" with zero real
requests. Added NewsClient.enabled and gated the GNews block on it; with no
key the counter stays 0/5 and no spurious SKIP_GNEWS_PRIORITY is logged.
No behaviour change when a key is present.
Prioritisation itself was confirmed correct and is left untouched: politics
markets are sorted by gnews_priority DESC and prior-extreme markets return
before the budget is consumed, so no query is ever spent on a market that
cannot trade.
Tests: tests/test_news_query.py (4 new); full suite 66 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sharpe_ratio was hardcoded to 0.0 in MetricsTracker and exposed as
'or 0' in /api/summary. With only 1 resolved trade (~40 flat days plus
one +299 jump) any computed Sharpe is statistically meaningless, so:
- bot/metrics/sharpe.py: annualized Sharpe (sqrt(365)) from daily
total_pnl closes, normalized by bankroll; sharpe_with_gate() returns
None + status until >=30 days observed AND >=10 resolved trades.
- Database.get_daily_pnl_closes(): last metrics_daily snapshot per UTC
day, oldest first — the return series input.
- MetricsTracker: stores the real (gated) Sharpe in the snapshot, NULL
below the gate; log line now includes sharpe.
- /api/summary: live Sharpe + sharpe_status/days_observed/min_* fields
explaining why it is null; resolved_count now live from COUNT(*).
- promotion_ready: requires resolved>=10, days>=30, and non-null
win_rate/calibration/sharpe plus existing thresholds — a single lucky
resolved trade can no longer promote.
- Dashboard Sharpe card shows the insufficient-sample explanation when
null instead of a bare em dash.
Tests: 13 new in tests/test_sharpe_gate.py (formula, gate, API contract,
tracker snapshot); verified failing pre-fix. Suite: 62 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bug #5: metrics.record_trade() only delegated to save_trade(), which
executor.execute() already calls — every trade was written twice (deduped
only by ON CONFLICT DO NOTHING). Remove the redundant call and the now-dead
method. RealExecutor.execute() raises NotImplementedError, so real mode is
unaffected.
Bug #6 (CYCLE SUMMARY): manifold accepted/rejected counters only increment
on the active-signal path, so with MANIFOLD_SIGNAL_ENABLED=false they always
printed 0/0 — print 'manifold_signal: disabled' instead.
family_conflicts_prevented duplicated blocked_by_family (same counter
printed twice); removed. gnews_cap was a dead variable with a misleading
comment; removed.
Bug #7 (/api/summary): total_trades was len() over a LIMIT-500 query —
capped once history grows; counts now come from COUNT(*) via
compute_metrics_from_db. cash_available was reimplemented in the API;
extract cash_available() in paper.py (same formula, unchanged) and feed it
from get_open_position_data() — the exact source/helper
PaperExecutor.initialize() uses. Test asserts API and executor report
identical cash for the same DB state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 3 excluded momentum and Fear & Greed from politics/tech/events
markets; Phase 4 fixed ticker detection. But the BTC-dominance signal was
still applied to non-price markets that legitimately mention a ticker
('Will the ETH ETF be approved?'), despite having no demonstrated causality
for non-price outcomes. Reuse the existing is_non_price gate so the
contribution stays 0.0 -> feat_btc_dom_lo = 0.0 for those markets.
Price-market behavior unchanged: ETH/altcoin/general-crypto markets keep
the +/-0.03 dominance adjustment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Substring matching over question_lower flagged non-crypto markets as crypto:
'dissolved' matched 'sol', 'Canada' matched 'ada', 'Seth' matched 'eth'.
Those false flags armed the BTC-dominance signal (btc_dom_lo=+0.06 observed
on politics markets in production).
Short tickers (btc, eth, sol, xrp, doge, ltc, bnb, ada, avax) now go through
has_token(), which requires non-alphanumeric boundaries so 'ETH', '$ETH' and
'ETH/USD' still match. Long unambiguous names (bitcoin, ethereum, solana,
cardano, ...) remain substring checks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
For politics/tech/events markets there is no above/below price notion, so
is_price_above defaulted to False (or flipped on accidental wording like
"reach") and sign-inverted the macro adjustments: BTC +5% or high Fear&Greed
subtracted probability from YES on "Will X win the election?" markets.
Skip both signals entirely for non-price markets: contributions stay 0.0,
feat_mom_lo / feat_fg_lo persist as 0.0. Price markets (BTC/ETH/crypto)
keep the exact current behavior, including the below-market sign flip.
Removes the now-dead BTC(sentiment) momentum branch and its 0.5 attenuator.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cycle-10 resolution check found market 562186 resolved (Paxton YES) but the
close failed with asyncpg AmbiguousParameterError: Postgres cannot infer the
type of a bare '$3 IS NOT NULL' in the close_pnl CASE. Reproduced via PREPARE
in the postgres pod; fixed by casting every $3 use to double precision.
The failed DB write also left memory/DB diverged: close_position() popped the
position and credited cash before persisting, so the retry at cycle 20 skipped
the market (pnl=n/a) while the DB row stayed open. Now the DB write happens
first and memory mutates only on success; check_resolutions() also isolates
per-market close failures so one error doesn't abort the cycle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- PolymarketClient.get_market_resolution(): query Gamma API by market id;
resolved only when closed AND uma status final AND outcome prices binary
(never settle on disputed/ambiguous outcomes)
- bot/main.py: check_resolutions() runs every 10 cycles (~10 min) in paper
mode, settles open positions via PaperExecutor.close_position()
- close_reason now persisted as 'resolved' (resolution has its own column)
- tests/test_resolution_detector.py: 10 tests covering API parsing shapes
and the BUY_NO settlement flow; 27/27 suite green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Legacy scan called ManifoldClient.get_probability(), removed in the v3
matcher migration, causing AttributeError when positions had changed
family keys. The block used Manifold to escalate positions to
CLOSE_RECOMMENDED (inversion detection) — a trading decision forbidden
under MANIFOLD_SIGNAL_ENABLED=false — so the dependency is removed
entirely; the scan keeps family re-keying and sibling-conflict logic.
- PaperExecutor.close_position() computed cash += position_cost * resolution,
ignoring direction: a winning BUY_NO (resolution=0.0) paid out $0 and
reported a loss. Now settles per trade:
BUY_YES: payout = shares * resolution
BUY_NO: payout = shares * (1 - resolution)
with pnl = payout - net_cost; Telegram win/loss keys off pnl > 0.
Adds read-only Database.get_open_trades_for_market().
- tests/test_paper_close.py covers the 4 deterministic payout cases;
tests/conftest.py shims datetime.UTC for local Python 3.10 (prod is 3.11).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reject false-positive matches where Jaccard overlap is high but the outcome is
not equivalent (e.g. Poly nomination vs Manifold "If X is nominee, will he win").
- _is_conditional(): detect conditional Manifold markets (If/Conditional on/
Assuming/Given that prefixes + mid-sentence " if ...," clauses) -> reject with
reason "conditional_market".
- _classify_outcome(): classify into nomination|primary_win|general_win|
conditional|other; reject when poly/mfld types differ or either is conditional
-> reason "outcome_mismatch: poly=... manifold=...".
- Persist poly_outcome_type/mfld_outcome_type on ManifoldMatchResult, in
manifold_match_audit (CREATE + idempotent ALTER), save_manifold_audit() and
the bayesian call site.
- Tests covering classification, conditional detection and the Graham Platner
regression (now rejected); valid nomination<->nomination still accepted.
Untouched: _MATCH_THRESHOLD (0.40), MANIFOLD_LOGODDS_WEIGHT, edge thresholds,
exposure, trading logic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
