Compare commits

...
Author SHA1 Message Date
Renovate Bot 733b5c1caa chore(deps): update dependency fastapi to v0.139.0 2026-07-01 18:01:15 +00:00
chemavx 9e21ecac21 Merge pull request 'fix(security): stop httpx from logging GNEWS_API_KEY in plaintext' (#13) from fix/redact-gnews-token-logs into main
CI/CD / build-and-push (push) Successful in 8s
2026-06-26 15:15:44 +00:00
chemavxandClaude Opus 4.8 a3ec69d2be fix(security): stop httpx from logging GNEWS_API_KEY in plaintext
httpx logs every request URL at INFO level, and the GNews search URL
carries the API key as a `?token=` query param, so GNEWS_API_KEY was
written in plaintext into the pod logs on every news query. Raise the
httpx/httpcore loggers to WARNING so request URLs never reach INFO.

The bot's own GNews log lines only print the sanitised keyword query
(NewsClient._build_query), never the token, so they are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 15:13:32 +00:00
chemavx af0d1fbc59 Merge pull request 'fix(news): strip GNews operator dashes and stop phantom query-budget counting' (#12) from fix/gnews-minor into main
CI/CD / build-and-push (push) Successful in 24s
2026-06-26 08:09:04 +00:00
chemavxandClaude Opus 4.8 54fc8fa11a fix(news): strip GNews operator dashes and stop phantom query-budget counting
Two minor faults found during the GNews capture/prioritisation diagnostic:

1. Hyphens/dashes reached the GNews query verbatim. '-' is GNews's exclusion
   operator, so a token like "El-Sayed" returned HTTP 400 and wasted a query.
   _PUNCT_RE now strips '-', en dash and em dash to spaces.

2. The per-cycle GNews budget counter incremented in evaluate() before
   get_sentiment() checked the API key, so with no key configured the
   [CYCLE SUMMARY] reported a phantom "gnews_queries_used: 5/5" with zero real
   requests. Added NewsClient.enabled and gated the GNews block on it; with no
   key the counter stays 0/5 and no spurious SKIP_GNEWS_PRIORITY is logged.
   No behaviour change when a key is present.

Prioritisation itself was confirmed correct and is left untouched: politics
markets are sorted by gnews_priority DESC and prior-extreme markets return
before the budget is consumed, so no query is ever spent on a market that
cannot trade.

Tests: tests/test_news_query.py (4 new); full suite 66 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 08:07:05 +00:00
chemavxandClaude Fable 5 b6153f5859 docs: add README with component map and selective CI behavior
CI/CD / build-and-push (push) Successful in 3s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:08:42 +00:00
chemavxandClaude Fable 5 4928a3c1e4 ci: build only the images whose source files changed
CI/CD / build-and-push (push) Successful in 7s
Path-based build selection diffed against github.event.before:
- bot/, api/, requirements.txt -> bot + api images (both COPY the same
  python sources; only the CMD differs)
- Dockerfile -> bot only; Dockerfile.api -> api only
- dashboard/ -> dashboard only
- .gitea/workflows/ci.yml, first push or force push -> all (safe fallback)
- anything else (tests/, docs) -> no builds, no manifest update

The k8s-manifests sed only bumps tags of rebuilt images, so unchanged
deployments keep their current tag and don't restart. Registry login,
buildx, verification and manifest update are all skipped when nothing
needs building. Telegram message now lists what was built.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:07:49 +00:00
chemavxandClaude Fable 5 43d9577fb2 feat(metrics): real Sharpe ratio from daily PnL curve with minimum-sample gate
CI/CD / build-and-push (push) Successful in 10s
sharpe_ratio was hardcoded to 0.0 in MetricsTracker and exposed as
'or 0' in /api/summary. With only 1 resolved trade (~40 flat days plus
one +299 jump) any computed Sharpe is statistically meaningless, so:

- bot/metrics/sharpe.py: annualized Sharpe (sqrt(365)) from daily
  total_pnl closes, normalized by bankroll; sharpe_with_gate() returns
  None + status until >=30 days observed AND >=10 resolved trades.
- Database.get_daily_pnl_closes(): last metrics_daily snapshot per UTC
  day, oldest first — the return series input.
- MetricsTracker: stores the real (gated) Sharpe in the snapshot, NULL
  below the gate; log line now includes sharpe.
- /api/summary: live Sharpe + sharpe_status/days_observed/min_* fields
  explaining why it is null; resolved_count now live from COUNT(*).
- promotion_ready: requires resolved>=10, days>=30, and non-null
  win_rate/calibration/sharpe plus existing thresholds — a single lucky
  resolved trade can no longer promote.
- Dashboard Sharpe card shows the insufficient-sample explanation when
  null instead of a bare em dash.

Tests: 13 new in tests/test_sharpe_gate.py (formula, gate, API contract,
tracker snapshot); verified failing pre-fix. Suite: 62 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 07:12:55 +00:00
chemavxandClaude Fable 5 1797b79f7b fix(api): aggregate metrics history by day so days=42 spans days, not minutes
CI/CD / build-and-push (push) Successful in 7s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 19:55:48 +00:00
chemavxandClaude Fable 5 060fc89953 fix(accounting): store net PnL (payout - net_cost) in close_pnl, migrate Paxton record from gross to net
CI/CD / build-and-push (push) Successful in 7s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 19:39:34 +00:00
chemavxandClaude Fable 5 c5ffc37820 feat(dashboard): add Cash Disponible card to metrics grid
CI/CD / build-and-push (push) Successful in 11s
Shows /api/summary cash_available (now consistent with the executor's cash
model) next to Capital Deployed, with its share of bankroll as subtitle and
progress bar.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:33:56 +00:00
chemavxandClaude Fable 5 7ebb87aede chore: cleanup duplicate trade save, misleading cycle counters, and /api/summary inconsistencies
CI/CD / build-and-push (push) Successful in 7s
Bug #5: metrics.record_trade() only delegated to save_trade(), which
executor.execute() already calls — every trade was written twice (deduped
only by ON CONFLICT DO NOTHING). Remove the redundant call and the now-dead
method. RealExecutor.execute() raises NotImplementedError, so real mode is
unaffected.

Bug #6 (CYCLE SUMMARY): manifold accepted/rejected counters only increment
on the active-signal path, so with MANIFOLD_SIGNAL_ENABLED=false they always
printed 0/0 — print 'manifold_signal: disabled' instead.
family_conflicts_prevented duplicated blocked_by_family (same counter
printed twice); removed. gnews_cap was a dead variable with a misleading
comment; removed.

Bug #7 (/api/summary): total_trades was len() over a LIMIT-500 query —
capped once history grows; counts now come from COUNT(*) via
compute_metrics_from_db. cash_available was reimplemented in the API;
extract cash_available() in paper.py (same formula, unchanged) and feed it
from get_open_position_data() — the exact source/helper
PaperExecutor.initialize() uses. Test asserts API and executor report
identical cash for the same DB state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:21:32 +00:00
chemavxandClaude Fable 5 02cbfc0b94 fix(executor): keep strong references to fire-and-forget Telegram notification tasks
CI/CD / build-and-push (push) Successful in 7s
asyncio.create_task() results were discarded, and the event loop only holds
a weak reference to running tasks — a pending notification could be
garbage-collected before executing, silently dropping Telegram messages
(documented asyncio pitfall).

Route the three notification call sites (trade_opened, trade_legacy_closed,
trade_closed) through _notify_in_background(), which stores the task in a
module-level set and discards it on completion. Notifications stay
fire-and-forget; no business logic changed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:12:03 +00:00
chemavxandClaude Fable 5 4867141c4b fix(strategy): gate BTC-dominance signal behind is_non_price like momentum and fear-greed
CI/CD / build-and-push (push) Successful in 7s
Phase 3 excluded momentum and Fear & Greed from politics/tech/events
markets; Phase 4 fixed ticker detection.  But the BTC-dominance signal was
still applied to non-price markets that legitimately mention a ticker
('Will the ETH ETF be approved?'), despite having no demonstrated causality
for non-price outcomes.  Reuse the existing is_non_price gate so the
contribution stays 0.0 -> feat_btc_dom_lo = 0.0 for those markets.

Price-market behavior unchanged: ETH/altcoin/general-crypto markets keep
the +/-0.03 dominance adjustment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:42:59 +00:00
chemavxandClaude Fable 5 f5ac302a86 fix(strategy): use word-boundary token matching for short crypto tickers to prevent false positives
CI/CD / build-and-push (push) Successful in 8s
Substring matching over question_lower flagged non-crypto markets as crypto:
'dissolved' matched 'sol', 'Canada' matched 'ada', 'Seth' matched 'eth'.
Those false flags armed the BTC-dominance signal (btc_dom_lo=+0.06 observed
on politics markets in production).

Short tickers (btc, eth, sol, xrp, doge, ltc, bnb, ada, avax) now go through
has_token(), which requires non-alphanumeric boundaries so 'ETH', '$ETH' and
'ETH/USD' still match. Long unambiguous names (bitcoin, ethereum, solana,
cardano, ...) remain substring checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:34:09 +00:00
chemavxandClaude Fable 5 4002f03d0c fix(strategy): exclude momentum and fear-greed signals from non-price markets (politics/tech/events)
CI/CD / build-and-push (push) Successful in 7s
For politics/tech/events markets there is no above/below price notion, so
is_price_above defaulted to False (or flipped on accidental wording like
"reach") and sign-inverted the macro adjustments: BTC +5% or high Fear&Greed
subtracted probability from YES on "Will X win the election?" markets.

Skip both signals entirely for non-price markets: contributions stay 0.0,
feat_mom_lo / feat_fg_lo persist as 0.0. Price markets (BTC/ETH/crypto)
keep the exact current behavior, including the below-market sign flip.
Removes the now-dead BTC(sentiment) momentum branch and its 0.5 attenuator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:40:28 +00:00
chemavxandClaude Fable 5 96f31acf16 fix(metrics): count resolved trades without final_prob in resolved_count
CI/CD / build-and-push (push) Successful in 8s
resolved_count shared the final_prob IS NOT NULL filter with
calibration_score, so the resolved legacy Paxton trade (no signal data)
didn't count: realized_pnl=+309.06 and wins_realized=1 but resolved_count=0.
resolved_count now only requires resolution + not excluded; calibration
keeps the final_prob requirement since it scores against the estimate.

Validated against prod DB: new filter returns 1, old returned 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 14:32:51 +00:00
chemavxandClaude Fable 5 5aa54eb423 fix(resolution): cast $3 explicitly in close_paper_position and persist before mutating portfolio
CI/CD / build-and-push (push) Successful in 8s
Cycle-10 resolution check found market 562186 resolved (Paxton YES) but the
close failed with asyncpg AmbiguousParameterError: Postgres cannot infer the
type of a bare '$3 IS NOT NULL' in the close_pnl CASE. Reproduced via PREPARE
in the postgres pod; fixed by casting every $3 use to double precision.

The failed DB write also left memory/DB diverged: close_position() popped the
position and credited cash before persisting, so the retry at cycle 20 skipped
the market (pnl=n/a) while the DB row stayed open. Now the DB write happens
first and memory mutates only on success; check_resolutions() also isolates
per-market close failures so one error doesn't abort the cycle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 14:15:05 +00:00
chemavxandClaude Fable 5 e137116e7f feat(resolution): add automatic market resolution detector with conservative payout validation
CI/CD / build-and-push (push) Successful in 8s
- PolymarketClient.get_market_resolution(): query Gamma API by market id;
  resolved only when closed AND uma status final AND outcome prices binary
  (never settle on disputed/ambiguous outcomes)
- bot/main.py: check_resolutions() runs every 10 cycles (~10 min) in paper
  mode, settles open positions via PaperExecutor.close_position()
- close_reason now persisted as 'resolved' (resolution has its own column)
- tests/test_resolution_detector.py: 10 tests covering API parsing shapes
  and the BUY_NO settlement flow; 27/27 suite green

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:48:41 +00:00
chemavxandClaude Fable 5 340c8523cf fix(critical): remove dead manifold.get_probability() from legacy scan and fix BUY_NO payout calculation
CI/CD / build-and-push (push) Successful in 29s
- Legacy scan called ManifoldClient.get_probability(), removed in the v3
  matcher migration, causing AttributeError when positions had changed
  family keys. The block used Manifold to escalate positions to
  CLOSE_RECOMMENDED (inversion detection) — a trading decision forbidden
  under MANIFOLD_SIGNAL_ENABLED=false — so the dependency is removed
  entirely; the scan keeps family re-keying and sibling-conflict logic.

- PaperExecutor.close_position() computed cash += position_cost * resolution,
  ignoring direction: a winning BUY_NO (resolution=0.0) paid out $0 and
  reported a loss. Now settles per trade:
    BUY_YES: payout = shares * resolution
    BUY_NO:  payout = shares * (1 - resolution)
  with pnl = payout - net_cost; Telegram win/loss keys off pnl > 0.
  Adds read-only Database.get_open_trades_for_market().

- tests/test_paper_close.py covers the 4 deterministic payout cases;
  tests/conftest.py shims datetime.UTC for local Python 3.10 (prod is 3.11).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 12:23:44 +00:00
chemavxandClaude Opus 4.8 3a353c7e5b feat(manifold): decouple Manifold from edge model (observational-only)
CI/CD / build-and-push (push) Successful in 8s
Per-category coverage audit showed coverage_rate=0.0 across every category in the
bot's current universe, so any edge Manifold produced was false edge. Retire it as
an ACTIVE trading signal while keeping the full audit/coverage/cooldown trail for a
future reactivation decision.

Two module-level flags in bayesian.py (read from env):
- MANIFOLD_SIGNAL_ENABLED (default False): when False, Manifold never touches the
  edge model — manifold_log_adj stays 0.0 (no posterior shift), no confidence bump,
  feat_mfld_lo=0.0 (so it can never be the dominant feature), no trade contribution,
  and mfld_audit_id is not propagated so the audit's used_in_trade stays FALSE.
- MANIFOLD_AUDIT_ENABLED (default True): matcher still runs; audit/coverage rows and
  cooldowns are still written. The matcher is only called when a flag is on.

When signal is disabled, logs and reasoning carry "Manifold: observational_only".
Endpoints /api/metrics/manifold-matches and /api/metrics/manifold-coverage,
cooldowns, audit tables and existing trades are unchanged. No code or tables removed.

Other signals, thresholds, exposure, risk manager and the executor are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:58:02 +00:00
chemavxandClaude Opus 4.8 823914789d feat(api): add /api/metrics/manifold-coverage by-category endpoint
CI/CD / build-and-push (push) Successful in 8s
Measure real Manifold coverage per semantic market category counted by UNIQUE
market (not audit rows, which are inflated by repeated re-evaluation). Base table
is manifold_match_audit filtered to v3_outcome_guard; each poly_market_id is
collapsed to one row, LEFT JOIN trades for family_key, category inferred from
family_key when present else from poly_question (gubernatorial / mayoral / senate
/ primary-republican / primary-democrat / big-tech / geopolitics / other).

Per category: unique_evaluated / unique_accepted / unique_rejected /
unique_no_results / coverage_rate, ordered by unique_evaluated DESC. Summary adds
total_unique_evaluated, total_unique_accepted, overall_coverage_rate,
categories_with_coverage. Read-only: new db method + endpoint only; matcher,
scheduler, cooldowns, thresholds and exposure untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:41:45 +00:00
chemavxandClaude Opus 4.8 98abd96fd2 feat(manifold): add persistent cooldowns to reduce redundant evaluations
CI/CD / build-and-push (push) Successful in 8s
The trading loop re-evaluated the same ~22 politics/tech markets every ~60s,
flooding manifold_match_audit with ~76k rows (~3,500 attempts/market) of which
none carried new information, making the metrics uninterpretable.

Add per-market persistent cooldowns:
- New table manifold_eval_cooldown (poly_market_id PK, last_evaluated_at,
  last_status, retry_after, cooldown_reason) created via run_migrations.
- bayesian.evaluate() consults the cooldown BEFORE calling the matcher and skips
  the call entirely while now() < retry_after — no matcher call, no audit row,
  no signal (equivalent to no_results). After a real evaluation it upserts the
  cooldown with a verdict-dependent backoff: no_results/low_score/outcome_mismatch/
  ambiguous_inversion -> 24h, conditional_market -> 7d, accepted -> 1h.
- manifold.py stays a pure client/matcher with no DB access.
- New db methods get_manifold_cooldown / upsert_manifold_cooldown.
- /api/metrics/manifold-matches summary gains unique_markets
  {evaluated, accepted, coverage_rate} for per-market (not per-attempt) coverage.

Matching logic, MANIFOLD_MATCHER_VERSION, MANIFOLD_LOGODDS_WEIGHT, edge/exposure
thresholds and existing audit rows are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 12:19:31 +00:00
chemavxandClaude Opus 4.8 df988a36b6 fix(metrics): exclude excluded_from_metrics trades from trades_dominated_by_mfld counter
CI/CD / build-and-push (push) Successful in 7s
The trades_dominated_by_mfld counter omitted the excluded_from_metrics
filter, so the admin-closed Maine governor trade inflated it to 1 while
attribution/features (which exclude such trades) were empty.

Add excluded_from_metrics IS NOT TRUE and mfld_match_status = 'accepted'
to the query so the counter is consistent with the attribution and
feature-metrics endpoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 09:06:06 +00:00
chemavxandClaude Opus 4.8 664ecab174 feat(manifold): add matcher versioning to separate legacy accepted matches from v3_outcome_guard metrics
CI/CD / build-and-push (push) Successful in 9s
Add MANIFOLD_MATCHER_VERSION="v3_outcome_guard" tag persisted to
manifold_match_audit.matcher_version so metrics can isolate current-matcher
stats from pre-versioning records, whose accepted matches the outcome
guard would now reject.

- schema: add matcher_version column + index; idempotent startup backfill
  tagging NULL rows as legacy_pre_outcome_guard (no outcome types) or
  v2_outcome_guard_no_version (has outcome type, version not persisted)
- save_manifold_audit: write matcher_version on every new record
- get_manifold_matches: split summary into current_version / all_time /
  legacy; recent_matches now carry matcher_version

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 08:59:19 +00:00
chemavxandClaude Opus 4.8 34fd1f8719 feat(manifold): add outcome compatibility guard and conditional market rejection
CI/CD / build-and-push (push) Successful in 7s
Reject false-positive matches where Jaccard overlap is high but the outcome is
not equivalent (e.g. Poly nomination vs Manifold "If X is nominee, will he win").

- _is_conditional(): detect conditional Manifold markets (If/Conditional on/
  Assuming/Given that prefixes + mid-sentence " if ...," clauses) -> reject with
  reason "conditional_market".
- _classify_outcome(): classify into nomination|primary_win|general_win|
  conditional|other; reject when poly/mfld types differ or either is conditional
  -> reason "outcome_mismatch: poly=... manifold=...".
- Persist poly_outcome_type/mfld_outcome_type on ManifoldMatchResult, in
  manifold_match_audit (CREATE + idempotent ALTER), save_manifold_audit() and
  the bayesian call site.
- Tests covering classification, conditional detection and the Graham Platner
  regression (now rejected); valid nomination<->nomination still accepted.

Untouched: _MATCH_THRESHOLD (0.40), MANIFOLD_LOGODDS_WEIGHT, edge thresholds,
exposure, trading logic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 15:28:26 +00:00
chemavxandClaude Sonnet 4.6 d51d47c921 feat(notify): checkpoint alerts for first match, trade, resolution and exposure cap
CI/CD / build-and-push (push) Successful in 8s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 08:47:51 +00:00
chemavxandClaude Sonnet 4.6 8febd32136 feat(metrics): add excluded_from_metrics flag and exclude admin-closed trades from win_rate/calibration
CI/CD / build-and-push (push) Successful in 7s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 16:12:52 +00:00
chemavxandClaude Sonnet 4.6 9abaae44fd feat(manifold): audit matching quality with ManifoldMatchResult and manifold_match_audit table
CI/CD / build-and-push (push) Successful in 14s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 15:58:07 +00:00
27 changed files with 3249 additions and 259 deletions
+89 -10
View File
@@ -19,15 +19,64 @@ jobs:
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
ssl-verify: false ssl-verify: false
# Full history: needed to diff against github.event.before
fetch-depth: 0
- name: Set image tag - name: Set image tag
id: tag id: tag
run: echo "TAG=${GITHUB_SHA::8}" >> $GITHUB_OUTPUT run: echo "TAG=${GITHUB_SHA::8}" >> $GITHUB_OUTPUT
- name: Detect changed components
id: changes
run: |
BEFORE="${{ github.event.before }}"
CHANGED=""
case "$BEFORE" in
""|0000000000000000000000000000000000000000)
echo "First push or unknown base — building all images"
CHANGED="__all__"
;;
*)
if git cat-file -e "$BEFORE" 2>/dev/null; then
CHANGED=$(git diff --name-only "$BEFORE" "$GITHUB_SHA")
else
echo "Base commit $BEFORE not in history (force push?) — building all images"
CHANGED="__all__"
fi
;;
esac
echo "Changed files:"
echo "$CHANGED"
if [ "$CHANGED" = "__all__" ]; then
BOT=true; API=true; DASH=true
else
BOT=false; API=false; DASH=false
matches() { echo "$CHANGED" | grep -qE "$1"; }
# The workflow itself affects every image build
if matches '^\.gitea/workflows/ci\.yml$'; then BOT=true; API=true; DASH=true; fi
# bot and api images both COPY bot/, api/ and requirements.txt
if matches '^(bot/|api/|requirements\.txt$)'; then BOT=true; API=true; fi
if matches '^Dockerfile$'; then BOT=true; fi
if matches '^Dockerfile\.api$'; then API=true; fi
# dashboard image builds from the dashboard/ context only
if matches '^dashboard/'; then DASH=true; fi
fi
ANY=false
if [ "$BOT" = "true" ] || [ "$API" = "true" ] || [ "$DASH" = "true" ]; then ANY=true; fi
echo "build_bot=$BOT" >> $GITHUB_OUTPUT
echo "build_api=$API" >> $GITHUB_OUTPUT
echo "build_dashboard=$DASH" >> $GITHUB_OUTPUT
echo "build_any=$ANY" >> $GITHUB_OUTPUT
echo "Will build: bot=$BOT api=$API dashboard=$DASH"
- name: Log in to registry - name: Log in to registry
if: steps.changes.outputs.build_any == 'true'
run: echo "${{ secrets.CI_TOKEN }}" | docker login gitea.gitea.svc.cluster.local:3000 -u chemavx --password-stdin run: echo "${{ secrets.CI_TOKEN }}" | docker login gitea.gitea.svc.cluster.local:3000 -u chemavx --password-stdin
- name: Create buildx builder - name: Create buildx builder
if: steps.changes.outputs.build_any == 'true'
run: | run: |
cat > /tmp/buildkitd.toml << 'EOF' cat > /tmp/buildkitd.toml << 'EOF'
[registry."registry-cache.registry-cache.svc.cluster.local:5000"] [registry."registry-cache.registry-cache.svc.cluster.local:5000"]
@@ -50,6 +99,7 @@ jobs:
docker buildx inspect --bootstrap docker buildx inspect --bootstrap
- name: Build and push bot image - name: Build and push bot image
if: steps.changes.outputs.build_bot == 'true'
run: | run: |
TAG=${{ steps.tag.outputs.TAG }} TAG=${{ steps.tag.outputs.TAG }}
docker buildx build \ docker buildx build \
@@ -61,6 +111,7 @@ jobs:
-f Dockerfile . -f Dockerfile .
- name: Build and push API image - name: Build and push API image
if: steps.changes.outputs.build_api == 'true'
run: | run: |
TAG=${{ steps.tag.outputs.TAG }} TAG=${{ steps.tag.outputs.TAG }}
docker buildx build \ docker buildx build \
@@ -72,6 +123,7 @@ jobs:
-f Dockerfile.api . -f Dockerfile.api .
- name: Build and push dashboard image - name: Build and push dashboard image
if: steps.changes.outputs.build_dashboard == 'true'
run: | run: |
TAG=${{ steps.tag.outputs.TAG }} TAG=${{ steps.tag.outputs.TAG }}
docker buildx build \ docker buildx build \
@@ -84,6 +136,7 @@ jobs:
dashboard dashboard
- name: Verify images in registry - name: Verify images in registry
if: steps.changes.outputs.build_any == 'true'
run: | run: |
TAG=${{ steps.tag.outputs.TAG }} TAG=${{ steps.tag.outputs.TAG }}
check_image() { check_image() {
@@ -98,11 +151,18 @@ jobs:
fi fi
echo "OK: chemavx/${image}:${TAG} verified in registry" echo "OK: chemavx/${image}:${TAG} verified in registry"
} }
check_image polymarket-bot if [ "${{ steps.changes.outputs.build_bot }}" = "true" ]; then
check_image polymarket-bot-api check_image polymarket-bot
check_image polymarket-bot-dashboard fi
if [ "${{ steps.changes.outputs.build_api }}" = "true" ]; then
check_image polymarket-bot-api
fi
if [ "${{ steps.changes.outputs.build_dashboard }}" = "true" ]; then
check_image polymarket-bot-dashboard
fi
- name: Update k8s manifests - name: Update k8s manifests
if: steps.changes.outputs.build_any == 'true'
run: | run: |
pip3 install pyyaml -q pip3 install pyyaml -q
@@ -114,12 +174,20 @@ jobs:
git clone ${{ env.K8S_MANIFESTS_REPO }} /tmp/k8s-manifests git clone ${{ env.K8S_MANIFESTS_REPO }} /tmp/k8s-manifests
cd /tmp/k8s-manifests cd /tmp/k8s-manifests
sed -i "s|image: .*polymarket-bot[^-].*|image: git.chemavx.xyz/chemavx/polymarket-bot:${TAG}|g" \ # Only bump the tag of images that were actually rebuilt: the others
polymarket-bot/deployment-bot.yaml # keep their current (still existing) tag in the registry.
sed -i "s|image: .*polymarket-bot-api.*|image: git.chemavx.xyz/chemavx/polymarket-bot-api:${TAG}|g" \ if [ "${{ steps.changes.outputs.build_bot }}" = "true" ]; then
polymarket-bot/deployment-api.yaml sed -i "s|image: .*polymarket-bot[^-].*|image: git.chemavx.xyz/chemavx/polymarket-bot:${TAG}|g" \
sed -i "s|image: .*polymarket-bot-dashboard.*|image: git.chemavx.xyz/chemavx/polymarket-bot-dashboard:${TAG}|g" \ polymarket-bot/deployment-bot.yaml
polymarket-bot/deployment-dashboard.yaml fi
if [ "${{ steps.changes.outputs.build_api }}" = "true" ]; then
sed -i "s|image: .*polymarket-bot-api.*|image: git.chemavx.xyz/chemavx/polymarket-bot-api:${TAG}|g" \
polymarket-bot/deployment-api.yaml
fi
if [ "${{ steps.changes.outputs.build_dashboard }}" = "true" ]; then
sed -i "s|image: .*polymarket-bot-dashboard.*|image: git.chemavx.xyz/chemavx/polymarket-bot-dashboard:${TAG}|g" \
polymarket-bot/deployment-dashboard.yaml
fi
sed -i "s|imagePullPolicy: Never|imagePullPolicy: Always|g" \ sed -i "s|imagePullPolicy: Never|imagePullPolicy: Always|g" \
polymarket-bot/deployment-bot.yaml \ polymarket-bot/deployment-bot.yaml \
polymarket-bot/deployment-api.yaml \ polymarket-bot/deployment-api.yaml \
@@ -154,10 +222,21 @@ jobs:
TAG: ${{ steps.tag.outputs.TAG }} TAG: ${{ steps.tag.outputs.TAG }}
JOB_STATUS: ${{ job.status }} JOB_STATUS: ${{ job.status }}
TELEGRAM_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} TELEGRAM_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
BUILD_BOT: ${{ steps.changes.outputs.build_bot }}
BUILD_API: ${{ steps.changes.outputs.build_api }}
BUILD_DASH: ${{ steps.changes.outputs.build_dashboard }}
run: | run: |
TAG="${TAG:-${GITHUB_SHA:0:8}}" TAG="${TAG:-${GITHUB_SHA:0:8}}"
BUILT=""
[ "$BUILD_BOT" = "true" ] && BUILT="${BUILT}bot "
[ "$BUILD_API" = "true" ] && BUILT="${BUILT}api "
[ "$BUILD_DASH" = "true" ] && BUILT="${BUILT}dashboard "
if [ "$JOB_STATUS" = "success" ]; then if [ "$JOB_STATUS" = "success" ]; then
MSG="✅ Deploy polymarket-bot:${TAG} completado" if [ -n "$BUILT" ]; then
MSG="✅ Deploy polymarket-bot:${TAG} completado (imágenes: ${BUILT% })"
else
MSG="✅ CI polymarket-bot:${TAG} OK — sin cambios de imagen, nada que desplegar"
fi
else else
MSG="❌ Deploy polymarket-bot:${TAG} fallido (status: ${JOB_STATUS})" MSG="❌ Deploy polymarket-bot:${TAG} fallido (status: ${JOB_STATUS})"
fi fi
+35
View File
@@ -0,0 +1,35 @@
# polymarket-bot
Bot de paper-trading para Polymarket con estrategia bayesiana, API FastAPI y
dashboard React. Corre en k3s vía GitOps (Gitea Actions → registry → ArgoCD).
## Componentes
| Componente | Código | Imagen | CMD |
|---|---|---|---|
| bot | `bot/` | `polymarket-bot` | `python3 -m bot.main` |
| api | `api/` (+ `bot/` como librería) | `polymarket-bot-api` | `uvicorn api.main:app` |
| dashboard | `dashboard/` | `polymarket-bot-dashboard` | nginx estático |
Dashboard: https://polymarket.chemavx.xyz
## CI/CD
`.gitea/workflows/ci.yml` construye **solo las imágenes cuyos ficheros
cambiaron** en el push (diff contra `github.event.before`):
- `bot/`, `api/`, `requirements.txt` → bot + api (ambas imágenes copian las
mismas fuentes Python; solo cambia el CMD)
- `Dockerfile` → bot · `Dockerfile.api` → api · `dashboard/` → dashboard
- `.gitea/workflows/ci.yml`, primer push o force-push → todas (fallback seguro)
- `tests/`, docs → ninguna (la CI no construye ni despliega nada)
Las imágenes se tagean con `${GITHUB_SHA::8}`; el CI actualiza solo los
deployments reconstruidos en `k8s-manifests/polymarket-bot/` y ArgoCD
sincroniza vía webhook en segundos.
## Tests
```bash
python3 -m pytest tests/ -q
```
+121 -24
View File
@@ -11,6 +11,12 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from bot.data.db import Database from bot.data.db import Database
from bot.executor.paper import cash_available
from bot.metrics.sharpe import (
MIN_DAYS_OBSERVED,
MIN_RESOLVED_TRADES,
sharpe_with_gate,
)
# Phase 6 format (Phase 6+): values already in log-odds space. # Phase 6 format (Phase 6+): values already in log-odds space.
# "fg_lo=+0.1200 mom_lo=+0.0000 news_lo=+0.0000 mfld_lo=-0.7483 btc_dom_lo=+0.0000" # "fg_lo=+0.1200 mom_lo=+0.0000 news_lo=+0.0000 mfld_lo=-0.7483 btc_dom_lo=+0.0000"
@@ -209,6 +215,66 @@ async def get_attribution():
return {"attribution": attribution, "total_attributed_trades": total} return {"attribution": attribution, "total_attributed_trades": total}
@app.get("/api/metrics/manifold-matches")
async def get_manifold_matches():
"""Manifold match audit — version-split summary and recent match attempts.
summary.current_version — stats for the active matcher (MANIFOLD_MATCHER_VERSION):
version — the matcher version string
total_accepted — matches accepted (score >= 0.40, inversion unambiguous)
total_rejected — matches rejected (low score or ambiguous inversion)
total_no_results — no Manifold market found or API error
avg_match_score — mean Jaccard score for accepted matches
used_in_trade — accepted matches that were actually executed
summary.all_time — accepted/rejected/no_results across every matcher version.
summary.legacy.accepted_without_outcome_type — pre-outcome-guard accepted
records that the current matcher would reject (not counted in current_version).
summary.trades_dominated_by_mfld — non-excluded accepted-match trades where
feat_mfld_lo is the largest signal (consistent with attribution/features,
which also exclude excluded_from_metrics trades).
summary.unique_markets — distinct-market coverage (per-market, not per-attempt):
evaluated — DISTINCT poly_market_id in manifold_match_audit (all versions)
accepted — DISTINCT poly_market_id accepted by the current matcher
coverage_rate — accepted / evaluated (null when evaluated=0)
recent_matches: last 50 rows from manifold_match_audit, newest first, each
tagged with matcher_version.
used_in_trade=True only when status='accepted' AND a trade was actually executed.
"""
data = await db.get_manifold_matches(limit=50)
for match in data["recent_matches"]:
ts = match.get("timestamp")
if ts is not None and hasattr(ts, "isoformat"):
match["timestamp"] = ts.isoformat()
return data
@app.get("/api/metrics/manifold-coverage")
async def get_manifold_coverage():
"""Manifold coverage by semantic market category, counted by UNIQUE market.
Unlike the raw audit counters (which count per-attempt and are inflated by the
trading loop re-evaluating the same markets), this measures real coverage:
how many DISTINCT markets in each category the matcher found an accepted
Manifold counterpart for. Base table is manifold_match_audit filtered to the
current matcher (v3_outcome_guard); category is inferred from trade family_key
when available, otherwise from the question text.
coverage_by_category — one entry per category, ordered by unique_evaluated DESC:
category — gubernatorial | mayoral | senate | primary-republican |
primary-democrat | big-tech | geopolitics | other
unique_evaluated — distinct markets audited in this category
unique_accepted — distinct markets with at least one accepted match
unique_rejected — distinct markets with at least one rejected match
unique_no_results — distinct markets with at least one no_results outcome
coverage_rate — unique_accepted / unique_evaluated (null if evaluated=0)
summary — total_unique_evaluated, total_unique_accepted, overall_coverage_rate
(null if nothing evaluated), categories_with_coverage (categories with
unique_accepted > 0).
"""
return await db.get_manifold_coverage_by_category()
@app.get("/api/summary") @app.get("/api/summary")
async def get_summary(): async def get_summary():
"""Dashboard summary card data. """Dashboard summary card data.
@@ -219,28 +285,49 @@ async def get_summary():
PnL and performance metrics come from the latest metrics_daily snapshot, PnL and performance metrics come from the latest metrics_daily snapshot,
which is written by the bot every cycle via MetricsTracker.update_daily_summary(). which is written by the bot every cycle via MetricsTracker.update_daily_summary().
After Fix 3, that snapshot is also DB-computed — not dependent on pod restarts. After Fix 3, that snapshot is also DB-computed — not dependent on pod restarts.
sharpe_ratio is the exception: it is recomputed live here from the daily
PnL-close series (same sharpe_with_gate the tracker uses), so the
explanation fields (sharpe_status, days_observed) always match the value.
""" """
latest_metrics, open_trades, all_trades, inverted, legacy_count = await asyncio.gather( latest_metrics, counts, position_data, inverted, legacy_count, daily_closes = (
db.get_metrics_history(days=1), await asyncio.gather(
db.get_recent_trades(limit=500, status="open"), db.get_metrics_history(days=1),
db.get_recent_trades(limit=500), db.compute_metrics_from_db(),
db.get_recently_closed_inverted(hours=24), db.get_open_position_data(),
db.get_legacy_incomplete_count(), db.get_recently_closed_inverted(hours=24),
db.get_legacy_incomplete_count(),
db.get_daily_pnl_closes(),
)
) )
latest = latest_metrics[0] if latest_metrics else {} latest = latest_metrics[0] if latest_metrics else {}
paper_bankroll = float(os.getenv("PAPER_BANKROLL", "10000")) paper_bankroll = float(os.getenv("PAPER_BANKROLL", "10000"))
total_deployed = sum(t.get("net_cost", 0) for t in open_trades) total_trades = int(counts["total_trades"] or 0)
resolved_count = int(counts.get("resolved_count") or 0)
# Same source PaperExecutor.initialize() uses to reconstruct cash:
# total_net_cost_open = SUM(net_cost) over open trades, uncapped.
_, total_net_cost_open = position_data
total_deployed = total_net_cost_open
# Sharpe: computed live from the daily PnL curve (same function the
# tracker uses for the snapshot). None + status while the minimum-sample
# gate (>=30 days observed, >=10 resolved trades) is not met — a Sharpe
# over 1 resolved trade is statistically meaningless.
days_observed = len(daily_closes)
sharpe, sharpe_status = sharpe_with_gate(daily_closes, paper_bankroll, resolved_count)
win_rate = latest.get("win_rate")
calibration = latest.get("calibration_score")
return { return {
# ── Portfolio state (live from DB) ────────────────────────────────── # ── Portfolio state (live from DB) ──────────────────────────────────
"paper_mode": os.getenv("PAPER_MODE", "true") == "true", "paper_mode": os.getenv("PAPER_MODE", "true") == "true",
"paper_bankroll": paper_bankroll, "paper_bankroll": paper_bankroll,
"total_trades": len(all_trades), # exact, from DB "total_trades": total_trades, # COUNT(*), uncapped
"open_trades_count": len(open_trades), # exact, from DB "open_trades_count": int(counts["open_count"] or 0), # COUNT(*), uncapped
"closed_trades_count": len(all_trades) - len(open_trades), # exact "closed_trades_count": int(counts["closed_count"] or 0), # COUNT(*), uncapped
"total_deployed": total_deployed, # exact, from DB "total_deployed": total_deployed, # exact, from DB
"cash_available": max(0.0, paper_bankroll - total_deployed), # exact "cash_available": cash_available(paper_bankroll, total_net_cost_open),
"legacy_incomplete_count": legacy_count, # exact, from DB "legacy_incomplete_count": legacy_count, # exact, from DB
"reentry_guard_blocks_24h": len(inverted), # exact, from DB "reentry_guard_blocks_24h": len(inverted), # exact, from DB
@@ -248,31 +335,41 @@ async def get_summary():
# unrealized_pnl_est: open positions, edge_net × net_cost fee. # unrealized_pnl_est: open positions, edge_net × net_cost fee.
# Estimated — uses model signal, not live price. Source: open trades. # Estimated — uses model signal, not live price. Source: open trades.
# realized_pnl: closed positions with known resolution. # realized_pnl: closed positions with known resolution.
# Exact — computed from (resolution entry_price) × shares. # Exact — payout net_cost per trade (net of fee), matches logs/Telegram.
# total_pnl: sum of both. # total_pnl: sum of both.
"unrealized_pnl_est": latest.get("unrealized_pnl_est") or 0, "unrealized_pnl_est": latest.get("unrealized_pnl_est") or 0,
"realized_pnl": latest.get("realized_pnl") or 0, "realized_pnl": latest.get("realized_pnl") or 0,
"total_pnl": latest.get("total_pnl") or 0, "total_pnl": latest.get("total_pnl") or 0,
# ── Performance metrics (from latest metrics_daily snapshot) ───────── # ── Performance metrics ──────────────────────────────────────────────
# win_rate: fraction of resolved closed trades where close_pnl > 0. # win_rate: fraction of resolved closed trades where close_pnl > 0.
# null if fewer than 5 resolved trades. Source: closed+resolved trades. # null if fewer than 5 resolved trades. Source: closed+resolved trades.
# sharpe_ratio: 0.0 — requires daily-return time series (not yet tracked). # sharpe_ratio: annualized Sharpe of the daily total_pnl curve, computed
# live from metrics_daily. null while the minimum-sample gate fails
# (sharpe_status explains why). Source: bot/metrics/sharpe.py.
# calibration_score: 1 Brier score on resolved trades (higher = better). # calibration_score: 1 Brier score on resolved trades (higher = better).
# null if fewer than 10 resolved trades. Source: closed+resolved trades. # null if fewer than 10 resolved trades. Source: closed+resolved trades.
"win_rate": latest.get("win_rate"), # null if < 5 resolved "win_rate": win_rate, # null if < 5 resolved
"sharpe_ratio": latest.get("sharpe_ratio") or 0, # 0.0 until tracked "sharpe_ratio": sharpe, # null if gate fails
"calibration_score": latest.get("calibration_score"), # null if < 10 resolved "sharpe_status": sharpe_status, # ok | insufficient_sample | zero_variance
"days_observed": days_observed,
"min_days_required": MIN_DAYS_OBSERVED,
"min_resolved_required": MIN_RESOLVED_TRADES,
"calibration_score": calibration, # null if < 10 resolved
# ── Counters from snapshot ─────────────────────────────────────────── # ── Counters (live from DB) ──────────────────────────────────────────
"resolved_count": latest.get("resolved_count") or 0, "resolved_count": resolved_count,
# ── Promotion gate ─────────────────────────────────────────────────── # ── Promotion gate ───────────────────────────────────────────────────
# All thresholds must pass; null metrics count as not-ready. # Never promote on a tiny sample: requires the resolved/days minimums
# AND non-null metrics AND all thresholds. A single lucky resolved
# trade must not flip this to true.
"promotion_ready": ( "promotion_ready": (
(latest.get("sharpe_ratio") or 0) >= 0.5 resolved_count >= MIN_RESOLVED_TRADES
and (latest.get("win_rate") or 0) >= 0.52 and days_observed >= MIN_DAYS_OBSERVED
and (latest.get("calibration_score") or 0) >= 0.7 and win_rate is not None and win_rate >= 0.52
and len(all_trades) >= 50 and calibration is not None and calibration >= 0.7
and sharpe is not None and sharpe >= 0.5
and total_trades >= 50
), ),
} }
+376 -16
View File
@@ -4,6 +4,8 @@ import os
from typing import Optional from typing import Optional
import asyncpg import asyncpg
from bot.data.manifold import MANIFOLD_MATCHER_VERSION
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@@ -36,11 +38,15 @@ class Database:
entry_price, shares, fee_usdc, net_cost, timestamp, reasoning, paper, entry_price, shares, fee_usdc, net_cost, timestamp, reasoning, paper,
edge_gross, edge_net, prior_prob, final_prob, edge_gross, edge_net, prior_prob, final_prob,
mid_price, spread_estimate, commission, family_key, mid_price, spread_estimate, commission, family_key,
feat_fg_lo, feat_mom_lo, feat_news_lo, feat_mfld_lo, feat_btc_dom_lo feat_fg_lo, feat_mom_lo, feat_news_lo, feat_mfld_lo, feat_btc_dom_lo,
mfld_market_id, mfld_market_title, mfld_market_url,
mfld_prob_raw, mfld_prob_final, mfld_inverted,
mfld_match_score, mfld_match_reason, mfld_match_status
) VALUES ( ) VALUES (
$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12, $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,
$13,$14,$15,$16,$17,$18,$19,$20, $13,$14,$15,$16,$17,$18,$19,$20,
$21,$22,$23,$24,$25 $21,$22,$23,$24,$25,
$26,$27,$28,$29,$30,$31,$32,$33,$34
) )
ON CONFLICT (id) DO NOTHING ON CONFLICT (id) DO NOTHING
""", """,
@@ -53,6 +59,10 @@ class Database:
# Phase 6 feature log-odds # Phase 6 feature log-odds
trade.feat_fg_lo, trade.feat_mom_lo, trade.feat_news_lo, trade.feat_fg_lo, trade.feat_mom_lo, trade.feat_news_lo,
trade.feat_mfld_lo, trade.feat_btc_dom_lo, trade.feat_mfld_lo, trade.feat_btc_dom_lo,
# Manifold audit fields
trade.mfld_market_id, trade.mfld_market_title, trade.mfld_market_url,
trade.mfld_prob_raw, trade.mfld_prob_final, trade.mfld_inverted,
trade.mfld_match_score, trade.mfld_match_reason, trade.mfld_match_status,
) )
async def save_daily_metrics(self, metrics: dict) -> None: async def save_daily_metrics(self, metrics: dict) -> None:
@@ -142,6 +152,20 @@ class Database:
""") """)
return [dict(r) for r in rows] return [dict(r) for r in rows]
async def get_open_trades_for_market(self, market_id: str) -> list[dict]:
"""Return direction, shares and net_cost for each open trade in a market.
Used by PaperExecutor.close_position() to compute the settlement
payout per direction (BUY_NO pays out when resolution = 0.0).
"""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"SELECT direction, shares, net_cost FROM trades "
"WHERE market_id = $1 AND closed_at IS NULL",
market_id,
)
return [dict(r) for r in rows]
async def close_paper_position( async def close_paper_position(
self, market_id: str, reason: str = "", resolution: Optional[float] = None self, market_id: str, reason: str = "", resolution: Optional[float] = None
) -> None: ) -> None:
@@ -149,19 +173,29 @@ class Database:
resolution: 1.0 if YES resolved, 0.0 if NO resolved, None if unknown resolution: 1.0 if YES resolved, 0.0 if NO resolved, None if unknown
(legacy closes, inversion fixes). When resolution is provided, close_pnl (legacy closes, inversion fixes). When resolution is provided, close_pnl
is computed in SQL so it matches the stored entry_price and shares exactly. is computed in SQL per row as payout net_cost — NET of fee, the single
PnL definition shared with PaperExecutor.close_position() (logs/Telegram):
BUY_YES: resolution * shares net_cost
BUY_NO: (1 resolution) * shares net_cost
paper.py aggregates payout net_cost over these same open rows, so
SUM(close_pnl) per market equals the pnl it reports exactly. The
aggregate is intentionally NOT passed in as a parameter: writing it to
every row would double-count markets with more than one open trade.
""" """
async with self._pool.acquire() as conn: async with self._pool.acquire() as conn:
# $3 is cast on every use: Postgres cannot infer the parameter type
# from a bare "$3 IS NOT NULL" and fails the prepare with
# AmbiguousParameterError otherwise.
await conn.execute(""" await conn.execute("""
UPDATE trades UPDATE trades
SET closed_at = NOW(), SET closed_at = NOW(),
close_reason = $2, close_reason = $2,
resolution = $3, resolution = $3::double precision,
close_pnl = CASE close_pnl = CASE
WHEN $3 IS NOT NULL AND direction = 'BUY_YES' WHEN $3::double precision IS NOT NULL AND direction = 'BUY_YES'
THEN ($3::double precision - entry_price) * shares THEN ($3::double precision * shares) - net_cost
WHEN $3 IS NOT NULL AND direction = 'BUY_NO' WHEN $3::double precision IS NOT NULL AND direction = 'BUY_NO'
THEN ((1.0 - $3::double precision) - entry_price) * shares THEN ((1.0 - $3::double precision) * shares) - net_cost
ELSE NULL ELSE NULL
END END
WHERE market_id = $1 AND closed_at IS NULL WHERE market_id = $1 AND closed_at IS NULL
@@ -218,8 +252,14 @@ class Database:
COUNT(*) AS total_trades, COUNT(*) AS total_trades,
COUNT(*) FILTER (WHERE closed_at IS NULL) AS open_count, COUNT(*) FILTER (WHERE closed_at IS NULL) AS open_count,
COUNT(*) FILTER (WHERE closed_at IS NOT NULL) AS closed_count, COUNT(*) FILTER (WHERE closed_at IS NOT NULL) AS closed_count,
-- excluded_from_metrics trades are omitted from resolved_count,
-- realized_pnl, wins_realized, and calibration_score.
-- resolved_count does NOT require final_prob: legacy trades
-- without signal data still count as resolved. Calibration
-- below keeps the final_prob requirement (it needs the
-- estimated probability to score).
COUNT(*) FILTER (WHERE resolution IS NOT NULL COUNT(*) FILTER (WHERE resolution IS NOT NULL
AND final_prob IS NOT NULL) AS resolved_count, AND (excluded_from_metrics IS NOT TRUE)) AS resolved_count,
COALESCE(SUM(net_cost) COALESCE(SUM(net_cost)
FILTER (WHERE closed_at IS NULL), 0) AS total_deployed, FILTER (WHERE closed_at IS NULL), 0) AS total_deployed,
@@ -232,15 +272,17 @@ class Database:
FILTER (WHERE closed_at IS NULL FILTER (WHERE closed_at IS NULL
AND edge_net IS NOT NULL), 0) AS unrealized_pnl_est, AND edge_net IS NOT NULL), 0) AS unrealized_pnl_est,
-- Realized PnL: closed trades with a known resolution. -- Realized PnL: admin-excluded trades omitted (close_pnl=0 by convention
-- close_pnl is computed at close time from actual resolution. -- but excluded explicitly so they don't skew the aggregate).
COALESCE(SUM(close_pnl) COALESCE(SUM(close_pnl)
FILTER (WHERE closed_at IS NOT NULL FILTER (WHERE closed_at IS NOT NULL
AND close_pnl IS NOT NULL), 0) AS realized_pnl, AND close_pnl IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)), 0) AS realized_pnl,
COUNT(*) FILTER (WHERE closed_at IS NOT NULL COUNT(*) FILTER (WHERE closed_at IS NOT NULL
AND close_pnl IS NOT NULL AND close_pnl IS NOT NULL
AND close_pnl > 0) AS wins_realized, AND close_pnl > 0
AND (excluded_from_metrics IS NOT TRUE)) AS wins_realized,
-- Calibration (Brier score transformed to higher-is-better): -- Calibration (Brier score transformed to higher-is-better):
-- 1 AVG((final_prob resolution)²) on resolved trades. -- 1 AVG((final_prob resolution)²) on resolved trades.
@@ -248,12 +290,15 @@ class Database:
-- resolution is 1.0 (YES won) or 0.0 (NO won). -- resolution is 1.0 (YES won) or 0.0 (NO won).
-- Perfect calibration → 1.0 | Random → ~0.75 | Worst → 0.0 -- Perfect calibration → 1.0 | Random → ~0.75 | Worst → 0.0
-- Returns NULL if fewer than 10 resolved trades with final_prob. -- Returns NULL if fewer than 10 resolved trades with final_prob.
-- Admin-excluded trades omitted from both threshold and average.
CASE CASE
WHEN COUNT(*) FILTER (WHERE resolution IS NOT NULL WHEN COUNT(*) FILTER (WHERE resolution IS NOT NULL
AND final_prob IS NOT NULL) >= 10 AND final_prob IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)) >= 10
THEN 1.0 - AVG((final_prob - resolution) * (final_prob - resolution)) THEN 1.0 - AVG((final_prob - resolution) * (final_prob - resolution))
FILTER (WHERE resolution IS NOT NULL FILTER (WHERE resolution IS NOT NULL
AND final_prob IS NOT NULL) AND final_prob IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE))
ELSE NULL ELSE NULL
END AS calibration_score END AS calibration_score
@@ -285,12 +330,42 @@ class Database:
return result return result
async def get_metrics_history(self, days: int = 42) -> list[dict]: async def get_metrics_history(self, days: int = 42) -> list[dict]:
"""Return the closing snapshot of each UTC day, newest day first.
metrics_daily receives one snapshot per trading cycle (~1/min), so a
plain LIMIT over raw rows would cover minutes, not days. DISTINCT ON
collapses each calendar day to its last snapshot, making `days` bound
actual days. history[0] remains the most recent snapshot overall.
"""
async with self._pool.acquire() as conn: async with self._pool.acquire() as conn:
rows = await conn.fetch( rows = await conn.fetch(
"SELECT * FROM metrics_daily ORDER BY timestamp DESC LIMIT $1", days """
SELECT DISTINCT ON (timestamp::date) *
FROM metrics_daily
ORDER BY timestamp::date DESC, timestamp DESC
LIMIT $1
""", days
) )
return [dict(r) for r in rows] return [dict(r) for r in rows]
async def get_daily_pnl_closes(self) -> list[float]:
"""Return the closing total_pnl of every observed UTC day, oldest first.
One value per calendar day with at least one metrics_daily snapshot
(the day's last snapshot, same collapse rule as get_metrics_history).
This is the input series for the Sharpe ratio: len() = days observed,
consecutive deltas = daily PnL changes.
"""
async with self._pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT DISTINCT ON (timestamp::date) total_pnl
FROM metrics_daily
ORDER BY timestamp::date ASC, timestamp DESC
"""
)
return [float(r["total_pnl"] or 0) for r in rows]
async def backfill_feature_columns(self) -> int: async def backfill_feature_columns(self) -> int:
"""Back-populate feat_*_lo for trades created before Phase 6. """Back-populate feat_*_lo for trades created before Phase 6.
@@ -360,22 +435,27 @@ class Database:
feat_fg_lo AS fval, feat_fg_lo AS fval,
edge_net, net_cost, fee_usdc, closed_at, close_pnl edge_net, net_cost, fee_usdc, closed_at, close_pnl
FROM trades WHERE feat_fg_lo IS NOT NULL FROM trades WHERE feat_fg_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
UNION ALL UNION ALL
SELECT 'mom', 0.05, feat_mom_lo, SELECT 'mom', 0.05, feat_mom_lo,
edge_net, net_cost, fee_usdc, closed_at, close_pnl edge_net, net_cost, fee_usdc, closed_at, close_pnl
FROM trades WHERE feat_mom_lo IS NOT NULL FROM trades WHERE feat_mom_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
UNION ALL UNION ALL
SELECT 'news', 0.10, feat_news_lo, SELECT 'news', 0.10, feat_news_lo,
edge_net, net_cost, fee_usdc, closed_at, close_pnl edge_net, net_cost, fee_usdc, closed_at, close_pnl
FROM trades WHERE feat_news_lo IS NOT NULL FROM trades WHERE feat_news_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
UNION ALL UNION ALL
SELECT 'mfld', 0.10, feat_mfld_lo, SELECT 'mfld', 0.10, feat_mfld_lo,
edge_net, net_cost, fee_usdc, closed_at, close_pnl edge_net, net_cost, fee_usdc, closed_at, close_pnl
FROM trades WHERE feat_mfld_lo IS NOT NULL FROM trades WHERE feat_mfld_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
UNION ALL UNION ALL
SELECT 'btc_dom', 0.05, feat_btc_dom_lo, SELECT 'btc_dom', 0.05, feat_btc_dom_lo,
edge_net, net_cost, fee_usdc, closed_at, close_pnl edge_net, net_cost, fee_usdc, closed_at, close_pnl
FROM trades WHERE feat_btc_dom_lo IS NOT NULL FROM trades WHERE feat_btc_dom_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
) )
SELECT SELECT
feature, feature,
@@ -459,6 +539,7 @@ class Database:
) AS dominant ) AS dominant
FROM trades FROM trades
WHERE feat_fg_lo IS NOT NULL WHERE feat_fg_lo IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
) )
SELECT SELECT
COALESCE(dominant, 'none') AS dominant_feature, COALESCE(dominant, 'none') AS dominant_feature,
@@ -493,6 +574,285 @@ class Database:
return result return result
async def save_manifold_audit(
self,
audit_id: str,
poly_market_id: str,
poly_question: str,
search_query: str,
mfld_market_id: Optional[str],
mfld_market_title: Optional[str],
mfld_market_url: Optional[str],
prob_raw: Optional[float],
prob_final: Optional[float],
inverted: bool,
match_score: Optional[float],
match_reason: Optional[str],
match_status: str,
poly_outcome_type: Optional[str] = None,
mfld_outcome_type: Optional[str] = None,
matcher_version: Optional[str] = None,
) -> None:
async with self._pool.acquire() as conn:
await conn.execute("""
INSERT INTO manifold_match_audit (
id, poly_market_id, poly_question, search_query,
mfld_market_id, mfld_market_title, mfld_market_url,
prob_raw, prob_final, inverted,
match_score, match_reason, match_status, used_in_trade,
poly_outcome_type, mfld_outcome_type, matcher_version
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,FALSE,$14,$15,$16)
""",
audit_id, poly_market_id, poly_question, search_query,
mfld_market_id, mfld_market_title, mfld_market_url,
prob_raw, prob_final, inverted,
match_score, match_reason, match_status,
poly_outcome_type, mfld_outcome_type, matcher_version,
)
async def get_manifold_cooldown(self, poly_market_id: str) -> Optional[dict]:
"""Return the cooldown row for a market, or None if no cooldown is recorded.
The caller decides whether the cooldown is still active by comparing
now() against retry_after (see bayesian.evaluate()).
"""
async with self._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT poly_market_id, last_evaluated_at, last_status, "
"retry_after, cooldown_reason FROM manifold_eval_cooldown "
"WHERE poly_market_id = $1",
poly_market_id,
)
return dict(row) if row else None
async def upsert_manifold_cooldown(
self,
poly_market_id: str,
last_status: str,
retry_after,
cooldown_reason: Optional[str] = None,
) -> None:
"""Insert or refresh the cooldown for a market after a real evaluation.
last_evaluated_at is stamped server-side with now(); retry_after is the
caller-computed earliest re-evaluation time.
"""
async with self._pool.acquire() as conn:
await conn.execute("""
INSERT INTO manifold_eval_cooldown (
poly_market_id, last_evaluated_at, last_status,
retry_after, cooldown_reason
) VALUES ($1, now(), $2, $3, $4)
ON CONFLICT (poly_market_id) DO UPDATE SET
last_evaluated_at = now(),
last_status = EXCLUDED.last_status,
retry_after = EXCLUDED.retry_after,
cooldown_reason = EXCLUDED.cooldown_reason
""", poly_market_id, last_status, retry_after, cooldown_reason)
async def mark_manifold_audit_used(self, audit_id: str) -> None:
async with self._pool.acquire() as conn:
await conn.execute(
"UPDATE manifold_match_audit SET used_in_trade = TRUE WHERE id = $1",
audit_id,
)
async def get_manifold_matches(self, limit: int = 50) -> dict:
"""Manifold match audit, with summary split by matcher version.
The summary separates the current matcher (MANIFOLD_MATCHER_VERSION) from
all-time totals and from legacy pre-outcome-guard records, whose accepted
matches would now be rejected by the outcome-compatibility guard and so
must not be conflated with current-version stats.
"""
async with self._pool.acquire() as conn:
current = await conn.fetchrow("""
SELECT
COUNT(*) FILTER (WHERE match_status = 'accepted') AS total_accepted,
COUNT(*) FILTER (WHERE match_status = 'rejected') AS total_rejected,
COUNT(*) FILTER (WHERE match_status = 'no_results') AS total_no_results,
AVG(match_score) FILTER (WHERE match_status = 'accepted') AS avg_match_score,
COUNT(*) FILTER (WHERE used_in_trade = TRUE) AS used_in_trade
FROM manifold_match_audit
WHERE matcher_version = $1
""", MANIFOLD_MATCHER_VERSION)
all_time = await conn.fetchrow("""
SELECT
COUNT(*) FILTER (WHERE match_status = 'accepted') AS total_accepted,
COUNT(*) FILTER (WHERE match_status = 'rejected') AS total_rejected,
COUNT(*) FILTER (WHERE match_status = 'no_results') AS total_no_results
FROM manifold_match_audit
""")
legacy = await conn.fetchrow("""
SELECT COUNT(*) AS accepted_without_outcome_type
FROM manifold_match_audit
WHERE matcher_version = 'legacy_pre_outcome_guard'
AND match_status = 'accepted'
""")
unique_markets = await conn.fetchrow("""
SELECT
COUNT(DISTINCT poly_market_id) AS evaluated,
COUNT(DISTINCT poly_market_id) FILTER (
WHERE match_status = 'accepted'
AND matcher_version = $1
) AS accepted
FROM manifold_match_audit
""", MANIFOLD_MATCHER_VERSION)
mfld_dominated = await conn.fetchrow("""
SELECT COUNT(*) AS cnt FROM trades
WHERE (excluded_from_metrics IS NOT TRUE)
AND mfld_match_status = 'accepted'
AND feat_mfld_lo IS NOT NULL
AND ABS(feat_mfld_lo) > 0.0001
AND ABS(feat_mfld_lo) > ABS(COALESCE(feat_fg_lo, 0))
AND ABS(feat_mfld_lo) > ABS(COALESCE(feat_mom_lo, 0))
AND ABS(feat_mfld_lo) > ABS(COALESCE(feat_news_lo, 0))
AND ABS(feat_mfld_lo) > ABS(COALESCE(feat_btc_dom_lo, 0))
""")
rows = await conn.fetch(
"SELECT * FROM manifold_match_audit ORDER BY timestamp DESC LIMIT $1",
limit,
)
return {
"summary": {
"current_version": {
"version": MANIFOLD_MATCHER_VERSION,
"total_accepted": int(current["total_accepted"] or 0),
"total_rejected": int(current["total_rejected"] or 0),
"total_no_results": int(current["total_no_results"] or 0),
"avg_match_score": _f(current["avg_match_score"]),
"used_in_trade": int(current["used_in_trade"] or 0),
},
"all_time": {
"total_accepted": int(all_time["total_accepted"] or 0),
"total_rejected": int(all_time["total_rejected"] or 0),
"total_no_results": int(all_time["total_no_results"] or 0),
},
"legacy": {
"accepted_without_outcome_type":
int(legacy["accepted_without_outcome_type"] or 0),
},
"trades_dominated_by_mfld": int(mfld_dominated["cnt"] or 0),
"unique_markets": {
"evaluated": int(unique_markets["evaluated"] or 0),
"accepted": int(unique_markets["accepted"] or 0),
"coverage_rate": (
float(unique_markets["accepted"]) / float(unique_markets["evaluated"])
if unique_markets["evaluated"] else None
),
},
},
"recent_matches": [dict(r) for r in rows],
}
async def get_manifold_coverage_by_category(self) -> dict:
"""Manifold coverage by semantic market category, counted by UNIQUE market.
Base table is manifold_match_audit filtered to the current matcher
(v3_outcome_guard). Each poly_market_id is collapsed to one row first, so
a market is counted once regardless of how many audit attempts or trades it
has — this measures coverage, not retry volume.
Category is inferred from the market's trade family_key when available, else
from poly_question (LEFT JOIN: audited markets that never produced a trade
are kept). Buckets (accepted/rejected/no_results) are not mutually exclusive
at the market level — a market that was no_results then later rejected counts
in both — matching COUNT(DISTINCT CASE WHEN status=...) semantics.
"""
async with self._pool.acquire() as conn:
rows = await conn.fetch("""
WITH audit AS (
SELECT
poly_market_id,
MAX(poly_question) AS poly_question,
bool_or(match_status = 'accepted') AS has_accepted,
bool_or(match_status = 'rejected') AS has_rejected,
bool_or(match_status = 'no_results') AS has_no_results
FROM manifold_match_audit
WHERE matcher_version = 'v3_outcome_guard'
GROUP BY poly_market_id
),
fam AS (
SELECT market_id, MAX(family_key) AS family_key
FROM trades
GROUP BY market_id
),
categorized AS (
SELECT
a.has_accepted, a.has_rejected, a.has_no_results,
CASE
WHEN f.family_key ILIKE '%gubernatorial%' THEN 'gubernatorial'
WHEN f.family_key ILIKE '%mayoral%' THEN 'mayoral'
WHEN f.family_key ILIKE '%senate%' THEN 'senate'
WHEN f.family_key ILIKE '%republican%' THEN 'primary-republican'
WHEN f.family_key ILIKE '%democrat%' THEN 'primary-democrat'
WHEN f.family_key ILIKE '%openai%'
OR f.family_key ILIKE '%nvidia%'
OR f.family_key ILIKE '%anthropic%' THEN 'big-tech'
-- family_key NULL or unmatched → infer from question
WHEN a.poly_question ILIKE '%governor%'
OR a.poly_question ILIKE '%gubernatorial%' THEN 'gubernatorial'
WHEN a.poly_question ILIKE '%mayor%'
OR a.poly_question ILIKE '%mayoral%' THEN 'mayoral'
WHEN a.poly_question ILIKE '%senate%' THEN 'senate'
WHEN a.poly_question ILIKE '%republican primary%' THEN 'primary-republican'
WHEN a.poly_question ILIKE '%democratic primary%'
OR a.poly_question ILIKE '%democrat primary%' THEN 'primary-democrat'
WHEN a.poly_question ILIKE '%openai%'
OR a.poly_question ILIKE '%nvidia%'
OR a.poly_question ILIKE '%anthropic%' THEN 'big-tech'
WHEN a.poly_question ILIKE '%russia%'
OR a.poly_question ILIKE '%ukraine%'
OR a.poly_question ILIKE '%israel%'
OR a.poly_question ILIKE '%ceasefire%'
OR a.poly_question ILIKE '%military%' THEN 'geopolitics'
ELSE 'other'
END AS category
FROM audit a
LEFT JOIN fam f ON f.market_id = a.poly_market_id
)
SELECT
category,
COUNT(*) AS unique_evaluated,
COUNT(*) FILTER (WHERE has_accepted) AS unique_accepted,
COUNT(*) FILTER (WHERE has_rejected) AS unique_rejected,
COUNT(*) FILTER (WHERE has_no_results) AS unique_no_results
FROM categorized
GROUP BY category
ORDER BY unique_evaluated DESC
""")
coverage_by_category = []
total_evaluated = 0
total_accepted = 0
categories_with_coverage = 0
for r in rows:
evaluated = int(r["unique_evaluated"] or 0)
accepted = int(r["unique_accepted"] or 0)
total_evaluated += evaluated
total_accepted += accepted
if accepted > 0:
categories_with_coverage += 1
coverage_by_category.append({
"category": r["category"],
"unique_evaluated": evaluated,
"unique_accepted": accepted,
"unique_rejected": int(r["unique_rejected"] or 0),
"unique_no_results": int(r["unique_no_results"] or 0),
"coverage_rate": (accepted / evaluated) if evaluated else None,
})
return {
"coverage_by_category": coverage_by_category,
"summary": {
"total_unique_evaluated": total_evaluated,
"total_unique_accepted": total_accepted,
"overall_coverage_rate": (total_accepted / total_evaluated) if total_evaluated else None,
"categories_with_coverage": categories_with_coverage,
},
}
def _f(v) -> Optional[float]: def _f(v) -> Optional[float]:
"""None-safe float cast for asyncpg Decimal/None values.""" """None-safe float cast for asyncpg Decimal/None values."""
return float(v) if v is not None else None return float(v) if v is not None else None
+227 -75
View File
@@ -2,34 +2,49 @@
Manifold Markets client — cross-platform prediction market probability signals. Manifold Markets client — cross-platform prediction market probability signals.
For each Polymarket question, searches Manifold for a matching binary market For each Polymarket question, searches Manifold for a matching binary market
by keyword overlap and returns its probability as a calibration signal. by keyword overlap and returns a ManifoldMatchResult with full audit metadata.
Inversion guard: if the Manifold market's winning side (Republican / Democrat) Match threshold: >= 0.40 Jaccard overlap (raised from 0.25 for stricter semantics).
is the complement of the Polymarket question's winning side, the probability is
automatically inverted (1 - prob). This prevents "Democrats win Ohio governor"
from consuming the probability of a Manifold market titled "Republicans win Ohio
governor" without adjustment.
Rejection guard: if the match score falls below _MATCH_THRESHOLD the market is Outcome compatibility guard (conservative):
rejected, even if inversion would otherwise apply. All decisions are logged at - Conditional Manifold markets ("If X, will Y?" / "Conditional on..." / "Assuming..."
INFO so they can be audited per-cycle. / "Given that..." / mid-sentence "...if X is nominated, will...") are rejected:
a premise-gated question is not equivalent to a direct outcome question even when
token overlap is high. reason='conditional_market'.
- Each side is classified into an outcome_type (nomination | primary_win |
general_win | conditional | other). Matches with differing outcome_type — or any
conditional side — are rejected. reason='outcome_mismatch: poly=... manifold=...'.
Cache TTL: 30 minutes (Manifold markets move slowly vs our 60 s cycle). Inversion guard (conservative):
Match threshold: >= 0.25 keyword overlap ratio between significant tokens. - If Polymarket question names a party (democrat/republican) AND the matched
Manifold market names the OPPOSITE party → invert probability (1 - prob).
- If Polymarket question names a party AND Manifold market has NO party keyword
→ reject with reason='ambiguous_inversion' (can't determine if inversion applies).
- All other cases: no inversion, accept if score >= threshold.
- Ante duda, reject.
Cache TTL: 30 minutes.
""" """
import logging import logging
import re import re
import time import time
from dataclasses import dataclass, field
from typing import Optional from typing import Optional
import httpx import httpx
# Version tag for every audit record this matcher produces. Persisted to
# manifold_match_audit.matcher_version so metrics can isolate current-version
# stats from legacy/pre-versioning records. Do NOT change this value once set;
# bump to a new string only when matcher semantics change materially.
MANIFOLD_MATCHER_VERSION = "v3_outcome_guard"
MANIFOLD_API = "https://api.manifold.markets/v0" MANIFOLD_API = "https://api.manifold.markets/v0"
CACHE_TTL_SEC = 1800 # 30 minutes CACHE_TTL_SEC = 1800 # 30 minutes
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
_MATCH_THRESHOLD = 0.25 _MATCH_THRESHOLD = 0.40 # raised from 0.25
_STOP_WORDS = frozenset([ _STOP_WORDS = frozenset([
"will", "the", "a", "an", "is", "are", "was", "were", "be", "been", "will", "the", "a", "an", "is", "are", "was", "were", "be", "been",
@@ -43,9 +58,24 @@ _STOP_WORDS = frozenset([
"before", "during", "until", "against", "between", "through", "before", "during", "until", "against", "between", "through",
]) ])
# Mutually exclusive political parties used for complement detection
_REPUBLICAN_WORDS = frozenset(["republican", "republicans", "gop"]) _REPUBLICAN_WORDS = frozenset(["republican", "republicans", "gop"])
_DEMOCRAT_WORDS = frozenset(["democrat", "democrats", "democratic"]) _DEMOCRAT_WORDS = frozenset(["democrat", "democrats", "democratic"])
@dataclass
class ManifoldMatchResult:
status: str # 'accepted' | 'rejected' | 'no_results'
prob_final: Optional[float] = None
prob_raw: Optional[float] = None
market_id: Optional[str] = None # Manifold internal market ID
market_title: Optional[str] = None
market_url: Optional[str] = None
match_score: Optional[float] = None # 0-1 Jaccard
match_reason: Optional[str] = None # human-readable explanation
inverted: bool = False
search_query: str = ""
poly_outcome_type: Optional[str] = None # nomination|primary_win|general_win|conditional|other
mfld_outcome_type: Optional[str] = None
def _significant_words(text: str) -> set[str]: def _significant_words(text: str) -> set[str]:
@@ -69,27 +99,53 @@ def _detect_party(text: str) -> Optional[str]:
return None return None
def _best_match_with_audit( # ── Conditional-market detection (Task 1) ──────────────────────────────────────
poly_question: str, # A market is "conditional" when its resolution is gated on a premise rather than
results: list[dict], # asking the outcome directly (e.g. "If X is the nominee, will he win?"). Such a
) -> tuple[Optional[dict], float, bool]: # market is NOT equivalent to a direct outcome question even with high token overlap.
""" _CONDITIONAL_PREFIXES = ("if ", "conditional on", "assuming ", "given that")
Find the best-matching open binary Manifold market. # " if <clause>," — a mid-sentence conditional clause closed by a comma.
_CONDITIONAL_CLAUSE_RE = re.compile(r"\sif\s[^,]*,")
Returns (match, score, needs_inversion):
match — best result dict, or None if below threshold def _is_conditional(text: str) -> bool:
score — keyword overlap score of best candidate (even if rejected) """True if the question is phrased conditionally (premise-gated)."""
needs_inversion — True when Manifold market favours the OPPOSITE party/side t = (text or "").strip().lower()
to the Polymarket question (probability should be 1 - prob) if t.startswith(_CONDITIONAL_PREFIXES):
return True
return bool(_CONDITIONAL_CLAUSE_RE.search(t))
def _classify_outcome(text: str) -> str:
""" """
Coarse classification of what a question is *asking about*, used to reject
matches whose outcomes are not equivalent even when tokens overlap.
Returns one of: nomination | primary_win | general_win | conditional | other.
Order matters: conditional is checked first (premise-gated), then nomination
(which subsumes "primary nominee"), then primary, then general election.
"""
t = (text or "").strip().lower()
if t.startswith(_CONDITIONAL_PREFIXES):
return "conditional"
if any(k in t for k in ("nominee", "nominated", "nomination")):
return "nomination"
if any(k in t for k in ("primary", "win the primary", "first round")):
return "primary_win"
if any(k in t for k in ("win the election", "win the race",
"win the seat", "general election")):
return "general_win"
return "other"
def _find_best_candidate(poly_question: str, results: list[dict]) -> tuple[Optional[dict], float]:
"""Find the highest-scoring open binary Manifold market by Jaccard overlap."""
poly_words = _significant_words(poly_question) poly_words = _significant_words(poly_question)
poly_party = _detect_party(poly_question)
if not poly_words: if not poly_words:
return None, 0.0, False return None, 0.0
best_score = 0.0 best_score = 0.0
best: Optional[dict] = None best: Optional[dict] = None
best_needs_inv = False
for result in results: for result in results:
if result.get("outcomeType") != "BINARY": if result.get("outcomeType") != "BINARY":
@@ -106,18 +162,14 @@ def _best_match_with_audit(
if score > best_score: if score > best_score:
best_score = score best_score = score
best = result best = result
manifold_party = _detect_party(title)
# Inversion is warranted only when both sides are unambiguously detected
# and they are confirmed opposites (republican ≠ democrat).
best_needs_inv = (
poly_party is not None
and manifold_party is not None
and poly_party != manifold_party
)
if best_score >= _MATCH_THRESHOLD and best is not None: return best, best_score
return best, best_score, best_needs_inv
return None, best_score, False
def _market_url(match: dict) -> Optional[str]:
slug = match.get("slug", "")
creator = match.get("creatorUsername", "")
return f"https://manifold.markets/{creator}/{slug}" if slug else None
class ManifoldClient: class ManifoldClient:
@@ -125,27 +177,32 @@ class ManifoldClient:
def __init__(self) -> None: def __init__(self) -> None:
self._client = httpx.AsyncClient(timeout=15) self._client = httpx.AsyncClient(timeout=15)
# question → (fetched_at_monotonic, probability_or_None) # question → (fetched_at_monotonic, ManifoldMatchResult)
self._cache: dict[str, tuple[float, Optional[float]]] = {} self._cache: dict[str, tuple[float, ManifoldMatchResult]] = {}
async def get_probability(self, question: str) -> Optional[float]: async def get_match(self, question: str) -> ManifoldMatchResult:
""" """
Return Manifold probability for a matching market, or None. Return a ManifoldMatchResult for the given Polymarket question.
Probability is already adjusted for party-direction inversion when status='accepted' → prob_final is set and ready to use as signal
the matched Manifold market is the complement of our question. status='rejected' → match found but failed quality/inversion check
status='no_results' → API returned no results or call failed
Full audit log is emitted at INFO for every resolved query.
""" """
now = time.monotonic() now = time.monotonic()
cached = self._cache.get(question) cached = self._cache.get(question)
if cached and (now - cached[0]) < CACHE_TTL_SEC: if cached and (now - cached[0]) < CACHE_TTL_SEC:
return cached[1] return cached[1]
poly_outcome = _classify_outcome(question)
query = _build_search_query(question) query = _build_search_query(question)
if not query: if not query:
self._cache[question] = (now, None) result = ManifoldMatchResult(
return None status="no_results", search_query="",
poly_outcome_type=poly_outcome,
)
self._cache[question] = (now, result)
return result
try: try:
resp = await self._client.get( resp = await self._client.get(
@@ -154,45 +211,140 @@ class ManifoldClient:
) )
resp.raise_for_status() resp.raise_for_status()
results = resp.json() results = resp.json()
except Exception as e: except Exception as exc:
log.warning("Manifold API error for %r: %s", question[:40], e) log.warning("Manifold API error for %r: %s", question[:40], exc)
self._cache[question] = (now, None) result = ManifoldMatchResult(
return None status="no_results", search_query=query,
poly_outcome_type=poly_outcome,
)
self._cache[question] = (now, result)
return result
match, score, needs_inv = _best_match_with_audit(question, results) if not results:
result = ManifoldMatchResult(
status="no_results", search_query=query,
poly_outcome_type=poly_outcome,
)
self._cache[question] = (now, result)
return result
if match is None: best, score = _find_best_candidate(question, results)
# ── Score threshold ───────────────────────────────────────────────────
if best is None or score < _MATCH_THRESHOLD:
reason = f"jaccard={score:.2f}<{_MATCH_THRESHOLD:.2f}"
log.info( log.info(
"Manifold no_match: %-50s | best_score=%.2f < %.2f | query=%r", "Manifold REJECTED %-50s | score=%.2f < threshold=%.2f | query=%r",
question[:50], score, _MATCH_THRESHOLD, query, question[:50], score, _MATCH_THRESHOLD, query,
) )
self._cache[question] = (now, None) result = ManifoldMatchResult(
return None status="rejected",
market_title=best.get("question") if best else None,
match_score=score if best else None,
match_reason=reason,
search_query=query,
poly_outcome_type=poly_outcome,
mfld_outcome_type=_classify_outcome(best.get("question", "")) if best else None,
)
self._cache[question] = (now, result)
return result
prob_raw = float(match["probability"]) # ── Outcome compatibility + inversion analysis (conservative) ─────────
prob_final = (1.0 - prob_raw) if needs_inv else prob_raw mfld_title = best.get("question", "")
mfld_outcome = _classify_outcome(mfld_title)
poly_party = _detect_party(question)
manifold_party = _detect_party(mfld_title)
# Build market URL from slug (best-effort; may be missing) poly_words = _significant_words(question)
slug = match.get("slug", "") mfld_words = _significant_words(mfld_title)
creator = match.get("creatorUsername", "") matched_tokens = sorted(poly_words & mfld_words)[:6]
url = f"https://manifold.markets/{creator}/{slug}" if slug else "n/a"
inverted = False
rejection_reason: Optional[str] = None
# Task 1 — conditional Manifold market is never equivalent to a direct
# outcome question, regardless of token overlap.
if _is_conditional(mfld_title):
rejection_reason = "conditional_market: manifold question is conditional"
# Task 2 — outcome types must match; any conditional side is rejected.
elif (poly_outcome == "conditional" or mfld_outcome == "conditional"
or poly_outcome != mfld_outcome):
rejection_reason = (
f"outcome_mismatch: poly={poly_outcome} manifold={mfld_outcome}"
)
elif poly_party is not None:
if manifold_party is None:
# Poly specifies a party; Manifold does not → can't verify inversion safety
rejection_reason = (
f"ambiguous_inversion: poly_party={poly_party}, mfld_party=none"
)
elif manifold_party != poly_party:
# Clear opposite parties — apply inversion
inverted = True
# manifold_party == poly_party → same party, no inversion needed
if rejection_reason is not None:
url = _market_url(best)
log.info(
"Manifold REJECTED %-50s | score=%.2f | reason=%s\n"
" mfld_title: %s",
question[:50], score, rejection_reason, best.get("question", "")[:70],
)
result = ManifoldMatchResult(
status="rejected",
market_id=str(best.get("id", "")) or None,
market_title=best.get("question"),
market_url=url,
match_score=score,
match_reason=(
f"jaccard={score:.2f}, tokens={matched_tokens}, {rejection_reason}"
),
search_query=query,
poly_outcome_type=poly_outcome,
mfld_outcome_type=mfld_outcome,
)
self._cache[question] = (now, result)
return result
# ── Accepted ──────────────────────────────────────────────────────────
prob_raw = float(best["probability"])
prob_final = (1.0 - prob_raw) if inverted else prob_raw
url = _market_url(best)
match_reason = f"jaccard={score:.2f}, tokens={matched_tokens}"
if inverted:
match_reason += f", inverted=party({poly_party}{manifold_party})"
log.info( log.info(
"Manifold %s: %-50s\n" "Manifold %s %-50s\n"
" poly_question: %s\n" " poly: %s\n"
" manifold_title: %s\n" " mfld: %s\n"
" manifold_url: %s\n" " url: %s\n"
" match_score: %.2f | prob_raw=%.3f | inverted=%s | prob_final=%.3f", " score=%.2f | raw=%.3f | inverted=%s | final=%.3f",
"MATCH_INVERTED" if needs_inv else "MATCH", "ACCEPTED_INVERTED" if inverted else "ACCEPTED ",
question[:50], question[:50],
question, question,
match.get("question", ""), best.get("question", ""),
url, url or "n/a",
score, prob_raw, needs_inv, prob_final, score, prob_raw, inverted, prob_final,
) )
self._cache[question] = (now, prob_final) result = ManifoldMatchResult(
return prob_final status="accepted",
prob_final=prob_final,
prob_raw=prob_raw,
market_id=str(best.get("id", "")) or None,
market_title=best.get("question"),
market_url=url,
match_score=score,
match_reason=match_reason,
inverted=inverted,
search_query=query,
poly_outcome_type=poly_outcome,
mfld_outcome_type=mfld_outcome,
)
self._cache[question] = (now, result)
return result
async def close(self) -> None: async def close(self) -> None:
await self._client.aclose() await self._client.aclose()
+17 -1
View File
@@ -51,7 +51,11 @@ _DATE_RE = re.compile(
r"|\bQ[1-4]\b", r"|\bQ[1-4]\b",
flags=re.IGNORECASE, flags=re.IGNORECASE,
) )
_PUNCT_RE = re.compile(r"[?!\"'.,;:()\[\]{}]") # Hyphens/dashes are GNews query operators (a leading '-' means "exclude the
# next term"), so a token like "El-Sayed" makes the API return HTTP 400. Strip
# them to spaces along with the rest of the punctuation so the query stays a
# plain keyword list. = en dash, — = em dash.
_PUNCT_RE = re.compile(r"[?!\"'.,;:()\[\]{}\-–—]")
class NewsClient: class NewsClient:
@@ -79,6 +83,18 @@ class NewsClient:
# Public API # Public API
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@property
def enabled(self) -> bool:
"""True only when a GNews API key is configured.
When False, get_sentiment() is a no-op that returns 0.0 without any
network call, so callers must skip GNews entirely — including the
per-cycle query budget accounting — instead of "spending" a query that
never reaches the API (which inflated gnews_queries_used to a phantom
5/5 while the key was missing).
"""
return bool(self._api_key)
async def get_sentiment(self, question: str) -> float: async def get_sentiment(self, question: str) -> float:
""" """
Return a sentiment score ∈ [-1.0, +1.0] for the market question. Return a sentiment score ∈ [-1.0, +1.0] for the market question.
+94
View File
@@ -211,6 +211,32 @@ class Market:
category: str = "" category: str = ""
@dataclass
class MarketResolution:
"""Resolution state of a market, from Gamma API.
resolution is the final YES outcome price: 1.0 = YES won, 0.0 = NO won.
resolved is True only when the outcome is definitive — a market that is
closed but still in UMA dispute/proposal reports resolved=False.
"""
resolved: bool
resolution: Optional[float] = None
resolved_at: Optional[datetime] = None
def _parse_resolution_timestamp(raw: Optional[str]) -> Optional[datetime]:
"""Parse Gamma timestamps: '2026-06-11 13:15:01+00' or '2026-06-11T13:15:01Z'."""
if not raw:
return None
try:
dt = datetime.fromisoformat(raw.replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
except (ValueError, TypeError):
return None
@dataclass @dataclass
class OrderBook: class OrderBook:
market_id: str market_id: str
@@ -447,6 +473,74 @@ class PolymarketClient:
) )
return markets return markets
async def get_market_resolution(self, market_id: str) -> Optional[MarketResolution]:
"""Fetch resolution state for a market by Gamma market id.
Observed Gamma API behaviour (GET /markets/{id}):
open market → closed=false, umaResolutionStatus absent
resolved market → closed=true, umaResolutionStatus="resolved",
outcomePrices='["0", "1"]' (final YES price = outcome)
unknown id → HTTP 404
Returns None on API errors (caller retries next check). A closed market
whose outcome prices are not degenerate (0/1) or whose UMA status is not
"resolved" yet (proposed/disputed) reports resolved=False — we never
settle a position on an ambiguous outcome.
"""
try:
resp = await self._client.get(f"{GAMMA_API}/markets/{market_id}")
if resp.status_code == 404:
log.warning("get_market_resolution: market %s not found (404)", market_id)
return None
resp.raise_for_status()
m = resp.json()
except httpx.HTTPError as e:
log.warning("get_market_resolution: API error for %s: %s", market_id, e)
return None
if not m.get("closed"):
return MarketResolution(resolved=False)
uma_status = (m.get("umaResolutionStatus") or "").lower()
if uma_status and uma_status != "resolved":
# Closed but UMA outcome still proposed/disputed — wait for finality
return MarketResolution(resolved=False)
raw_prices = m.get("outcomePrices", [])
if isinstance(raw_prices, str):
import json as _json
try:
raw_prices = _json.loads(raw_prices)
except ValueError:
raw_prices = []
try:
yes_final = float(raw_prices[0])
except (IndexError, TypeError, ValueError):
log.warning(
"get_market_resolution: market %s closed but outcomePrices "
"unparseable: %r", market_id, m.get("outcomePrices"),
)
return MarketResolution(resolved=False)
if yes_final >= 0.99:
resolution = 1.0
elif yes_final <= 0.01:
resolution = 0.0
else:
# Closed but prices not settled at 0/1 (partial / ambiguous outcome)
log.warning(
"get_market_resolution: market %s closed with non-binary final "
"price %.3f — not settling", market_id, yes_final,
)
return MarketResolution(resolved=False)
resolved_at = (
_parse_resolution_timestamp(m.get("closedTime"))
or _parse_resolution_timestamp(m.get("umaEndDate"))
or _parse_resolution_timestamp(m.get("endDate"))
)
return MarketResolution(resolved=True, resolution=resolution, resolved_at=resolved_at)
async def get_order_book(self, token_id: str) -> Optional[OrderBook]: async def get_order_book(self, token_id: str) -> Optional[OrderBook]:
"""Get order book for a specific token.""" """Get order book for a specific token."""
try: try:
+139 -3
View File
@@ -113,9 +113,10 @@ CREATE INDEX IF NOT EXISTS idx_trades_closed ON trades(closed_at) WHERE closed_a
-- Fix 3: market resolution and realized P&L per trade -- Fix 3: market resolution and realized P&L per trade
-- --
-- resolution: 1.0 if YES resolved, 0.0 if NO resolved, NULL if not yet settled. -- resolution: 1.0 if YES resolved, 0.0 if NO resolved, NULL if not yet settled.
-- close_pnl: realized P&L in USDC at close time. -- close_pnl: realized P&L in USDC at close time — NET of fee (payout net_cost),
-- BUY_YES: (resolution - entry_price) * shares -- the same definition PaperExecutor.close_position() reports in logs/Telegram.
-- BUY_NO: ((1 - resolution) - entry_price) * shares -- BUY_YES: resolution * shares - net_cost
-- BUY_NO: (1 - resolution) * shares - net_cost
-- NULL if closed without a known resolution (legacy closes, inversion fixes). -- NULL if closed without a known resolution (legacy closes, inversion fixes).
-- ───────────────────────────────────────────────────────────────────────────── -- ─────────────────────────────────────────────────────────────────────────────
ALTER TABLE trades ADD COLUMN IF NOT EXISTS close_pnl DOUBLE PRECISION; ALTER TABLE trades ADD COLUMN IF NOT EXISTS close_pnl DOUBLE PRECISION;
@@ -168,6 +169,103 @@ ALTER TABLE trades ADD COLUMN IF NOT EXISTS feat_btc_dom_lo DOUBLE PRECISION;
CREATE INDEX IF NOT EXISTS idx_trades_feat_fg ON trades(feat_fg_lo) WHERE feat_fg_lo IS NOT NULL; CREATE INDEX IF NOT EXISTS idx_trades_feat_fg ON trades(feat_fg_lo) WHERE feat_fg_lo IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_trades_feat_mfld ON trades(feat_mfld_lo) WHERE feat_mfld_lo IS NOT NULL; CREATE INDEX IF NOT EXISTS idx_trades_feat_mfld ON trades(feat_mfld_lo) WHERE feat_mfld_lo IS NOT NULL;
-- ─────────────────────────────────────────────────────────────────────────────
-- Manifold match audit — per-trade columns in trades
--
-- Persisted for every trade where Manifold was queried (status='accepted').
-- mfld_match_status: 'accepted' | 'rejected' | 'no_results'
-- mfld_inverted: TRUE when prob_final = 1 - prob_raw (party complement match)
-- ─────────────────────────────────────────────────────────────────────────────
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_market_id TEXT;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_market_title TEXT;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_market_url TEXT;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_prob_raw DOUBLE PRECISION;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_prob_final DOUBLE PRECISION;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_inverted BOOLEAN;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_match_score DOUBLE PRECISION;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_match_reason TEXT;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS mfld_match_status TEXT;
-- ─────────────────────────────────────────────────────────────────────────────
-- Manifold match audit table — records every Manifold query attempt
--
-- Populated for ALL queries: accepted, rejected, and no_results.
-- used_in_trade=TRUE is set after executor confirms a trade was executed.
-- poly_market_id: Market.id from the Polymarket Market dataclass (never NULL).
-- ─────────────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS manifold_match_audit (
id TEXT PRIMARY KEY,
timestamp TIMESTAMPTZ DEFAULT NOW(),
poly_market_id TEXT NOT NULL,
poly_question TEXT NOT NULL,
search_query TEXT,
mfld_market_id TEXT,
mfld_market_title TEXT,
mfld_market_url TEXT,
prob_raw DOUBLE PRECISION,
prob_final DOUBLE PRECISION,
inverted BOOLEAN DEFAULT FALSE,
match_score DOUBLE PRECISION,
match_reason TEXT,
match_status TEXT NOT NULL,
used_in_trade BOOLEAN DEFAULT FALSE,
poly_outcome_type TEXT,
mfld_outcome_type TEXT
);
CREATE INDEX IF NOT EXISTS idx_mfld_audit_timestamp ON manifold_match_audit(timestamp DESC);
CREATE INDEX IF NOT EXISTS idx_mfld_audit_status ON manifold_match_audit(match_status);
CREATE INDEX IF NOT EXISTS idx_mfld_audit_poly_mkt ON manifold_match_audit(poly_market_id);
-- Backfill outcome-type columns on pre-existing tables (idempotent).
ALTER TABLE manifold_match_audit ADD COLUMN IF NOT EXISTS poly_outcome_type TEXT;
ALTER TABLE manifold_match_audit ADD COLUMN IF NOT EXISTS mfld_outcome_type TEXT;
-- ─────────────────────────────────────────────────────────────────────────────
-- Matcher versioning — separate current-matcher metrics from legacy records
--
-- matcher_version tags each audit row with the matcher that produced it
-- (MANIFOLD_MATCHER_VERSION in bot/data/manifold.py). This lets the metrics
-- endpoint isolate current_version stats from pre-versioning records, whose
-- accepted matches would now be rejected by the outcome-compatibility guard.
--
-- Backfill is one-shot and idempotent (only touches NULL matcher_version rows):
-- * rows with no outcome types → 'legacy_pre_outcome_guard' (pre outcome-guard;
-- accepted without any outcome-type validation)
-- * rows with an outcome type → 'v2_outcome_guard_no_version' (existed between
-- the outcome-guard and this versioning; real version not persisted)
-- We tag rather than infer the exact version that wasn't recorded.
-- ─────────────────────────────────────────────────────────────────────────────
ALTER TABLE manifold_match_audit ADD COLUMN IF NOT EXISTS matcher_version TEXT;
UPDATE manifold_match_audit
SET matcher_version = 'legacy_pre_outcome_guard'
WHERE matcher_version IS NULL
AND poly_outcome_type IS NULL
AND mfld_outcome_type IS NULL;
UPDATE manifold_match_audit
SET matcher_version = 'v2_outcome_guard_no_version'
WHERE matcher_version IS NULL
AND (poly_outcome_type IS NOT NULL OR mfld_outcome_type IS NOT NULL);
CREATE INDEX IF NOT EXISTS idx_mfld_audit_version ON manifold_match_audit(matcher_version);
-- ─────────────────────────────────────────────────────────────────────────────
-- Metric exclusion — administrative closure flag
--
-- excluded_from_metrics: TRUE for trades closed for non-signal reasons
-- (bad matcher, data error, admin close). These trades are excluded from
-- win_rate, calibration_score, realized_pnl, and feature attribution.
-- exclusion_reason: free-text label for the exclusion cause.
-- e.g. 'invalid_manifold_match_legacy'
-- ─────────────────────────────────────────────────────────────────────────────
ALTER TABLE trades ADD COLUMN IF NOT EXISTS excluded_from_metrics BOOLEAN DEFAULT FALSE;
ALTER TABLE trades ADD COLUMN IF NOT EXISTS exclusion_reason TEXT;
CREATE INDEX IF NOT EXISTS idx_trades_excluded ON trades(excluded_from_metrics)
WHERE excluded_from_metrics = TRUE;
-- ───────────────────────────────────────────────────────────────────────────── -- ─────────────────────────────────────────────────────────────────────────────
-- Fix 3: extended metrics_daily columns for DB-computed metrics -- Fix 3: extended metrics_daily columns for DB-computed metrics
-- --
@@ -182,3 +280,41 @@ ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS realized_pnl DOUBLE PRE
ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS open_count INTEGER; ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS open_count INTEGER;
ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS closed_count INTEGER; ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS closed_count INTEGER;
ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS resolved_count INTEGER; ALTER TABLE metrics_daily ADD COLUMN IF NOT EXISTS resolved_count INTEGER;
-- ─────────────────────────────────────────────────────────────────────────────
-- Checkpoint alerts — one-shot and rate-limited Telegram observation alerts
--
-- fired_at: timestamp of the first fire (immutable for one-shot checkpoints)
-- last_fired_at: updated on every fire (used for rate-limiting repeatable alerts)
-- ─────────────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS checkpoint_alerts (
checkpoint_name TEXT PRIMARY KEY,
fired_at TIMESTAMPTZ NOT NULL,
last_fired_at TIMESTAMPTZ
);
-- ─────────────────────────────────────────────────────────────────────────────
-- Manifold evaluation cooldown — per-market backoff for the Manifold matcher
--
-- The trading loop re-evaluates the same ~stable set of politics/tech markets
-- every cycle (~60s). Most resolve to a stable terminal verdict (no Manifold
-- coverage, low-score, outcome mismatch, conditional market) that will not change
-- on the next cycle. Re-querying them every minute floods manifold_match_audit
-- with redundant rows and makes the metrics uninterpretable.
--
-- This table records, per poly_market_id, when the market was last evaluated and
-- the earliest time it should be evaluated again (retry_after). evaluate() in
-- bot/strategy/bayesian.py consults it BEFORE calling the matcher and skips the
-- call (and the audit write) entirely while now() < retry_after.
--
-- last_status / cooldown_reason are stored for observability only.
-- ─────────────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS manifold_eval_cooldown (
poly_market_id TEXT PRIMARY KEY,
last_evaluated_at TIMESTAMPTZ NOT NULL,
last_status TEXT NOT NULL,
retry_after TIMESTAMPTZ NOT NULL,
cooldown_reason TEXT
);
CREATE INDEX IF NOT EXISTS idx_mfld_cooldown_retry ON manifold_eval_cooldown(retry_after);
+87 -14
View File
@@ -22,6 +22,30 @@ log = logging.getLogger(__name__)
# NOTE: this is a heuristic — see COMMISSION_RATE in bayesian.py for context. # NOTE: this is a heuristic — see COMMISSION_RATE in bayesian.py for context.
POLYMARKET_FEE = 0.02 # 2% POLYMARKET_FEE = 0.02 # 2%
# Strong references to in-flight notification tasks. The event loop only
# keeps a weak reference to tasks created via create_task(), so without this
# set a pending Telegram notification could be garbage-collected before it
# runs. Tasks remove themselves from the set on completion.
_background_tasks: set[asyncio.Task] = set()
def _notify_in_background(coro) -> None:
"""Fire-and-forget a Telegram notification, keeping the task referenced."""
task = asyncio.create_task(coro)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
def cash_available(bankroll: float, total_net_cost_open: float) -> float:
"""Cash left after the net cost (fees included) of all open positions.
Single source of truth for the cash figure, shared by
PaperExecutor.initialize() and the /api/summary endpoint so both always
report the same number for the same DB state.
total_net_cost_open comes from Database.get_open_position_data().
"""
return max(0.0, bankroll - total_net_cost_open)
@dataclass @dataclass
class Trade: class Trade:
@@ -57,6 +81,16 @@ class Trade:
feat_news_lo: float = 0.0 feat_news_lo: float = 0.0
feat_mfld_lo: float = 0.0 feat_mfld_lo: float = 0.0
feat_btc_dom_lo: float = 0.0 feat_btc_dom_lo: float = 0.0
# ── Manifold match audit ──────────────────────────────────────────────────
mfld_market_id: Optional[str] = None
mfld_market_title: Optional[str] = None
mfld_market_url: Optional[str] = None
mfld_prob_raw: Optional[float] = None
mfld_prob_final: Optional[float] = None
mfld_inverted: bool = False
mfld_match_score: Optional[float] = None
mfld_match_reason: Optional[str] = None
mfld_match_status: Optional[str] = None
def __str__(self) -> str: def __str__(self) -> str:
return ( return (
@@ -98,7 +132,7 @@ class PaperExecutor:
positions_value = sum(positions_size.values()) positions_value = sum(positions_size.values())
self._portfolio.positions = positions_size self._portfolio.positions = positions_size
self._portfolio.cash = max(0.0, self._portfolio.cash - total_net_cost) self._portfolio.cash = cash_available(self._portfolio.cash, total_net_cost)
total_value = self._portfolio.cash + positions_value total_value = self._portfolio.cash + positions_value
exposure_pct = positions_value / total_value if total_value > 0 else 0.0 exposure_pct = positions_value / total_value if total_value > 0 else 0.0
@@ -176,6 +210,16 @@ class PaperExecutor:
feat_news_lo=order.feat_news_lo, feat_news_lo=order.feat_news_lo,
feat_mfld_lo=order.feat_mfld_lo, feat_mfld_lo=order.feat_mfld_lo,
feat_btc_dom_lo=order.feat_btc_dom_lo, feat_btc_dom_lo=order.feat_btc_dom_lo,
# Manifold audit
mfld_market_id=order.mfld_market_id,
mfld_market_title=order.mfld_market_title,
mfld_market_url=order.mfld_market_url,
mfld_prob_raw=order.mfld_prob_raw,
mfld_prob_final=order.mfld_prob_final,
mfld_inverted=order.mfld_inverted,
mfld_match_score=order.mfld_match_score,
mfld_match_reason=order.mfld_match_reason,
mfld_match_status=order.mfld_match_status,
) )
# Update paper portfolio # Update paper portfolio
@@ -185,7 +229,7 @@ class PaperExecutor:
# Persist to DB # Persist to DB
await self._db.save_trade(trade) await self._db.save_trade(trade)
asyncio.create_task( _notify_in_background(
telegram.trade_opened(trade.question, trade.direction, trade.size_usdc, trade.edge_net) telegram.trade_opened(trade.question, trade.direction, trade.size_usdc, trade.edge_net)
) )
@@ -206,7 +250,7 @@ class PaperExecutor:
"LEGACY_CLOSE market=%s | returned $%.2f to cash | %s", "LEGACY_CLOSE market=%s | returned $%.2f to cash | %s",
market_id, cost, reason[:80], market_id, cost, reason[:80],
) )
asyncio.create_task( _notify_in_background(
telegram.trade_legacy_closed(question or market_id, cost, reason) telegram.trade_legacy_closed(question or market_id, cost, reason)
) )
return cost return cost
@@ -215,24 +259,53 @@ class PaperExecutor:
"""Close a paper position after market resolution. """Close a paper position after market resolution.
resolution: 1.0 if YES won, 0.0 if NO won. resolution: 1.0 if YES won, 0.0 if NO won.
Persists resolution and close_pnl to DB (computed via SQL from stored Settlement payout per trade:
entry_price and shares). Returns approximate P&L for logging. BUY_YES: shares * resolution
BUY_NO: shares * (1 - resolution)
pnl = payout - net_cost.
Persists resolution and close_pnl to DB. Returns realized P&L for
logging, or None if no position is open.
""" """
if market_id not in self._portfolio.positions: if market_id not in self._portfolio.positions:
return None return None
position_cost = self._portfolio.positions.pop(market_id) position_cost = self._portfolio.positions[market_id]
self._portfolio.cash += position_cost * resolution # pay out winnings open_trades = await self._db.get_open_trades_for_market(market_id)
if open_trades:
payout = sum(
float(t["shares"])
* (resolution if t["direction"] == "BUY_YES" else 1.0 - resolution)
for t in open_trades
)
net_cost = sum(float(t["net_cost"]) for t in open_trades)
pnl = payout - net_cost
else:
# In-memory position with no open DB trades: direction/shares are
# unknown, so settle at break-even instead of guessing the payout.
log.warning(
"close_position: no open DB trades for market %s"
"settling at break-even", market_id,
)
payout = position_cost
pnl = 0.0
# Persist first, mutate memory after: if the DB write fails, the
# in-memory portfolio must keep the position so the next resolution
# check can retry the close.
await self._db.close_paper_position( await self._db.close_paper_position(
market_id, market_id,
reason=f"market_resolved resolution={resolution:.1f}", reason="resolved",
resolution=resolution, resolution=resolution,
) )
approx_pnl = position_cost * resolution - position_cost
log.info("Closed position in %s, resolution=%.1f", market_id, resolution) self._portfolio.positions.pop(market_id)
asyncio.create_task( self._portfolio.cash += payout
telegram.trade_closed(question or market_id, approx_pnl) log.info(
"Closed position in %s, resolution=%.1f payout=$%.2f pnl=%+.2f",
market_id, resolution, payout, pnl,
) )
# Approximate PnL: settlement value minus cost. Exact value is in close_pnl. _notify_in_background(
return approx_pnl telegram.trade_closed(question or market_id, pnl)
)
return pnl
+115 -44
View File
@@ -11,21 +11,81 @@ from bot.data.polymarket import PolymarketClient, Market, market_family_key
from bot.data.external import ExternalDataClient from bot.data.external import ExternalDataClient
from bot.data.news import NewsClient from bot.data.news import NewsClient
from bot.data.manifold import ManifoldClient from bot.data.manifold import ManifoldClient
from bot.strategy.bayesian import BayesianStrategy, gnews_priority, MAX_NEWS_QUERIES_PER_CYCLE from bot.strategy.bayesian import (
BayesianStrategy,
gnews_priority,
MAX_NEWS_QUERIES_PER_CYCLE,
MANIFOLD_SIGNAL_ENABLED,
)
from bot.risk.manager import RiskManager from bot.risk.manager import RiskManager
from bot.executor.paper import PaperExecutor from bot.executor.paper import PaperExecutor
from bot.metrics.tracker import MetricsTracker from bot.metrics.tracker import MetricsTracker
from bot.data.db import Database from bot.data.db import Database
from bot.notify.checkpoints import CheckpointMonitor
logging.basicConfig( logging.basicConfig(
level=logging.INFO, level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s", format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
) )
# httpx logs every request URL at INFO, and the GNews URL carries the API key as
# a `?token=` query param — that would leak GNEWS_API_KEY in plaintext into the
# pod logs. Raise httpx/httpcore to WARNING so request URLs never reach INFO.
# The bot's own GNews log lines only print the sanitised query, not the token.
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
log = logging.getLogger("bot.main") log = logging.getLogger("bot.main")
PAPER_MODE = os.getenv("PAPER_MODE", "true").lower() == "true" PAPER_MODE = os.getenv("PAPER_MODE", "true").lower() == "true"
PAPER_BANKROLL = float(os.getenv("PAPER_BANKROLL", "10000")) PAPER_BANKROLL = float(os.getenv("PAPER_BANKROLL", "10000"))
# Check open positions for market resolution every N trading cycles (~N minutes
# at the 60s cycle cadence). Keeps Gamma API load at ~1 request per open
# position per 10 minutes.
RESOLUTION_CHECK_INTERVAL = 10
async def check_resolutions(
poly: PolymarketClient,
executor: PaperExecutor,
db: Database,
) -> None:
"""Detect resolved markets and settle their open paper positions.
For each open position, asks the Gamma API whether the market resolved.
On a definitive resolution, PaperExecutor.close_position() settles the
payout, persists close_reason='resolved' + resolution + close_pnl, and
sends the Telegram notification.
"""
positions = await db.get_open_position_details()
checked = 0
resolved = 0
for pos in positions:
market_id = str(pos["market_id"])
try:
res = await poly.get_market_resolution(market_id)
except Exception as exc:
log.warning("Resolution check failed for market %s: %s", market_id, exc)
continue
checked += 1
if res is None or not res.resolved or res.resolution is None:
continue
try:
pnl = await executor.close_position(
market_id, res.resolution, question=pos.get("question") or "",
)
except Exception as exc:
log.error("Failed to close resolved market %s: %s", market_id, exc)
continue
resolved += 1
log.info(
"MARKET_RESOLVED market_id=%s resolution=%.1f pnl=%s | %s",
market_id,
res.resolution,
f"{pnl:+.2f}" if pnl is not None else "n/a",
(pos.get("question") or "")[:60],
)
log.info("Resolution check: %d positions checked, %d resolved", checked, resolved)
async def run_trading_loop( async def run_trading_loop(
poly: PolymarketClient, poly: PolymarketClient,
@@ -38,9 +98,23 @@ async def run_trading_loop(
) -> None: ) -> None:
"""Main trading loop — runs every 60 seconds.""" """Main trading loop — runs every 60 seconds."""
log.info("Trading loop started. PAPER_MODE=%s", PAPER_MODE) log.info("Trading loop started. PAPER_MODE=%s", PAPER_MODE)
checkpoint_monitor = CheckpointMonitor()
cycle_count = 0
while True: while True:
try: try:
cycle_count += 1
# 0. Resolution detector — every RESOLUTION_CHECK_INTERVAL cycles,
# settle paper positions whose market resolved on Polymarket.
# Runs before evaluation so freed cash/families are usable this cycle.
if (
PAPER_MODE
and isinstance(executor, PaperExecutor)
and cycle_count % RESOLUTION_CHECK_INTERVAL == 0
):
await check_resolutions(poly, executor, db)
# 1. Fetch active markets (90-day window) # 1. Fetch active markets (90-day window)
markets = await poly.get_active_markets() markets = await poly.get_active_markets()
log.info("Found %d active markets", len(markets)) log.info("Found %d active markets", len(markets))
@@ -136,11 +210,16 @@ async def run_trading_loop(
# 7. Execute (paper) # 7. Execute (paper)
trade = await executor.execute(order) trade = await executor.execute(order)
if trade: if trade:
await metrics.record_trade(trade)
log.info("Trade executed: %s", trade) log.info("Trade executed: %s", trade)
# Block this family for the rest of the cycle (Phase 2) # Block this family for the rest of the cycle (Phase 2)
occupied_families.add(signal.family_key) occupied_families.add(signal.family_key)
cycle_trades += 1 cycle_trades += 1
# Mark manifold audit record as used in this trade
if signal.mfld_audit_id:
try:
await db.mark_manifold_audit_used(signal.mfld_audit_id)
except Exception as exc:
log.warning("Failed to mark manifold audit used: %s", exc)
# 8. [CYCLE SUMMARY] — one block per cycle, stable format for grep/compare # 8. [CYCLE SUMMARY] — one block per cycle, stable format for grep/compare
stats = strategy.get_cycle_stats() stats = strategy.get_cycle_stats()
@@ -152,7 +231,17 @@ async def run_trading_loop(
if denom == 0: if denom == 0:
return "0% (0/0)" return "0% (0/0)"
return f"{n * 100 // denom}% ({n}/{denom})" return f"{n * 100 // denom}% ({n}/{denom})"
gnews_cap = strategy._news_queries_this_cycle # already updated by reset below
# The accepted/rejected counters only increment on the active-signal
# path, so with the signal disabled they always print 0/0 — say
# "disabled" instead of pretending the matcher found nothing.
if MANIFOLD_SIGNAL_ENABLED:
manifold_summary = (
f" manifold_matches_accepted: {stats['manifold_matches_accepted']}\n"
f" manifold_matches_rejected: {stats['manifold_matches_rejected']}"
)
else:
manifold_summary = " manifold_signal: disabled"
log.info( log.info(
"[CYCLE SUMMARY]\n" "[CYCLE SUMMARY]\n"
@@ -170,9 +259,7 @@ async def run_trading_loop(
" gnews_queries_used: %d/%d\n" " gnews_queries_used: %d/%d\n"
" reentry_guard_blocked: %d\n" " reentry_guard_blocked: %d\n"
" legacy_incomplete_seen: %d\n" " legacy_incomplete_seen: %d\n"
" family_conflicts_prevented: %d\n" "%s",
" manifold_matches_accepted: %d\n"
" manifold_matches_rejected: %d",
n_total, n_total,
n_uncertainty, n_uncertainty,
stats["max_edge_gross"], stats["max_edge_gross"],
@@ -187,14 +274,23 @@ async def run_trading_loop(
stats["gnews_queries_used"], MAX_NEWS_QUERIES_PER_CYCLE, stats["gnews_queries_used"], MAX_NEWS_QUERIES_PER_CYCLE,
reentry_guard_count, reentry_guard_count,
legacy_incomplete_count, legacy_incomplete_count,
stats["skip_family"], manifold_summary,
stats["manifold_matches_accepted"],
stats["manifold_matches_rejected"],
) )
# 9. Update daily metrics # 9. Update daily metrics
await metrics.update_daily_summary() await metrics.update_daily_summary()
# 10. Checkpoint alerts — one-shot / rate-limited Telegram notifications
current_portfolio = executor.get_portfolio()
try:
await checkpoint_monitor.check_all(
db,
exposure_pct=current_portfolio.exposure_pct,
exposure_cap_pct=risk.max_exposure_pct,
)
except Exception as exc:
log.warning("checkpoint_monitor.check_all failed: %s", exc)
except Exception as e: except Exception as e:
log.error("Error in trading loop: %s", e, exc_info=True) log.error("Error in trading loop: %s", e, exc_info=True)
@@ -204,14 +300,17 @@ async def run_trading_loop(
async def run_legacy_scan( async def run_legacy_scan(
db: Database, db: Database,
markets: list, markets: list,
manifold: ManifoldClient,
executor: PaperExecutor, executor: PaperExecutor,
paper_mode: bool, paper_mode: bool,
) -> None: ) -> None:
""" """
One-time startup scan: re-key all open DB positions with the current One-time startup scan: re-key all open DB positions with the current
market_family_key() logic, detect contradictions, re-validate Manifold market_family_key() logic, detect family conflicts, and report
signals, and report KEEP / REVIEW / CLOSE_RECOMMENDED per position. KEEP / REVIEW / CLOSE_RECOMMENDED per position.
Manifold is intentionally not consulted here: with
MANIFOLD_SIGNAL_ENABLED=false it is observational-only and must not
drive position closures.
In paper_mode: auto-closes all CLOSE_RECOMMENDED positions after logging. In paper_mode: auto-closes all CLOSE_RECOMMENDED positions after logging.
""" """
@@ -250,8 +349,6 @@ async def run_legacy_scan(
"family_key_old": old_fk, "family_key_old": old_fk,
"family_key_new": new_fk, "family_key_new": new_fk,
"fk_changed": new_fk != old_fk, "fk_changed": new_fk != old_fk,
"manifold_prob_new": None,
"manifold_inverted": False,
"recommendation": "legacy_incomplete" if is_legacy_incomplete else "OK", "recommendation": "legacy_incomplete" if is_legacy_incomplete else "OK",
"rec_reason": "edge_net and live market unavailable" if is_legacy_incomplete else "no family conflict", "rec_reason": "edge_net and live market unavailable" if is_legacy_incomplete else "no family conflict",
}) })
@@ -289,31 +386,7 @@ async def run_legacy_scan(
p["market_id"], p["family_key_old"] or "none", p["family_key_new"], p["market_id"], p["family_key_old"] or "none", p["family_key_new"],
) )
# Step 3: Manifold re-query for positions whose family key changed # Step 3: log the full scan report (before any closures)
for p in enriched:
if p["live_market"] and p["fk_changed"]:
prob = await manifold.get_probability(p["question"])
p["manifold_prob_new"] = prob
if prob is not None:
# Detect if original trade direction conflicts with corrected Manifold signal
if prob < 0.40 and p["direction"] == "BUY_YES":
p["manifold_inverted"] = True
note = f"Manifold:{prob:.3f} contradicts BUY_YES (inversion bug confirmed)"
if p["recommendation"] in ("OK", "REVIEW"):
p["recommendation"] = "CLOSE_RECOMMENDED"
p["rec_reason"] = note
else:
p["rec_reason"] += f" | {note}"
elif prob > 0.60 and p["direction"] == "BUY_NO":
p["manifold_inverted"] = True
note = f"Manifold:{prob:.3f} contradicts BUY_NO (inversion bug confirmed)"
if p["recommendation"] in ("OK", "REVIEW"):
p["recommendation"] = "CLOSE_RECOMMENDED"
p["rec_reason"] = note
else:
p["rec_reason"] += f" | {note}"
# Step 4: log the full scan report (before any closures)
n_close = sum(1 for p in enriched if p["recommendation"] == "CLOSE_RECOMMENDED") n_close = sum(1 for p in enriched if p["recommendation"] == "CLOSE_RECOMMENDED")
n_keep = sum(1 for p in enriched if p["recommendation"] == "KEEP") n_keep = sum(1 for p in enriched if p["recommendation"] == "KEEP")
n_ok = sum(1 for p in enriched if p["recommendation"] == "OK") n_ok = sum(1 for p in enriched if p["recommendation"] == "OK")
@@ -329,7 +402,6 @@ async def run_legacy_scan(
" [%-18s] market=%-8s | dir=%-8s | edge_net=%+.3f\n" " [%-18s] market=%-8s | dir=%-8s | edge_net=%+.3f\n"
" stored_family: %s\n" " stored_family: %s\n"
" new_family: %s%s\n" " new_family: %s%s\n"
" manifold_new: %s\n"
" reason: %s", " reason: %s",
p["recommendation"], p["recommendation"],
p["market_id"], p["direction"], p["market_id"], p["direction"],
@@ -337,12 +409,11 @@ async def run_legacy_scan(
p["family_key_old"] or "none", p["family_key_old"] or "none",
p["family_key_new"], p["family_key_new"],
" [CHANGED]" if p["fk_changed"] else "", " [CHANGED]" if p["fk_changed"] else "",
f"{p['manifold_prob_new']:.3f}" if p["manifold_prob_new"] is not None else "n/a",
p["rec_reason"], p["rec_reason"],
) )
log.warning("" * 70) log.warning("" * 70)
# Step 5: auto-close in paper mode # Step 4: auto-close in paper mode
if paper_mode and n_close > 0 and isinstance(executor, PaperExecutor): if paper_mode and n_close > 0 and isinstance(executor, PaperExecutor):
log.warning("PAPER MODE: auto-closing %d CLOSE_RECOMMENDED position(s)...", n_close) log.warning("PAPER MODE: auto-closing %d CLOSE_RECOMMENDED position(s)...", n_close)
for p in enriched: for p in enriched:
@@ -375,7 +446,7 @@ async def main() -> None:
external = ExternalDataClient() external = ExternalDataClient()
news = NewsClient() news = NewsClient()
manifold = ManifoldClient() manifold = ManifoldClient()
strategy = BayesianStrategy(news=news, manifold=manifold) strategy = BayesianStrategy(news=news, manifold=manifold, db=db)
risk = RiskManager(max_position_pct=0.05, max_exposure_pct=0.30) risk = RiskManager(max_position_pct=0.05, max_exposure_pct=0.30)
executor = PaperExecutor(db=db, bankroll=PAPER_BANKROLL) if PAPER_MODE else None executor = PaperExecutor(db=db, bankroll=PAPER_BANKROLL) if PAPER_MODE else None
metrics = MetricsTracker(db=db) metrics = MetricsTracker(db=db)
@@ -395,7 +466,7 @@ async def main() -> None:
except Exception as e: except Exception as e:
log.warning("Could not fetch markets for legacy scan: %s — scan skipped", e) log.warning("Could not fetch markets for legacy scan: %s — scan skipped", e)
scan_markets = [] scan_markets = []
await run_legacy_scan(db, scan_markets, manifold, executor, PAPER_MODE) await run_legacy_scan(db, scan_markets, executor, PAPER_MODE)
try: try:
await run_trading_loop(poly, external, strategy, risk, executor, metrics, db) await run_trading_loop(poly, external, strategy, risk, executor, metrics, db)
+79
View File
@@ -0,0 +1,79 @@
"""
Sharpe ratio from the paper portfolio's daily PnL curve, with a minimum-sample gate.
The input series is the closing total_pnl of each observed UTC day
(Database.get_daily_pnl_closes). Daily returns are PnL deltas normalized by
the paper bankroll:
r_t = (pnl_t pnl_{t1}) / bankroll
Sharpe = mean(r) / sample_std(r) × 365, annualized prediction markets
resolve every calendar day, so 365 is used instead of 252 trading days.
Risk-free rate is taken as 0.
Gate: with a tiny sample (e.g. 1 resolved trade over a flat curve plus one
+299 jump) any Sharpe value is statistically meaningless artificially huge
or tiny depending on where the jump lands. So no numeric Sharpe is exposed
until BOTH minimums are met:
days observed >= MIN_DAYS_OBSERVED (30)
resolved trades >= MIN_RESOLVED_TRADES (10)
Below either minimum the value is None with status "insufficient_sample".
A perfectly flat curve (zero variance) also yields None ("zero_variance"):
Sharpe is undefined there, not infinite.
"""
from statistics import mean, stdev
from typing import Optional
MIN_DAYS_OBSERVED = 30
MIN_RESOLVED_TRADES = 10
ANNUALIZATION_DAYS = 365
SHARPE_OK = "ok"
SHARPE_INSUFFICIENT = "insufficient_sample"
SHARPE_ZERO_VARIANCE = "zero_variance"
def daily_returns(daily_pnl_closes: list[float], bankroll: float) -> list[float]:
"""Bankroll-normalized day-over-day returns from a daily PnL-close series."""
return [
(curr - prev) / bankroll
for prev, curr in zip(daily_pnl_closes, daily_pnl_closes[1:])
]
def compute_sharpe(daily_pnl_closes: list[float], bankroll: float) -> Optional[float]:
"""Annualized Sharpe of the daily PnL curve, or None if undefined.
None when there are fewer than 2 returns (need 3+ daily closes) or the
return series has zero variance. No sample-size gate here see
sharpe_with_gate() for the exposed value.
"""
returns = daily_returns(daily_pnl_closes, bankroll)
if len(returns) < 2:
return None
sd = stdev(returns)
if sd == 0:
return None
return mean(returns) / sd * ANNUALIZATION_DAYS ** 0.5
def sharpe_with_gate(
daily_pnl_closes: list[float],
bankroll: float,
resolved_count: int,
) -> tuple[Optional[float], str]:
"""Return (sharpe, status) applying the minimum-sample gate.
status: "ok" sharpe is a meaningful float
"insufficient_sample" sample below minimums, sharpe is None
"zero_variance" sample OK but flat curve, sharpe is None
"""
days_observed = len(daily_pnl_closes)
if days_observed < MIN_DAYS_OBSERVED or resolved_count < MIN_RESOLVED_TRADES:
return None, SHARPE_INSUFFICIENT
sharpe = compute_sharpe(daily_pnl_closes, bankroll)
if sharpe is None:
return None, SHARPE_ZERO_VARIANCE
return sharpe, SHARPE_OK
+14 -9
View File
@@ -15,13 +15,16 @@ win_rate Fraction of resolved closed trades with close_pnl > 0.
NULL if fewer than 5 resolved trades. NULL if fewer than 5 resolved trades.
calibration_score 1 AVG((final_prob resolution)²) on resolved trades. calibration_score 1 AVG((final_prob resolution)²) on resolved trades.
Brier score (higher = better calibration). NULL if < 10 resolved. Brier score (higher = better calibration). NULL if < 10 resolved.
sharpe_ratio 0.0 requires a daily-return time series, not yet tracked. sharpe_ratio Annualized Sharpe of the daily total_pnl curve (see
bot/metrics/sharpe.py). NULL until the sample gate passes:
>= 30 days observed AND >= 10 resolved trades.
""" """
import logging import logging
import os
from datetime import datetime, UTC from datetime import datetime, UTC
from bot.data.db import Database from bot.data.db import Database
from bot.executor.paper import Trade from bot.metrics.sharpe import sharpe_with_gate
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@@ -30,11 +33,6 @@ class MetricsTracker:
def __init__(self, db: Database) -> None: def __init__(self, db: Database) -> None:
self._db = db self._db = db
async def record_trade(self, trade: Trade) -> None:
"""Persist a trade to the DB. No in-memory accumulation."""
await self._db.save_trade(trade)
log.info("Trade recorded: %s", trade)
async def update_daily_summary(self) -> None: async def update_daily_summary(self) -> None:
"""Compute metrics from DB and write a metrics_daily snapshot. """Compute metrics from DB and write a metrics_daily snapshot.
@@ -67,6 +65,12 @@ class MetricsTracker:
avg_edge = total_pnl / total_deployed if total_deployed > 0 else 0.0 avg_edge = total_pnl / total_deployed if total_deployed > 0 else 0.0
# Sharpe: real value from the daily PnL curve, NULL while the sample
# gate (>=30 days observed, >=10 resolved) is not met.
bankroll = float(os.getenv("PAPER_BANKROLL", "10000"))
daily_closes = await self._db.get_daily_pnl_closes()
sharpe, sharpe_status = sharpe_with_gate(daily_closes, bankroll, resolved)
metrics = { metrics = {
"timestamp": datetime.now(UTC), "timestamp": datetime.now(UTC),
"total_trades": int(raw["total_trades"]), "total_trades": int(raw["total_trades"]),
@@ -80,7 +84,7 @@ class MetricsTracker:
"total_pnl": total_pnl, "total_pnl": total_pnl,
"win_rate": win_rate, "win_rate": win_rate,
"avg_edge": avg_edge, "avg_edge": avg_edge,
"sharpe_ratio": 0.0, # requires daily-return series (not yet tracked) "sharpe_ratio": sharpe, # NULL until sample gate passes
"calibration_score": calibration, "calibration_score": calibration,
"paper_mode": True, "paper_mode": True,
} }
@@ -89,9 +93,10 @@ class MetricsTracker:
log.info( log.info(
"Daily metrics | trades=%d (open=%d closed=%d resolved=%d) | " "Daily metrics | trades=%d (open=%d closed=%d resolved=%d) | "
"unrealized=$%.2f realized=$%.2f total=$%.2f | " "unrealized=$%.2f realized=$%.2f total=$%.2f | "
"win_rate=%s calibration=%s", "win_rate=%s calibration=%s sharpe=%s",
metrics["total_trades"], open_count, closed_count, resolved, metrics["total_trades"], open_count, closed_count, resolved,
unrealized, realized, total_pnl, unrealized, realized, total_pnl,
f"{win_rate:.1%}" if win_rate is not None else "n/a (<5)", f"{win_rate:.1%}" if win_rate is not None else "n/a (<5)",
f"{calibration:.3f}" if calibration is not None else "n/a (<10)", f"{calibration:.3f}" if calibration is not None else "n/a (<10)",
f"{sharpe:.2f}" if sharpe is not None else f"n/a ({sharpe_status})",
) )
+184
View File
@@ -0,0 +1,184 @@
"""One-shot and rate-limited Telegram checkpoint alerts.
Called from the main trading loop at the end of each cycle.
Errors are swallowed checkpoint failures must never break the loop.
"""
import logging
from datetime import datetime, timezone
from typing import Optional
from bot.notify import telegram
log = logging.getLogger(__name__)
_EXPOSURE_COOLDOWN_HOURS = 6
class CheckpointMonitor:
async def check_all(
self,
db,
exposure_pct: float,
exposure_cap_pct: float,
) -> None:
for name, coro in [
("primer_match_accepted", self._check_primer_match_accepted(db)),
("primer_trade_phase6", self._check_primer_trade_phase6(db)),
("primer_resolved", self._check_primer_resolved(db)),
("exposure_cerca_cap", self._check_exposure_cerca_cap(db, exposure_pct, exposure_cap_pct)),
]:
try:
await coro
except Exception as exc:
log.warning("checkpoint %s failed: %s", name, exc)
# ── helpers ──────────────────────────────────────────────────────────────
async def _one_shot_fired(self, db, name: str) -> bool:
async with db._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT 1 FROM checkpoint_alerts WHERE checkpoint_name = $1", name
)
return row is not None
async def _mark_one_shot(self, db, name: str) -> None:
async with db._pool.acquire() as conn:
await conn.execute(
"INSERT INTO checkpoint_alerts (checkpoint_name, fired_at) VALUES ($1, NOW())",
name,
)
async def _last_fired_at(self, db, name: str) -> Optional[datetime]:
async with db._pool.acquire() as conn:
row = await conn.fetchrow(
"SELECT last_fired_at FROM checkpoint_alerts WHERE checkpoint_name = $1",
name,
)
if row is None:
return None
return row["last_fired_at"]
async def _upsert_repeatable(self, db, name: str) -> None:
async with db._pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO checkpoint_alerts (checkpoint_name, fired_at, last_fired_at)
VALUES ($1, NOW(), NOW())
ON CONFLICT (checkpoint_name) DO UPDATE SET last_fired_at = NOW()
""",
name,
)
# ── checkpoints ──────────────────────────────────────────────────────────
async def _check_primer_match_accepted(self, db) -> None:
if await self._one_shot_fired(db, "primer_match_accepted"):
return
async with db._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT match_score, poly_question, mfld_market_title
FROM manifold_match_audit
WHERE match_status = 'accepted'
ORDER BY timestamp ASC
LIMIT 1
"""
)
if not row:
return
score = float(row["match_score"] or 0.0)
poly_q = (row["poly_question"] or "")[:60]
mfld_t = (row["mfld_market_title"] or "")[:60]
await telegram._send(
f"✅ Primer match Manifold accepted — score={score:.2f} "
f"poly='{poly_q}' mfld='{mfld_t}'"
)
await self._mark_one_shot(db, "primer_match_accepted")
log.info("checkpoint primer_match_accepted fired")
async def _check_primer_trade_phase6(self, db) -> None:
if await self._one_shot_fired(db, "primer_trade_phase6"):
return
async with db._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT question, mfld_match_score, edge_net
FROM trades
WHERE mfld_match_score IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
ORDER BY timestamp ASC
LIMIT 1
"""
)
if not row:
return
question = (row["question"] or "")[:70]
score = float(row["mfld_match_score"] or 0.0)
edge = float(row["edge_net"] or 0.0)
await telegram._send(
f"🎯 Primer trade Phase-6 limpio — {question} "
f"score={score:.2f} edge={edge:.3f}"
)
await self._mark_one_shot(db, "primer_trade_phase6")
log.info("checkpoint primer_trade_phase6 fired")
async def _check_primer_resolved(self, db) -> None:
if await self._one_shot_fired(db, "primer_resolved"):
return
async with db._pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT question, resolution, close_pnl
FROM trades
WHERE resolution IS NOT NULL
AND (excluded_from_metrics IS NOT TRUE)
ORDER BY closed_at ASC
LIMIT 1
"""
)
if not row:
return
question = (row["question"] or "")[:70]
resolution = float(row["resolution"] or 0.0)
pnl = float(row["close_pnl"] or 0.0)
await telegram._send(
f"🏁 Primer mercado resuelto — {question} "
f"result={resolution} pnl={pnl:.2f}"
)
await self._mark_one_shot(db, "primer_resolved")
log.info("checkpoint primer_resolved fired")
async def _check_exposure_cerca_cap(
self, db, exposure_pct: float, exposure_cap_pct: float
) -> None:
if exposure_pct < 0.80 * exposure_cap_pct:
return
last = await self._last_fired_at(db, "exposure_cerca_cap")
if last is not None:
now_utc = datetime.now(timezone.utc)
if last.tzinfo is None:
last = last.replace(tzinfo=timezone.utc)
elapsed_hours = (now_utc - last).total_seconds() / 3600
if elapsed_hours < _EXPOSURE_COOLDOWN_HOURS:
return
await telegram._send(
f"⚠️ Exposure al 80% del cap — revisar posiciones "
f"({exposure_pct * 100:.1f}% / {exposure_cap_pct * 100:.1f}%)"
)
await self._upsert_repeatable(db, "exposure_cerca_cap")
log.info(
"checkpoint exposure_cerca_cap fired (%.1f%% / %.1f%%)",
exposure_pct * 100, exposure_cap_pct * 100,
)
+22
View File
@@ -62,6 +62,17 @@ class Order:
feat_news_lo: float = 0.0 feat_news_lo: float = 0.0
feat_mfld_lo: float = 0.0 feat_mfld_lo: float = 0.0
feat_btc_dom_lo: float = 0.0 feat_btc_dom_lo: float = 0.0
# Manifold audit fields (propagated from TradingSignal → Trade → DB)
mfld_audit_id: Optional[str] = None
mfld_market_id: Optional[str] = None
mfld_market_title: Optional[str] = None
mfld_market_url: Optional[str] = None
mfld_prob_raw: Optional[float] = None
mfld_prob_final: Optional[float] = None
mfld_inverted: bool = False
mfld_match_score: Optional[float] = None
mfld_match_reason: Optional[str] = None
mfld_match_status: Optional[str] = None
class RiskManager: class RiskManager:
@@ -159,4 +170,15 @@ class RiskManager:
feat_news_lo=signal.feat_news_lo, feat_news_lo=signal.feat_news_lo,
feat_mfld_lo=signal.feat_mfld_lo, feat_mfld_lo=signal.feat_mfld_lo,
feat_btc_dom_lo=signal.feat_btc_dom_lo, feat_btc_dom_lo=signal.feat_btc_dom_lo,
# Manifold audit
mfld_audit_id=signal.mfld_audit_id,
mfld_market_id=signal.mfld_market_id,
mfld_market_title=signal.mfld_market_title,
mfld_market_url=signal.mfld_market_url,
mfld_prob_raw=signal.mfld_prob_raw,
mfld_prob_final=signal.mfld_prob_final,
mfld_inverted=signal.mfld_inverted,
mfld_match_score=signal.mfld_match_score,
mfld_match_reason=signal.mfld_match_reason,
mfld_match_status=signal.mfld_match_status,
) )
+289 -60
View File
@@ -12,16 +12,21 @@ Polymarket might reflect in a slow-moving order book.
""" """
import logging import logging
import math import math
import os
import re
import uuid
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import datetime, timezone from datetime import datetime, timedelta, timezone
from typing import Optional, TYPE_CHECKING from typing import Optional, TYPE_CHECKING
from bot.data.polymarket import Market, market_family_key from bot.data.polymarket import Market, market_family_key
from bot.data.external import ExternalSignals from bot.data.external import ExternalSignals
from bot.data.manifold import MANIFOLD_MATCHER_VERSION, ManifoldMatchResult
if TYPE_CHECKING: if TYPE_CHECKING:
from bot.data.news import NewsClient from bot.data.news import NewsClient
from bot.data.manifold import ManifoldClient from bot.data.manifold import ManifoldClient
from bot.data.db import Database
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@@ -58,11 +63,60 @@ NEWS_LOGODDS_WEIGHT = 1.5
# Weaker than NEWS_LOGODDS_WEIGHT because Manifold can have illiquid/stale markets. # Weaker than NEWS_LOGODDS_WEIGHT because Manifold can have illiquid/stale markets.
MANIFOLD_LOGODDS_WEIGHT = 0.6 MANIFOLD_LOGODDS_WEIGHT = 0.6
def _env_bool(name: str, default: bool) -> bool:
return os.getenv(name, str(default)).strip().lower() in ("1", "true", "yes", "on")
# ── Manifold activation flags ──────────────────────────────────────────────────
# Manifold has been retired as an ACTIVE trading signal: a per-category coverage
# audit (see /api/metrics/manifold-coverage) showed coverage_rate=0.0 across every
# category in the bot's current universe, so any edge it produced was false edge.
#
# MANIFOLD_SIGNAL_ENABLED (default False): when False, Manifold is observational
# only — its probability never touches the edge model: no manifold_log_adj, no
# confidence bump, feat_mfld_lo stays 0.0 (so it can never be the dominant
# feature), and it never contributes to a trade.
# MANIFOLD_AUDIT_ENABLED (default True): when True the matcher still runs and
# audit/coverage rows + cooldowns are written, preserving the trail so we can
# decide later whether to reactivate Manifold in a universe with real coverage.
# The matcher is only called when at least one flag is on.
MANIFOLD_SIGNAL_ENABLED = _env_bool("MANIFOLD_SIGNAL_ENABLED", False)
MANIFOLD_AUDIT_ENABLED = _env_bool("MANIFOLD_AUDIT_ENABLED", True)
# GNews free tier: 100 req/day. We limit to 5 queries per trading cycle # GNews free tier: 100 req/day. We limit to 5 queries per trading cycle
# (politics markets only) and rely on 6 h cache to stay within budget. # (politics markets only) and rely on 6 h cache to stay within budget.
MAX_NEWS_QUERIES_PER_CYCLE = 5 MAX_NEWS_QUERIES_PER_CYCLE = 5
# ─────────────────────────────────────────────────────────────────────────────
# Manifold evaluation cooldown
#
# Per-market backoff so the trading loop stops re-querying Manifold (and flooding
# manifold_match_audit) for markets whose verdict is stable. Computed from the
# match result; longer for verdicts that essentially never change.
# no_results → 24 h (Manifold has no market on this topic)
# rejected/low_score → 24 h (best candidate below Jaccard threshold)
# rejected/outcome_mism. → 24 h (outcome types differ)
# rejected/ambiguous → 24 h (party named but inversion unverifiable)
# rejected/conditional → 7 d (premise-gated market; structural, won't change)
# accepted → 1 h (signal is live; refresh probability hourly)
# ─────────────────────────────────────────────────────────────────────────────
def _cooldown_for(result: ManifoldMatchResult) -> tuple[timedelta, str]:
"""Map a Manifold match result to (retry_delay, cooldown_reason)."""
if result.status == "accepted":
return timedelta(hours=1), "accepted"
if result.status == "no_results":
return timedelta(hours=24), "no_results"
# rejected — classify by the reason text the matcher produced
reason = result.match_reason or "rejected"
if "conditional_market" in reason:
return timedelta(days=7), reason
# outcome_mismatch, ambiguous_inversion, and low_score (jaccard<threshold)
# all settle in 24 h.
return timedelta(hours=24), reason
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Phase 4 — Regime-based minimum edge (uses edge_NET, not edge_gross) # Phase 4 — Regime-based minimum edge (uses edge_NET, not edge_gross)
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
@@ -106,6 +160,21 @@ def _days_to_resolution(end_date: str) -> int:
return 30 return 30
def has_token(text: str, token: str) -> bool:
"""
True if `token` appears in `text` as a standalone word.
Short crypto tickers (eth, sol, ada, ) must NOT match inside ordinary
words "Seth", "dissolved", "Canada" but must still match the usual
market phrasings: "ETH", "$ETH", "ETH/USD", "SOL reach $200". Boundaries
are any non-alphanumeric character (or start/end of string), so "$" and
"/" delimit correctly.
"""
return re.search(
rf"(?<![A-Za-z0-9]){re.escape(token)}(?![A-Za-z0-9])", text, re.IGNORECASE
) is not None
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
# Phase 3 — GNews priority scoring # Phase 3 — GNews priority scoring
# ───────────────────────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────────────────────
@@ -170,6 +239,19 @@ class TradingSignal:
feat_news_lo: float = 0.0 feat_news_lo: float = 0.0
feat_mfld_lo: float = 0.0 feat_mfld_lo: float = 0.0
feat_btc_dom_lo: float = 0.0 feat_btc_dom_lo: float = 0.0
# ── Manifold match audit (propagated → Order → Trade → DB) ───────────────
# mfld_audit_id: UUID of the manifold_match_audit row; used to mark
# used_in_trade=TRUE after executor confirms the trade was executed.
mfld_audit_id: Optional[str] = None
mfld_market_id: Optional[str] = None
mfld_market_title: Optional[str] = None
mfld_market_url: Optional[str] = None
mfld_prob_raw: Optional[float] = None
mfld_prob_final: Optional[float] = None
mfld_inverted: bool = False
mfld_match_score: Optional[float] = None
mfld_match_reason: Optional[str] = None
mfld_match_status: Optional[str] = None
class BayesianStrategy: class BayesianStrategy:
@@ -201,10 +283,12 @@ class BayesianStrategy:
self, self,
news: Optional["NewsClient"] = None, news: Optional["NewsClient"] = None,
manifold: Optional["ManifoldClient"] = None, manifold: Optional["ManifoldClient"] = None,
db: Optional["Database"] = None,
) -> None: ) -> None:
self._signal_count = 0 self._signal_count = 0
self._news = news self._news = news
self._manifold = manifold self._manifold = manifold
self._db = db
self._news_queries_this_cycle = 0 self._news_queries_this_cycle = 0
# Per-cycle counters — reset by reset_cycle(), read by get_cycle_stats() # Per-cycle counters — reset by reset_cycle(), read by get_cycle_stats()
self._skip_family: int = 0 self._skip_family: int = 0
@@ -277,13 +361,18 @@ class BayesianStrategy:
"below", "under", "less than", "lower", "drop", "below", "under", "less than", "lower", "drop",
]) ])
is_btc = "btc" in question_lower or "bitcoin" in question_lower # Short tickers need word boundaries: "Seth" contains "eth",
is_eth = "eth" in question_lower or "ethereum" in question_lower # "dissolved" contains "sol", "Canada" contains "ada". Long
is_sol = "sol" in question_lower or "solana" in question_lower # unambiguous names (bitcoin, ethereum, …) stay as substrings.
is_xrp = "xrp" in question_lower or "ripple" in question_lower is_btc = has_token(question_lower, "btc") or "bitcoin" in question_lower
is_doge = "doge" in question_lower or "dogecoin" in question_lower is_eth = has_token(question_lower, "eth") or "ethereum" in question_lower
is_sol = has_token(question_lower, "sol") or "solana" in question_lower
is_xrp = has_token(question_lower, "xrp") or "ripple" in question_lower
is_doge = has_token(question_lower, "doge") or "dogecoin" in question_lower
is_altcoin = is_sol or is_xrp or is_doge or any( is_altcoin = is_sol or is_xrp or is_doge or any(
w in question_lower for w in ["ltc", "litecoin", "bnb", "ada", "cardano", "avax", "avalanche"] has_token(question_lower, t) for t in ["ltc", "bnb", "ada", "avax"]
) or any(
w in question_lower for w in ["litecoin", "cardano", "avalanche"]
) )
is_general_crypto = any( is_general_crypto = any(
w in question_lower for w in ["crypto", "market cap", "total market", "altcoin", "defi"] w in question_lower for w in ["crypto", "market cap", "total market", "altcoin", "defi"]
@@ -351,59 +440,73 @@ class BayesianStrategy:
sources: list[str] = [f"Prior=poly({prior:.3f})"] sources: list[str] = [f"Prior=poly({prior:.3f})"]
adjustments: list[float] = [] adjustments: list[float] = []
# Signal 1: price momentum (asset-specific or BTC as sentiment proxy) # Momentum and Fear & Greed only make sense for price markets, where
if is_btc: # is_price_above gives the adjustment a meaningful sign. For
momentum = ext.btc_change_24h # politics/tech/events there is no above/below notion — is_price_above
asset_label = "BTC" # defaults to False (or flips on accidental wording like "reach"), so
elif is_eth: # applying these signals just injected sign noise. Skip them entirely;
momentum = ext.eth_change_24h # their contributions stay 0.0 → feat_mom_lo / feat_fg_lo = 0.0.
asset_label = "ETH" is_non_price = is_politics or is_tech or is_events
elif is_politics or is_tech or is_events:
momentum = ext.btc_change_24h
asset_label = "BTC(sentiment)"
else:
momentum = ext.total_market_cap_change
asset_label = "total mktcap"
# Signal 1: price momentum (asset-specific; price markets only)
_momentum_contribution = 0.0 _momentum_contribution = 0.0
if abs(momentum) > 2: if not is_non_price:
momentum_adj = math.tanh(momentum / 20) * 0.15 if is_btc:
if is_politics or is_tech or is_events: momentum = ext.btc_change_24h
momentum_adj *= 0.5 asset_label = "BTC"
_momentum_contribution = momentum_adj if is_price_above else -momentum_adj elif is_eth:
adjustments.append(_momentum_contribution) momentum = ext.eth_change_24h
sources.append(f"{asset_label} 24h: {momentum:+.1f}%") asset_label = "ETH"
else:
momentum = ext.total_market_cap_change
asset_label = "total mktcap"
# Signal 2: Fear & Greed if abs(momentum) > 2:
fg = ext.fear_greed_index momentum_adj = math.tanh(momentum / 20) * 0.15
if fg > 70: _momentum_contribution = momentum_adj if is_price_above else -momentum_adj
fg_adj = 0.06 adjustments.append(_momentum_contribution)
sources.append(f"Fear&Greed: {fg} (greed)") sources.append(f"{asset_label} 24h: {momentum:+.1f}%")
elif fg < 30:
fg_adj = -0.06
sources.append(f"Fear&Greed: {fg} (fear)")
else:
fg_adj = (fg - 50) / 50 * 0.04
sources.append(f"Fear&Greed: {fg} (neutral)")
_fg_contribution = fg_adj if is_price_above else -fg_adj
adjustments.append(_fg_contribution)
# Signal 3: BTC dominance — hurts altcoins when high # Signal 2: Fear & Greed (price markets only)
_fg_contribution = 0.0
if not is_non_price:
fg = ext.fear_greed_index
if fg > 70:
fg_adj = 0.06
sources.append(f"Fear&Greed: {fg} (greed)")
elif fg < 30:
fg_adj = -0.06
sources.append(f"Fear&Greed: {fg} (fear)")
else:
fg_adj = (fg - 50) / 50 * 0.04
sources.append(f"Fear&Greed: {fg} (neutral)")
_fg_contribution = fg_adj if is_price_above else -fg_adj
adjustments.append(_fg_contribution)
# Signal 3: BTC dominance — hurts altcoins when high (price markets only)
# Like momentum and Fear & Greed above: no demonstrated causality for
# politics/tech/events, even when they legitimately mention a ticker
# ("Will the ETH ETF be approved?"). For non-price markets the
# contribution stays 0.0 → feat_btc_dom_lo = 0.0.
_btc_dom_contribution = 0.0 _btc_dom_contribution = 0.0
if (is_eth or is_altcoin or is_general_crypto) and ext.btc_dominance > 55: if not is_non_price:
_btc_dom_contribution = -0.03 if is_price_above else 0.03 if (is_eth or is_altcoin or is_general_crypto) and ext.btc_dominance > 55:
adjustments.append(_btc_dom_contribution) _btc_dom_contribution = -0.03 if is_price_above else 0.03
sources.append(f"BTC dom: {ext.btc_dominance:.1f}% (high → alt pressure)") adjustments.append(_btc_dom_contribution)
elif (is_eth or is_altcoin or is_general_crypto) and ext.btc_dominance < 45: sources.append(f"BTC dom: {ext.btc_dominance:.1f}% (high → alt pressure)")
_btc_dom_contribution = 0.03 if is_price_above else -0.03 elif (is_eth or is_altcoin or is_general_crypto) and ext.btc_dominance < 45:
adjustments.append(_btc_dom_contribution) _btc_dom_contribution = 0.03 if is_price_above else -0.03
sources.append(f"BTC dom: {ext.btc_dominance:.1f}% (low → alt season)") adjustments.append(_btc_dom_contribution)
sources.append(f"BTC dom: {ext.btc_dominance:.1f}% (low → alt season)")
# Signal 4: GNews sentiment (politics only, budget-gated) # Signal 4: GNews sentiment (politics only, budget-gated)
# Phase 3: caller has pre-sorted markets by gnews_priority() so the # Phase 3: caller has pre-sorted markets by gnews_priority() so the
# highest-value markets reach this block first. # highest-value markets reach this block first.
news_log_adj = 0.0 news_log_adj = 0.0
if is_politics and self._news is not None: # self._news.enabled gates the whole block: with no GNews API key the
# client is a no-op, so we must not consume (or report) query budget for
# it — see NewsClient.enabled.
if is_politics and self._news is not None and self._news.enabled:
if self._news_queries_this_cycle < MAX_NEWS_QUERIES_PER_CYCLE: if self._news_queries_this_cycle < MAX_NEWS_QUERIES_PER_CYCLE:
self._news_queries_this_cycle += 1 self._news_queries_this_cycle += 1
sentiment = await self._news.get_sentiment(market.question) sentiment = await self._news.get_sentiment(market.question)
@@ -419,18 +522,129 @@ class BayesianStrategy:
# Signal 5: Manifold cross-market probability (politics + tech) # Signal 5: Manifold cross-market probability (politics + tech)
# Applies a log-odds adjustment proportional to divergence from prior. # Applies a log-odds adjustment proportional to divergence from prior.
# No query budget — 30 min cache means network cost is paid once per cycle. # No query budget — 30 min cache means network cost is paid once per cycle.
# Now uses ManifoldMatchResult for stricter semantic validation and audit.
manifold_log_adj = 0.0 manifold_log_adj = 0.0
manifold_used = False manifold_used = False
if (is_politics or is_tech) and self._manifold is not None: manifold_result: Optional[ManifoldMatchResult] = None
manifold_prob = await self._manifold.get_probability(market.question) audit_id: Optional[str] = None
if manifold_prob is not None:
manifold_used = True if ((is_politics or is_tech) and self._manifold is not None
self._manifold_fetched += 1 and (MANIFOLD_AUDIT_ENABLED or MANIFOLD_SIGNAL_ENABLED)):
m_clamped = max(0.05, min(0.95, manifold_prob)) # ── Cooldown gate ────────────────────────────────────────────────
m_log = math.log(m_clamped / (1 - m_clamped)) # Skip markets whose Manifold verdict was recently settled to a
p_log = math.log(prior / (1 - prior)) # stable value. A skip is equivalent to a no-signal: the matcher is
manifold_log_adj = (m_log - p_log) * MANIFOLD_LOGODDS_WEIGHT # NOT called and NO manifold_match_audit row is written, so only real
sources.append(f"Manifold:{manifold_prob:.2f}") # evaluations are recorded. See _cooldown_for() and the
# manifold_eval_cooldown table.
in_cooldown = False
if self._db is not None and market.id:
try:
cd = await self._db.get_manifold_cooldown(market.id)
except Exception as exc:
log.warning("Failed to read manifold cooldown: %s", exc)
cd = None
if cd is not None and datetime.now(timezone.utc) < cd["retry_after"]:
in_cooldown = True
log.info(
"MANIFOLD_COOLDOWN skip market=%s | last_status=%s "
"retry_after=%s | %s",
market.id, cd["last_status"],
cd["retry_after"].isoformat(), market.question[:50],
)
if not in_cooldown:
manifold_result = await self._manifold.get_match(market.question)
# Persist audit record for ALL outcomes (accepted / rejected / no_results).
# Gated by MANIFOLD_AUDIT_ENABLED so the audit/coverage trail and
# cooldowns can be kept even while Manifold is observational-only.
if MANIFOLD_AUDIT_ENABLED and self._db is not None:
if not market.id:
log.error(
"MANIFOLD_AUDIT: market.id is None/empty — skipping audit save | "
"question=%r", market.question[:60],
)
else:
audit_id = str(uuid.uuid4())
try:
await self._db.save_manifold_audit(
audit_id=audit_id,
poly_market_id=market.id,
poly_question=market.question,
search_query=manifold_result.search_query,
mfld_market_id=manifold_result.market_id,
mfld_market_title=manifold_result.market_title,
mfld_market_url=manifold_result.market_url,
prob_raw=manifold_result.prob_raw,
prob_final=manifold_result.prob_final,
inverted=manifold_result.inverted,
match_score=manifold_result.match_score,
match_reason=manifold_result.match_reason,
match_status=manifold_result.status,
poly_outcome_type=manifold_result.poly_outcome_type,
mfld_outcome_type=manifold_result.mfld_outcome_type,
matcher_version=MANIFOLD_MATCHER_VERSION,
)
except Exception as exc:
log.warning("Failed to save manifold audit: %s", exc)
audit_id = None
# Record the cooldown so this market is not re-queried every
# cycle. Written even if the audit save above failed — we
# still performed a real evaluation.
if market.id:
delay, cd_reason = _cooldown_for(manifold_result)
try:
await self._db.upsert_manifold_cooldown(
poly_market_id=market.id,
last_status=manifold_result.status,
retry_after=datetime.now(timezone.utc) + delay,
cooldown_reason=cd_reason,
)
except Exception as exc:
log.warning("Failed to save manifold cooldown: %s", exc)
# Structured log — both forms for compatibility
log.info(
"MANIFOLD_MATCH poly='%s' mfld='%s' score=%s raw=%s final=%s"
" inverted=%s status=%s reason=%s",
market.question, manifold_result.market_title,
manifold_result.match_score, manifold_result.prob_raw,
manifold_result.prob_final, manifold_result.inverted,
manifold_result.status, manifold_result.match_reason,
)
log.info("MANIFOLD_MATCH", extra={
"poly_question": market.question,
"mfld_title": manifold_result.market_title,
"score": manifold_result.match_score,
"prob_raw": manifold_result.prob_raw,
"prob_final": manifold_result.prob_final,
"inverted": manifold_result.inverted,
"status": manifold_result.status,
"reason": manifold_result.match_reason,
})
if (MANIFOLD_SIGNAL_ENABLED
and manifold_result.status == "accepted"
and manifold_result.prob_final is not None):
# ACTIVE signal path — only when explicitly enabled.
manifold_used = True
self._manifold_fetched += 1
m_clamped = max(0.05, min(0.95, manifold_result.prob_final))
m_log = math.log(m_clamped / (1 - m_clamped))
p_log = math.log(prior / (1 - prior))
manifold_log_adj = (m_log - p_log) * MANIFOLD_LOGODDS_WEIGHT
sources.append(f"Manifold:{manifold_result.prob_final:.2f}")
elif not MANIFOLD_SIGNAL_ENABLED:
# Observational-only: matched/audited but NEVER fed to the edge
# model. manifold_log_adj stays 0.0 → no confidence bump,
# feat_mfld_lo=0.0 (cannot be dominant), no trade contribution.
log.info(
"Manifold: observational_only — signal disabled "
"(MANIFOLD_SIGNAL_ENABLED=false) | market=%s status=%s",
market.id, manifold_result.status,
)
sources.append("Manifold: observational_only")
# Confidence cap: macro/politics/tech signals are weaker proxies # Confidence cap: macro/politics/tech signals are weaker proxies
confidence_cap = 0.65 if (is_macro or is_politics or is_tech or is_events) else 0.90 confidence_cap = 0.65 if (is_macro or is_politics or is_tech or is_events) else 0.90
@@ -560,6 +774,21 @@ class BayesianStrategy:
feat_news_lo=feat_news_lo, feat_news_lo=feat_news_lo,
feat_mfld_lo=feat_mfld_lo, feat_mfld_lo=feat_mfld_lo,
feat_btc_dom_lo=feat_btc_dom_lo, feat_btc_dom_lo=feat_btc_dom_lo,
# Manifold match audit — propagated through Order → Trade → DB.
# mfld_audit_id is the hook main.py uses to flip the audit row's
# used_in_trade=TRUE; suppress it when observational so the trail
# truthfully shows Manifold drove no trades. The mfld_* fields below
# stay as observational record (feat_mfld_lo is already 0.0).
mfld_audit_id=(audit_id if MANIFOLD_SIGNAL_ENABLED else None),
mfld_market_id=manifold_result.market_id if manifold_result else None,
mfld_market_title=manifold_result.market_title if manifold_result else None,
mfld_market_url=manifold_result.market_url if manifold_result else None,
mfld_prob_raw=manifold_result.prob_raw if manifold_result else None,
mfld_prob_final=manifold_result.prob_final if manifold_result else None,
mfld_inverted=manifold_result.inverted if manifold_result else False,
mfld_match_score=manifold_result.match_score if manifold_result else None,
mfld_match_reason=manifold_result.match_reason if manifold_result else None,
mfld_match_status=manifold_result.status if manifold_result else None,
) )
+12 -2
View File
@@ -200,8 +200,12 @@ export default function App() {
<MetricCard <MetricCard
title="Sharpe" title="Sharpe"
value={fmt(summary.sharpe_ratio)} value={fmt(summary.sharpe_ratio)}
subtitle="Objetivo ≥ 0.5" subtitle={
progress={Math.min(1, summary.sharpe_ratio / 2)} summary.sharpe_ratio == null
? `Muestra insuficiente: ${summary.resolved_count}/${summary.min_resolved_required} resueltos, ${summary.days_observed}/${summary.min_days_required} días`
: 'Objetivo ≥ 0.5'
}
progress={summary.sharpe_ratio == null ? 0 : Math.min(1, summary.sharpe_ratio / 2)}
progressColor={summary.sharpe_ratio >= 0.5 ? 'var(--green)' : 'var(--amber)'} progressColor={summary.sharpe_ratio >= 0.5 ? 'var(--green)' : 'var(--amber)'}
/> />
<MetricCard <MetricCard
@@ -216,6 +220,12 @@ export default function App() {
value={fmtUSD(summary.total_deployed)} value={fmtUSD(summary.total_deployed)}
subtitle={`${summary.total_trades} trades`} subtitle={`${summary.total_trades} trades`}
/> />
<MetricCard
title="Cash Disponible"
value={fmtUSD(summary.cash_available)}
subtitle={`${fmtPct(summary.cash_available / summary.paper_bankroll)} del bankroll`}
progress={summary.cash_available / summary.paper_bankroll}
/>
</div> </div>
{/* Performance chart */} {/* Performance chart */}
+1 -1
View File
@@ -1,7 +1,7 @@
# Core # Core
asyncpg==0.29.0 asyncpg==0.29.0
httpx==0.27.0 httpx==0.27.0
fastapi==0.111.0 fastapi==0.139.0
uvicorn[standard]==0.29.0 uvicorn[standard]==0.29.0
pydantic==2.7.0 pydantic==2.7.0
+10
View File
@@ -0,0 +1,10 @@
"""Test environment shims.
The bot runs on python:3.11-slim in production; local dev machines may have
3.10, which lacks datetime.UTC (added in 3.11). Alias it so modules using
`from datetime import UTC` import cleanly under 3.10.
"""
import datetime
if not hasattr(datetime, "UTC"):
datetime.UTC = datetime.timezone.utc
+106
View File
@@ -0,0 +1,106 @@
"""
Tests for bug #7 — /api/summary must agree with the executor's cash model.
Regression: /api/summary computed total_trades as len() over a LIMIT-500
query (capped once history grows) and reimplemented cash as
bankroll - sum(net_cost of open trades) from that same capped query.
Fix: counts come from COUNT(*) (compute_metrics_from_db) and cash comes from
cash_available() the same helper PaperExecutor.initialize() uses fed by
the same source (get_open_position_data). This test runs both consumers
against one fake DB state and asserts they report identical cash.
"""
import asyncio
import pytest
import api.main as api_main
from bot.executor.paper import PaperExecutor, cash_available
BANKROLL = 10_000.0 # PAPER_BANKROLL default used by both bot and API
class FakeDB:
"""One DB state served to both the API endpoint and the executor."""
def __init__(self, positions: dict[str, float], total_net_cost: float,
total_trades: int, open_count: int):
self._positions = positions
self._total_net_cost = total_net_cost
self._total = total_trades
self._open = open_count
# Shared source: executor.initialize() and /api/summary both call this.
async def get_open_position_data(self):
return dict(self._positions), self._total_net_cost
# /api/summary only:
async def get_metrics_history(self, days=1):
return []
async def compute_metrics_from_db(self):
return {
"total_trades": self._total,
"open_count": self._open,
"closed_count": self._total - self._open,
"resolved_count": 0,
}
async def get_recently_closed_inverted(self, hours=24):
return set()
async def get_legacy_incomplete_count(self):
return 0
async def get_daily_pnl_closes(self):
return []
def _run(db: FakeDB, monkeypatch) -> tuple[dict, PaperExecutor]:
monkeypatch.setattr(api_main, "db", db)
monkeypatch.delenv("PAPER_BANKROLL", raising=False)
async def run():
summary = await api_main.get_summary()
ex = PaperExecutor(db=db, bankroll=BANKROLL)
await ex.initialize()
return summary, ex
return asyncio.run(run())
def test_api_and_executor_report_same_cash(monkeypatch):
db = FakeDB(
positions={"m1": 100.0, "m2": 80.0},
total_net_cost=183.60, # 180 + fees
total_trades=12,
open_count=2,
)
summary, ex = _run(db, monkeypatch)
assert summary["cash_available"] == pytest.approx(ex.get_portfolio().cash)
assert summary["cash_available"] == pytest.approx(
cash_available(BANKROLL, 183.60)
)
assert summary["total_deployed"] == pytest.approx(183.60)
def test_total_trades_not_capped_by_query_limit(monkeypatch):
"""700 trades in DB: the old len(LIMIT 500) reported 500."""
db = FakeDB(
positions={"m1": 100.0},
total_net_cost=102.0,
total_trades=700,
open_count=1,
)
summary, _ = _run(db, monkeypatch)
assert summary["total_trades"] == 700
assert summary["open_trades_count"] == 1
assert summary["closed_trades_count"] == 699
def test_cash_consistency_with_no_open_positions(monkeypatch):
db = FakeDB(positions={}, total_net_cost=0.0, total_trades=0, open_count=0)
summary, ex = _run(db, monkeypatch)
assert summary["cash_available"] == pytest.approx(BANKROLL)
assert ex.get_portfolio().cash == pytest.approx(BANKROLL)
+159
View File
@@ -0,0 +1,159 @@
"""
Tests for FASE 4 crypto ticker detection must use word boundaries.
Regression: short tickers were detected with substring matching over
question_lower, so non-crypto markets triggered crypto flags:
"Israeli parliament dissolved" contains "sol" is_sol / is_altcoin
"Will Canada win Group B" contains "ada" is_altcoin
"Will Seth Moulton be the nominee" contains "eth" is_eth
Those flags armed the BTC-dominance signal (btc_dom_lo=+0.06 observed in
production on politics markets). The fix routes short tickers (btc, eth,
sol, xrp, doge, ltc, bnb, ada, avax) through has_token(), which requires
non-alphanumeric boundaries; long unambiguous names (bitcoin, ethereum,
solana, cardano, ) remain substrings.
The is_* flags are internal to evaluate(), so the integration tests assert
on btc_dom_lo parsed from the structured audit log (same technique as
test_bayesian_macro_signals.py), with btc_dominance=60 so the signal fires
whenever an ETH/altcoin flag is set.
"""
import asyncio
import logging
import re
import pytest
from bot.data.external import ExternalSignals
from bot.data.polymarket import Market
from bot.strategy.bayesian import BayesianStrategy, has_token
BTC_DOM_RE = re.compile(r"btc_dom_lo=([+-]\d+\.\d+)")
def _make_market(question: str, category: str) -> Market:
return Market(
id="mkt-test-1",
condition_id="cond-test-1",
question=question,
yes_token_id="yes-tok",
no_token_id="no-tok",
yes_price=0.50,
no_price=0.50,
volume_24h=50_000.0,
end_date="2026-07-15T00:00:00Z",
active=True,
category=category,
)
def _make_signals() -> ExternalSignals:
# btc_dominance=60 (>55) arms the BTC-dominance signal for any market
# flagged as ETH / altcoin / general-crypto.
return ExternalSignals(
btc_price=100_000.0,
btc_change_24h=10.0,
eth_price=4_000.0,
eth_change_24h=8.0,
btc_dominance=60.0,
fear_greed_index=80,
fear_greed_label="greed",
total_market_cap_change=5.0,
valid=True,
)
def _evaluate_and_parse_btc_dom(question: str, category: str, caplog) -> float:
"""Run BayesianStrategy.evaluate and return btc_dom_lo from the audit log."""
strategy = BayesianStrategy(news=None, manifold=None, db=None)
market = _make_market(question, category)
with caplog.at_level(logging.INFO, logger="bot.strategy.bayesian"):
asyncio.run(
strategy.evaluate(market, _make_signals(), occupied_families=set())
)
for record in caplog.records:
m = BTC_DOM_RE.search(record.getMessage())
if m:
return float(m.group(1))
pytest.fail(
"No SKIP_EDGE_NET/TRADE log line with btc_dom_lo found; "
f"got: {[r.getMessage() for r in caplog.records]}"
)
# ── has_token unit tests ─────────────────────────────────────────────────────
def test_has_token_rejects_substrings_inside_words():
assert has_token("israeli parliament dissolved by june 30?", "sol") is False
assert has_token("will canada win group b?", "ada") is False
assert has_token("will seth moulton be the nominee?", "eth") is False
def test_has_token_matches_common_market_formats():
assert has_token("will eth hit $5000?", "eth") is True
assert has_token("$eth above $5000?", "eth") is True
assert has_token("eth/usd above 5000?", "eth") is True
assert has_token("will sol reach $200?", "sol") is True
assert has_token("will ada reach $1?", "ada") is True
assert has_token("BTC to $150k?", "btc") is True # case-insensitive
# ── Regression: false positives must not arm the BTC-dominance signal ───────
def test_israeli_parliament_market_is_not_sol(caplog):
"""'dissolved' contains 'sol' — must NOT flag is_sol/is_altcoin."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Israeli parliament dissolved by June 30?", "politics", caplog
)
assert btc_dom_lo == 0.0
def test_canada_market_is_not_ada(caplog):
"""'Canada' contains 'ada' — must NOT flag is_altcoin."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Will Canada win Group B?", "events", caplog
)
assert btc_dom_lo == 0.0
def test_seth_moulton_market_is_not_eth(caplog):
"""'Seth' contains 'eth' — must NOT flag is_eth."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Will Seth Moulton be the nominee?", "politics", caplog
)
assert btc_dom_lo == 0.0
# ── Real ticker mentions must keep working ───────────────────────────────────
def test_eth_market_detected(caplog):
"""Standalone 'ETH' still flags is_eth: BTC-dom fires and momentum uses ETH."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Will ETH hit $5000?", "crypto/finance", caplog
)
assert btc_dom_lo != 0.0
# Momentum picks the ETH branch only when is_eth is True.
full_log = "\n".join(r.getMessage() for r in caplog.records)
assert "ETH 24h: +8.0%" in full_log
def test_dollar_eth_market_detected(caplog):
"""'$ETH' format still flags is_eth."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"$ETH above $5000?", "crypto/finance", caplog
)
assert btc_dom_lo != 0.0
full_log = "\n".join(r.getMessage() for r in caplog.records)
assert "ETH 24h: +8.0%" in full_log
def test_sol_market_detected(caplog):
"""'SOL reach $200' still flags is_sol → is_altcoin → BTC-dom signal."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Will SOL reach $200?", "crypto/finance", caplog
)
# 'reach' → is_price_above, dominance 60 → -0.03 contribution → -0.06 log-odds
assert btc_dom_lo == pytest.approx(-0.06, abs=1e-4)
def test_ada_market_detected(caplog):
"""'ADA reach $1' still flags is_altcoin."""
btc_dom_lo = _evaluate_and_parse_btc_dom(
"Will ADA reach $1?", "crypto/finance", caplog
)
assert btc_dom_lo == pytest.approx(-0.06, abs=1e-4)
+130
View File
@@ -0,0 +1,130 @@
"""
Tests for FASE 5 BTC-dominance signal must not apply to non-price markets.
FASE 3 gated momentum and Fear & Greed behind is_non_price (politics / tech /
events); FASE 4 fixed ticker detection so non-crypto questions no longer flag
crypto assets by accident. But a non-price market that LEGITIMATELY mentions
a ticker ("Will the ETH ETF be approved?") still armed the BTC-dominance
signal, which has no demonstrated causality for non-price outcomes. FASE 5
applies the same is_non_price gate to that signal.
Note: the dominance signal only fires for is_eth / is_altcoin /
is_general_crypto markets a pure-BTC question never receives it, so the
pro-Bitcoin test below is a regression guard rather than a gate exercise;
the ETH-ETF test is the one that fails without the gate.
Same caplog technique as test_bayesian_asset_detection.py: btc_dom_lo is
parsed from the structured audit log, with btc_dominance=65 (>55) so the
signal fires whenever it is allowed to.
"""
import asyncio
import logging
import re
import pytest
from bot.data.external import ExternalSignals
from bot.data.polymarket import Market
from bot.strategy.bayesian import BayesianStrategy
BTC_DOM_RE = re.compile(r"btc_dom_lo=([+-]\d+\.\d+)")
def _make_market(question: str, category: str) -> Market:
return Market(
id="mkt-test-1",
condition_id="cond-test-1",
question=question,
yes_token_id="yes-tok",
no_token_id="no-tok",
yes_price=0.50,
no_price=0.50,
volume_24h=50_000.0,
end_date="2026-07-15T00:00:00Z",
active=True,
category=category,
)
def _make_signals() -> ExternalSignals:
# btc_dominance=65 (>55) arms the dominance signal wherever it is allowed.
# Momentum kept below the 2% threshold so price-market tests isolate the
# dominance contribution.
return ExternalSignals(
btc_price=100_000.0,
btc_change_24h=1.0,
eth_price=4_000.0,
eth_change_24h=1.0,
btc_dominance=65.0,
fear_greed_index=50,
fear_greed_label="neutral",
total_market_cap_change=1.0,
valid=True,
)
def _evaluate(question: str, category: str, caplog) -> tuple[float, str]:
"""Run evaluate() and return (btc_dom_lo, full_log) from the audit log."""
strategy = BayesianStrategy(news=None, manifold=None, db=None)
market = _make_market(question, category)
with caplog.at_level(logging.INFO, logger="bot.strategy.bayesian"):
asyncio.run(
strategy.evaluate(market, _make_signals(), occupied_families=set())
)
full_log = "\n".join(r.getMessage() for r in caplog.records)
for record in caplog.records:
m = BTC_DOM_RE.search(record.getMessage())
if m:
return float(m.group(1)), full_log
pytest.fail(
"No SKIP_EDGE_NET/TRADE log line with btc_dom_lo found; "
f"got: {[r.getMessage() for r in caplog.records]}"
)
# ── Non-price markets: gate must zero the signal ─────────────────────────────
def test_politics_market_mentioning_eth_gets_no_btc_dom(caplog):
"""Legitimate ETH mention in a politics market → btc_dom_lo == 0.0."""
btc_dom_lo, full_log = _evaluate(
"Will the ETH ETF be approved?", "politics", caplog
)
assert btc_dom_lo == 0.0
assert "BTC dom" not in full_log
def test_politics_market_mentioning_bitcoin_gets_no_btc_dom(caplog):
"""Legitimate Bitcoin mention in a politics market → btc_dom_lo == 0.0."""
btc_dom_lo, full_log = _evaluate(
"Will a pro-Bitcoin candidate win the election?", "politics", caplog
)
assert btc_dom_lo == 0.0
assert "BTC dom" not in full_log
def test_tech_and_events_markets_get_no_btc_dom(caplog):
for category in ("tech", "events"):
caplog.clear()
btc_dom_lo, full_log = _evaluate(
"Will the ETH foundation launch the product?", category, caplog
)
assert btc_dom_lo == 0.0, f"BTC dominance applied to {category} market"
assert "BTC dom" not in full_log
# ── Price markets: current behavior preserved ───────────────────────────────
def test_eth_price_market_keeps_btc_dom(caplog):
"""ETH price market with dominance 65 → signal fires as before."""
btc_dom_lo, full_log = _evaluate(
"Will ETH be above $5000?", "crypto/finance", caplog
)
# 'above' → is_price_above, dominance 65 > 55 → -0.03 → -0.06 log-odds
assert btc_dom_lo == pytest.approx(-0.06, abs=1e-4)
assert "BTC dom: 65.0% (high → alt pressure)" in full_log
def test_altcoin_price_market_keeps_btc_dom(caplog):
"""SOL price market with dominance 65 → signal fires as before."""
btc_dom_lo, full_log = _evaluate(
"Will SOL reach $200?", "crypto/finance", caplog
)
assert btc_dom_lo == pytest.approx(-0.06, abs=1e-4)
assert "BTC dom: 65.0% (high → alt pressure)" in full_log
+123
View File
@@ -0,0 +1,123 @@
"""
Tests for FASE 3 macro signals (momentum, Fear & Greed) must not apply to
non-price markets (politics / tech / events).
Regression: for "Will X win the election?"-style questions, is_price_above is
False, so positive BTC momentum and high Fear & Greed were sign-flipped into
evidence AGAINST the YES outcome. The fix skips both signals entirely for
politics/tech/events, leaving their contributions (and feat_mom_lo /
feat_fg_lo) at 0.0.
evaluate_market only returns a TradingSignal on the TRADE path; on skips it
returns None but always emits a structured log line containing the per-feature
log-odds (fg_lo= mom_lo=). The tests parse that line via caplog.
"""
import asyncio
import logging
import math
import re
import pytest
from bot.data.external import ExternalSignals
from bot.data.polymarket import Market
from bot.strategy.bayesian import BayesianStrategy
FEAT_RE = re.compile(r"fg_lo=([+-]\d+\.\d+) mom_lo=([+-]\d+\.\d+)")
def _make_market(question: str, category: str) -> Market:
return Market(
id="mkt-test-1",
condition_id="cond-test-1",
question=question,
yes_token_id="yes-tok",
no_token_id="no-tok",
yes_price=0.50,
no_price=0.50,
volume_24h=50_000.0,
end_date="2026-07-15T00:00:00Z",
active=True,
category=category,
)
def _make_signals() -> ExternalSignals:
# Strong bullish macro environment: BTC +10%, extreme greed.
return ExternalSignals(
btc_price=100_000.0,
btc_change_24h=10.0,
eth_price=4_000.0,
eth_change_24h=8.0,
btc_dominance=50.0,
fear_greed_index=80,
fear_greed_label="greed",
total_market_cap_change=5.0,
valid=True,
)
def _evaluate_and_parse_feats(question: str, category: str, caplog) -> tuple[float, float]:
"""Run BayesianStrategy.evaluate and return (feat_fg_lo, feat_mom_lo) from the audit log."""
strategy = BayesianStrategy(news=None, manifold=None, db=None)
market = _make_market(question, category)
with caplog.at_level(logging.INFO, logger="bot.strategy.bayesian"):
asyncio.run(
strategy.evaluate(market, _make_signals(), occupied_families=set())
)
for record in caplog.records:
m = FEAT_RE.search(record.getMessage())
if m:
return float(m.group(1)), float(m.group(2))
pytest.fail(
"No SKIP_EDGE_NET/TRADE log line with feature contributions found; "
f"got: {[r.getMessage() for r in caplog.records]}"
)
def test_politics_market_ignores_momentum_and_fear_greed(caplog):
"""Political market with BTC +10% and F&G=80 → both contributions 0.0."""
feat_fg_lo, feat_mom_lo = _evaluate_and_parse_feats(
"Will John Smith win the election?", "politics", caplog
)
assert feat_mom_lo == 0.0
assert feat_fg_lo == 0.0
# The signal sources must not mention momentum or Fear & Greed either.
full_log = "\n".join(r.getMessage() for r in caplog.records)
assert "Fear&Greed" not in full_log
assert "24h" not in full_log
def test_tech_and_events_markets_ignore_macro_signals(caplog):
for category in ("tech", "events"):
caplog.clear()
feat_fg_lo, feat_mom_lo = _evaluate_and_parse_feats(
"Will the product launch happen this quarter?", category, caplog
)
assert feat_mom_lo == 0.0, f"momentum applied to {category} market"
assert feat_fg_lo == 0.0, f"Fear&Greed applied to {category} market"
def test_btc_market_keeps_momentum_and_fear_greed(caplog):
"""BTC price market with BTC +10% and F&G=80 → current behavior preserved."""
feat_fg_lo, feat_mom_lo = _evaluate_and_parse_feats(
"Will Bitcoin be above $150,000 on July 1?", "crypto/finance", caplog
)
assert feat_mom_lo > 0
assert feat_fg_lo > 0
# Exact values: is_price_above=True ("above"), so contributions are positive.
# momentum: tanh(10/20) * 0.15, ×2 → log-odds. F&G>70: +0.06, ×2 → log-odds.
assert feat_mom_lo == pytest.approx(math.tanh(10 / 20) * 0.15 * 2, abs=1e-4)
assert feat_fg_lo == pytest.approx(0.06 * 2, abs=1e-4)
full_log = "\n".join(r.getMessage() for r in caplog.records)
assert "Fear&Greed: 80 (greed)" in full_log
assert "BTC 24h: +10.0%" in full_log
def test_btc_below_market_sign_flip_preserved(caplog):
"""'below' market: bullish macro lowers YES probability (sign flip intact)."""
feat_fg_lo, feat_mom_lo = _evaluate_and_parse_feats(
"Will Bitcoin drop below $50,000 by August?", "crypto/finance", caplog
)
assert feat_mom_lo < 0
assert feat_fg_lo < 0
+152
View File
@@ -0,0 +1,152 @@
"""
Tests for the Manifold outcome-compatibility guard.
Regression: a Polymarket *nomination* question must not match a Manifold
*conditional* question ("If X is the nominee, will he win?") even at Jaccard=1.0.
"""
import asyncio
import pytest
from bot.data.manifold import (
ManifoldClient,
_classify_outcome,
_is_conditional,
)
# ── _is_conditional ────────────────────────────────────────────────────────────
def test_is_conditional_prefixes():
assert _is_conditional("If Graham Platner is the nominee, will he win?")
assert _is_conditional("Conditional on a recession, will rates fall?")
assert _is_conditional("Assuming Trump runs, will he win?")
assert _is_conditional("Given that X happens, will Y?")
def test_is_conditional_midsentence_clause():
assert _is_conditional("Will Biden, if he is nominated, win the election?")
def test_is_not_conditional():
assert not _is_conditional("Will Graham Platner be the Democratic nominee?")
assert not _is_conditional("Will the GOP win the Senate?")
# "if" without a closing comma clause is not flagged
assert not _is_conditional("What happens if everything goes right")
# ── _classify_outcome ───────────────────────────────────────────────────────────
def test_classify_nomination():
assert _classify_outcome("Will X be the Democratic nominee for Senate?") == "nomination"
assert _classify_outcome("Will X be nominated?") == "nomination"
# "primary nominee" → nomination (checked before primary)
assert _classify_outcome("Will X be the primary nominee?") == "nomination"
def test_classify_primary_win():
assert _classify_outcome("Will X win the primary?") == "primary_win"
assert _classify_outcome("Will X advance in the first round?") == "primary_win"
def test_classify_general_win():
assert _classify_outcome("Will X win the election?") == "general_win"
assert _classify_outcome("Will X win the seat?") == "general_win"
assert _classify_outcome("Will X win the general election?") == "general_win"
def test_classify_conditional():
assert _classify_outcome("If X is the nominee, will he win?") == "conditional"
assert _classify_outcome("Assuming a runoff, who wins?") == "conditional"
def test_classify_other():
assert _classify_outcome("Will it rain tomorrow?") == "other"
# ── End-to-end get_match with a stubbed Manifold API ────────────────────────────
class _StubResponse:
def __init__(self, payload):
self._payload = payload
def raise_for_status(self):
pass
def json(self):
return self._payload
class _StubHTTP:
def __init__(self, payload):
self._payload = payload
async def get(self, *args, **kwargs):
return _StubResponse(self._payload)
async def aclose(self):
pass
async def _match(poly, mfld_market):
client = ManifoldClient()
client._client = _StubHTTP([mfld_market])
try:
return await client.get_match(poly)
finally:
await client.close()
def test_graham_platner_conditional_rejected():
"""Poly nomination vs Manifold conditional → rejected (Task 4.1)."""
poly = ("Will Graham Platner be the Democratic nominee for Senate "
"in Maine in 2026?")
mfld_market = {
"outcomeType": "BINARY",
"probability": 0.55,
"question": ("If Graham Platner is the Democratic nominee for Senate "
"in Maine, will he win the general election?"),
"id": "abc123",
"slug": "graham-platner-win",
"creatorUsername": "someone",
}
result = asyncio.run(_match(poly, mfld_market))
assert result.status == "rejected"
assert result.match_reason is not None
assert ("conditional" in result.match_reason
or "outcome_mismatch" in result.match_reason)
# outcome types are classified and available for persistence
assert result.poly_outcome_type == "nomination"
assert result.mfld_outcome_type == "conditional"
def test_outcome_mismatch_nomination_vs_general_rejected():
"""Poly nomination vs Manifold general_win (non-conditional) → rejected."""
poly = "Will Jane Doe be the Republican nominee for Governor?"
mfld_market = {
"outcomeType": "BINARY",
"probability": 0.4,
"question": "Will Jane Doe win the election for Governor?",
"id": "x", "slug": "jane-doe", "creatorUsername": "u",
}
result = asyncio.run(_match(poly, mfld_market))
assert result.status == "rejected"
assert "outcome_mismatch" in result.match_reason
assert result.poly_outcome_type == "nomination"
assert result.mfld_outcome_type == "general_win"
def test_matching_nomination_accepted():
"""Poly nomination vs Manifold nomination (same outcome) → accepted."""
poly = "Will Graham Platner be the Democratic nominee for Senate in Maine?"
mfld_market = {
"outcomeType": "BINARY",
"probability": 0.62,
"question": "Will Graham Platner be the Democratic Senate nominee in Maine?",
"id": "ok", "slug": "platner-nominee", "creatorUsername": "u",
}
result = asyncio.run(_match(poly, mfld_market))
assert result.status == "accepted"
assert result.poly_outcome_type == "nomination"
assert result.mfld_outcome_type == "nomination"
assert result.prob_final == pytest.approx(0.62)
+77
View File
@@ -0,0 +1,77 @@
"""Tests for the GNews layer minor fixes.
Two faults found during the GNews capture/prioritisation diagnostic:
1. Hyphens/dashes in a market question reached the GNews query verbatim and,
because '-' is GNews's exclusion operator, produced HTTP 400
(e.g. "Abdul El-Sayed Michigan Democratic Primary").
2. The per-cycle GNews budget counter incremented in evaluate() *before*
get_sentiment() checked the API key, so with no key configured the
[CYCLE SUMMARY] reported a phantom "gnews_queries_used: 5/5" even though
zero real requests left the process.
"""
import asyncio
from bot.data.news import NewsClient
from bot.data.external import ExternalSignals
from bot.data.polymarket import Market
from bot.strategy.bayesian import BayesianStrategy
# ── Fix 1: query sanitisation ────────────────────────────────────────────────
def test_build_query_strips_hyphen_that_breaks_gnews():
q = NewsClient._build_query(
"Will Abdul El-Sayed win the 2026 Michigan Democratic Primary?"
)
assert "-" not in q # the exclusion operator must be gone
assert "El-Sayed" not in q
assert "Sayed" in q # the meaningful token survives as its own word
def test_build_query_strips_unicode_dashes():
q = NewsClient._build_query("TrumpPutin summit — final outcome")
assert "" not in q and "" not in q
assert "Trump" in q and "Putin" in q
# ── Fix 2: enabled property + budget accounting ──────────────────────────────
def test_enabled_reflects_api_key(monkeypatch):
monkeypatch.delenv("GNEWS_API_KEY", raising=False)
assert NewsClient().enabled is False
monkeypatch.setenv("GNEWS_API_KEY", "deadbeefdeadbeefdeadbeefdeadbeef")
assert NewsClient().enabled is True
def _politics_market() -> Market:
return Market(
id="m1", condition_id="c1",
question="Will candidate X win the 2026 governor election?",
yes_token_id="y", no_token_id="n",
yes_price=0.50, no_price=0.50, volume_24h=10_000.0,
end_date="2026-07-15T00:00:00Z", active=True, category="politics",
)
def _signals() -> ExternalSignals:
return ExternalSignals(
btc_price=1.0, btc_change_24h=0.0, eth_price=1.0, eth_change_24h=0.0,
btc_dominance=50.0, fear_greed_index=50, fear_greed_label="neutral",
total_market_cap_change=0.0, valid=True,
)
def test_disabled_news_consumes_no_gnews_budget(monkeypatch):
"""Regression: no API key → gnews_queries_used stays 0 (was a phantom 1+)."""
monkeypatch.delenv("GNEWS_API_KEY", raising=False)
news = NewsClient()
assert news.enabled is False
strategy = BayesianStrategy(news=news, manifold=None, db=None)
strategy.reset_cycle()
asyncio.run(
strategy.evaluate(_politics_market(), _signals(), occupied_families=set())
)
assert strategy.get_cycle_stats()["gnews_queries_used"] == 0
+130
View File
@@ -0,0 +1,130 @@
"""
Tests for PaperExecutor.close_position() settlement payout.
Regression: the old code computed cash += position_cost * resolution, which
ignores direction a winning BUY_NO (resolution = 0.0) paid out $0.
Correct settlement:
BUY_YES: payout = shares * resolution
BUY_NO: payout = shares * (1 - resolution)
pnl = payout - net_cost
"""
import asyncio
import pytest
from bot.executor import paper
from bot.executor.paper import PaperExecutor
class FakeDB:
"""Minimal Database stub for close_position()."""
def __init__(self, trades_by_market: dict[str, list[dict]]):
self._trades = trades_by_market
self.closed: list[tuple] = []
async def get_open_trades_for_market(self, market_id: str) -> list[dict]:
return self._trades.get(market_id, [])
async def close_paper_position(self, market_id, reason="", resolution=None):
self.closed.append((market_id, reason, resolution))
def _close(direction: str, resolution: float):
"""Open one paper trade (size $100 @ 0.5 → 200 shares, net_cost $102)
and settle it at `resolution`. Returns (pnl, executor, notifications)."""
notifications: list[tuple] = []
async def fake_trade_closed(question, pnl):
notifications.append((question, pnl))
async def run():
db = FakeDB({
"mkt1": [{"direction": direction, "shares": 200.0, "net_cost": 102.0}],
})
ex = PaperExecutor(db=db, bankroll=1000.0)
ex._portfolio.cash = 898.0 # 1000 - net_cost spent at entry
ex._portfolio.positions["mkt1"] = 100.0 # size_usdc, as execute() stores it
original = paper.telegram.trade_closed
paper.telegram.trade_closed = fake_trade_closed
try:
pnl = await ex.close_position("mkt1", resolution, question="Test market?")
await asyncio.sleep(0) # let the notification task run
finally:
paper.telegram.trade_closed = original
return pnl, ex, db
pnl, ex, db = asyncio.run(run())
return pnl, ex, db, notifications
def test_buy_yes_wins():
pnl, ex, db, notif = _close("BUY_YES", resolution=1.0)
assert pnl == pytest.approx(200.0 - 102.0) # payout = 200 * 1.0
assert pnl > 0
assert ex._portfolio.cash == pytest.approx(898.0 + 200.0)
assert notif[0][1] > 0 # Telegram reports a win
def test_buy_yes_loses():
pnl, ex, db, notif = _close("BUY_YES", resolution=0.0)
assert pnl == pytest.approx(-102.0) # payout = 0
assert pnl < 0
assert ex._portfolio.cash == pytest.approx(898.0)
assert notif[0][1] < 0 # Telegram reports a loss
def test_buy_no_wins():
pnl, ex, db, notif = _close("BUY_NO", resolution=0.0)
assert pnl == pytest.approx(200.0 - 102.0) # payout = 200 * (1 - 0.0)
assert pnl > 0
assert ex._portfolio.cash == pytest.approx(898.0 + 200.0)
assert notif[0][1] > 0 # win despite resolution = 0.0
def test_buy_no_loses():
pnl, ex, db, notif = _close("BUY_NO", resolution=1.0)
assert pnl == pytest.approx(-102.0) # payout = 200 * (1 - 1.0) = 0
assert pnl < 0
assert ex._portfolio.cash == pytest.approx(898.0)
assert notif[0][1] < 0 # loss despite resolution = 1.0
def test_position_is_removed_and_persisted():
pnl, ex, db, notif = _close("BUY_YES", resolution=1.0)
assert "mkt1" not in ex._portfolio.positions
assert db.closed == [("mkt1", "resolved", 1.0)]
def test_unknown_market_returns_none():
async def run():
ex = PaperExecutor(db=FakeDB({}), bankroll=1000.0)
return await ex.close_position("nope", 1.0)
assert asyncio.run(run()) is None
def test_db_failure_keeps_position_for_retry():
"""Regression: a DB error during close must not mutate the in-memory
portfolio otherwise the next resolution check skips the market
(not in positions) and the DB row stays open forever."""
class FailingDB(FakeDB):
async def close_paper_position(self, market_id, reason="", resolution=None):
raise RuntimeError("db down")
async def run():
db = FailingDB({
"mkt1": [{"direction": "BUY_YES", "shares": 200.0, "net_cost": 102.0}],
})
ex = PaperExecutor(db=db, bankroll=1000.0)
ex._portfolio.cash = 898.0
ex._portfolio.positions["mkt1"] = 100.0
with pytest.raises(RuntimeError):
await ex.close_position("mkt1", 1.0)
return ex
ex = asyncio.run(run())
assert ex._portfolio.positions == {"mkt1": 100.0} # still open in memory
assert ex._portfolio.cash == pytest.approx(898.0) # payout not credited
+219
View File
@@ -0,0 +1,219 @@
"""
Tests for the automatic market-resolution detector (Phase 2).
Covers:
- PolymarketClient.get_market_resolution() parsing of real Gamma API shapes
(resolved YES/NO, still open, UMA-disputed, ambiguous prices, 404, errors).
- check_resolutions() in bot/main.py: a resolved market settles the open
paper position via PaperExecutor.close_position() and persists
close_reason='resolved' with the resolution value.
"""
import asyncio
import json
import httpx
import pytest
from bot.data.polymarket import PolymarketClient, MarketResolution
from bot.executor import paper
from bot.executor.paper import PaperExecutor
from bot.main import check_resolutions
# ─────────────────────────────────────────────────────────────────────────────
# get_market_resolution() — Gamma API response parsing
# ─────────────────────────────────────────────────────────────────────────────
class FakeResponse:
def __init__(self, status_code: int, payload: dict | None = None):
self.status_code = status_code
self._payload = payload or {}
def json(self):
return self._payload
def raise_for_status(self):
if self.status_code >= 400:
raise httpx.HTTPStatusError(
f"HTTP {self.status_code}", request=None, response=None
)
class FakeHTTPClient:
def __init__(self, response):
self._response = response
self.requested_urls: list[str] = []
async def get(self, url, **kwargs):
self.requested_urls.append(url)
if isinstance(self._response, Exception):
raise self._response
return self._response
def _resolution_for(response) -> MarketResolution | None:
client = PolymarketClient()
client._client = FakeHTTPClient(response)
return asyncio.run(client.get_market_resolution("12345"))
def _gamma_market(closed: bool, yes_price: str, no_price: str,
uma_status: str | None = "resolved") -> dict:
"""Mirror the real Gamma /markets/{id} payload shape (observed 2026-06-11)."""
m = {
"id": "12345",
"question": "Test market?",
"closed": closed,
"active": True,
"outcomePrices": json.dumps([yes_price, no_price]),
"closedTime": "2026-06-11 13:15:01+00" if closed else None,
"umaEndDate": "2026-06-11T13:15:01Z" if closed else None,
"endDate": "2026-06-11T13:00:00Z",
}
if uma_status is not None:
m["umaResolutionStatus"] = uma_status
return m
def test_resolution_no_won():
res = _resolution_for(FakeResponse(200, _gamma_market(True, "0", "1")))
assert res.resolved is True
assert res.resolution == 0.0
assert res.resolved_at is not None
def test_resolution_yes_won():
res = _resolution_for(FakeResponse(200, _gamma_market(True, "1", "0")))
assert res.resolved is True
assert res.resolution == 1.0
def test_open_market_not_resolved():
res = _resolution_for(FakeResponse(
200, _gamma_market(False, "0.51", "0.49", uma_status=None)
))
assert res.resolved is False
assert res.resolution is None
def test_closed_but_uma_disputed_not_settled():
res = _resolution_for(FakeResponse(
200, _gamma_market(True, "0", "1", uma_status="disputed")
))
assert res.resolved is False
def test_closed_with_ambiguous_prices_not_settled():
res = _resolution_for(FakeResponse(200, _gamma_market(True, "0.6", "0.4")))
assert res.resolved is False
def test_market_not_found_returns_none():
assert _resolution_for(FakeResponse(404)) is None
def test_api_error_returns_none():
assert _resolution_for(httpx.ConnectError("boom")) is None
# ─────────────────────────────────────────────────────────────────────────────
# check_resolutions() — detector loop settles paper positions
# ─────────────────────────────────────────────────────────────────────────────
class FakeDB:
"""Database stub: one open BUY_NO paper position."""
def __init__(self, trades_by_market: dict[str, list[dict]]):
self._trades = trades_by_market
self.closed: list[tuple] = []
async def get_open_position_details(self) -> list[dict]:
return [
{"market_id": mid, "question": t[0].get("question", ""),
"direction": t[0]["direction"]}
for mid, t in self._trades.items()
]
async def get_open_trades_for_market(self, market_id: str) -> list[dict]:
return self._trades.get(market_id, [])
async def close_paper_position(self, market_id, reason="", resolution=None):
self.closed.append((market_id, reason, resolution))
class FakePoly:
def __init__(self, resolutions: dict[str, MarketResolution | None]):
self._resolutions = resolutions
self.checked: list[str] = []
async def get_market_resolution(self, market_id: str):
self.checked.append(market_id)
return self._resolutions.get(market_id)
def _run_check(resolutions: dict, trades: dict):
notifications: list[tuple] = []
async def fake_trade_closed(question, pnl):
notifications.append((question, pnl))
async def run():
db = FakeDB(trades)
ex = PaperExecutor(db=db, bankroll=1000.0)
for mid, t in trades.items():
ex._portfolio.positions[mid] = sum(x["net_cost"] for x in t) - 2.0
ex._portfolio.cash = 898.0
poly = FakePoly(resolutions)
original = paper.telegram.trade_closed
paper.telegram.trade_closed = fake_trade_closed
try:
await check_resolutions(poly, ex, db)
await asyncio.sleep(0) # let notification task run
finally:
paper.telegram.trade_closed = original
return db, ex, poly
db, ex, poly = asyncio.run(run())
return db, ex, poly, notifications
BUY_NO_TRADE = {
"mkt1": [{
"direction": "BUY_NO", "shares": 200.0, "net_cost": 102.0,
"question": "Will X happen?",
}],
}
def test_resolved_buy_no_position_is_closed():
"""BUY_NO position + market resolved NO (resolution=0.0) → winning close."""
db, ex, poly, notif = _run_check(
{"mkt1": MarketResolution(resolved=True, resolution=0.0)},
BUY_NO_TRADE,
)
assert poly.checked == ["mkt1"]
# close_paper_position called with close_reason='resolved' and the resolution
assert db.closed == [("mkt1", "resolved", 0.0)]
# Position removed and payout credited: 200 shares * (1 - 0.0) = $200
assert "mkt1" not in ex._portfolio.positions
assert ex._portfolio.cash == pytest.approx(898.0 + 200.0)
# Telegram notified with positive pnl (200 - 102)
assert notif == [("Will X happen?", pytest.approx(98.0))]
def test_unresolved_position_stays_open():
db, ex, poly, notif = _run_check(
{"mkt1": MarketResolution(resolved=False)},
BUY_NO_TRADE,
)
assert poly.checked == ["mkt1"]
assert db.closed == []
assert "mkt1" in ex._portfolio.positions
assert notif == []
def test_api_failure_leaves_position_open():
db, ex, poly, notif = _run_check({"mkt1": None}, BUY_NO_TRADE)
assert db.closed == []
assert "mkt1" in ex._portfolio.positions
+242
View File
@@ -0,0 +1,242 @@
"""
Tests for the real Sharpe ratio with minimum-sample gate.
Regression: sharpe_ratio was hardcoded to 0.0 in MetricsTracker and exposed
as `latest.get("sharpe_ratio") or 0` in /api/summary, and promotion_ready
could in principle flip on a statistically meaningless sample (e.g. 1
resolved trade over ~40 days of flat PnL plus a single +299 jump).
Fix: bot/metrics/sharpe.py computes an annualized Sharpe from the daily
total_pnl close series, gated to None ("insufficient_sample") below 30 days
observed / 10 resolved trades. /api/summary exposes the value plus an
explanation (sharpe_status, days_observed, min_* fields), and
promotion_ready additionally requires the sample minimums and non-null
metrics.
"""
import asyncio
from statistics import mean, stdev
import pytest
import api.main as api_main
from bot.metrics.sharpe import (
MIN_DAYS_OBSERVED,
MIN_RESOLVED_TRADES,
SHARPE_INSUFFICIENT,
SHARPE_OK,
SHARPE_ZERO_VARIANCE,
compute_sharpe,
daily_returns,
sharpe_with_gate,
)
from bot.metrics.tracker import MetricsTracker
BANKROLL = 10_000.0
def _closes_from_deltas(deltas: list[float], start: float = 0.0) -> list[float]:
closes = [start]
for d in deltas:
closes.append(closes[-1] + d)
return closes
# ── Pure computation ─────────────────────────────────────────────────────────
def test_daily_returns_are_bankroll_normalized_deltas():
closes = [0.0, 100.0, 50.0, 50.0]
assert daily_returns(closes, BANKROLL) == pytest.approx([0.01, -0.005, 0.0])
def test_compute_sharpe_matches_manual_formula():
deltas = [10.0, 14.0, 8.0, 12.0, 6.0, 13.0, 9.0]
closes = _closes_from_deltas(deltas)
rets = [d / BANKROLL for d in deltas]
expected = mean(rets) / stdev(rets) * 365 ** 0.5
assert compute_sharpe(closes, BANKROLL) == pytest.approx(expected)
assert compute_sharpe(closes, BANKROLL) > 0
def test_compute_sharpe_undefined_cases_return_none():
assert compute_sharpe([], BANKROLL) is None
assert compute_sharpe([0.0], BANKROLL) is None
assert compute_sharpe([0.0, 50.0], BANKROLL) is None # only 1 return
assert compute_sharpe([0.0] * 40, BANKROLL) is None # zero variance
# ── Minimum-sample gate ───────────────────────────────────────────────────────
def test_gate_blocks_current_situation_one_resolved_trade():
"""~40 flat days plus a single +299 jump, 1 resolved trade → no Sharpe."""
closes = [0.0] * 35 + [299.06] * 5
sharpe, status = sharpe_with_gate(closes, BANKROLL, resolved_count=1)
assert sharpe is None
assert status == SHARPE_INSUFFICIENT
# The raw (ungated) value would exist and be wildly misleading:
assert compute_sharpe(closes, BANKROLL) is not None
def test_gate_blocks_too_few_days_even_with_enough_resolved():
closes = _closes_from_deltas([10.0, -5.0] * 10) # 21 days < 30
sharpe, status = sharpe_with_gate(closes, BANKROLL, resolved_count=15)
assert sharpe is None
assert status == SHARPE_INSUFFICIENT
def test_gate_passes_with_sufficient_sample():
deltas = [10.0, 14.0, 8.0, 12.0, 6.0] * 8 # 40 returns → 41 days
closes = _closes_from_deltas(deltas)
sharpe, status = sharpe_with_gate(closes, BANKROLL, resolved_count=MIN_RESOLVED_TRADES)
assert status == SHARPE_OK
assert sharpe == pytest.approx(compute_sharpe(closes, BANKROLL))
def test_gate_flat_curve_with_sufficient_sample_is_zero_variance():
sharpe, status = sharpe_with_gate([0.0] * 40, BANKROLL, resolved_count=12)
assert sharpe is None
assert status == SHARPE_ZERO_VARIANCE
# ── /api/summary ─────────────────────────────────────────────────────────────
class FakeDB:
def __init__(self, daily_closes, resolved_count, total_trades=60,
win_rate=0.6, calibration=0.8):
self._closes = daily_closes
self._resolved = resolved_count
self._total = total_trades
self._win_rate = win_rate
self._calibration = calibration
async def get_metrics_history(self, days=1):
return [{
"win_rate": self._win_rate,
"calibration_score": self._calibration,
"unrealized_pnl_est": 0.0,
"realized_pnl": 299.06,
"total_pnl": 299.06,
}]
async def compute_metrics_from_db(self):
return {
"total_trades": self._total,
"open_count": self._total - self._resolved,
"closed_count": self._resolved,
"resolved_count": self._resolved,
}
async def get_open_position_data(self):
return {}, 0.0
async def get_recently_closed_inverted(self, hours=24):
return set()
async def get_legacy_incomplete_count(self):
return 0
async def get_daily_pnl_closes(self):
return list(self._closes)
def _summary(db, monkeypatch) -> dict:
monkeypatch.setattr(api_main, "db", db)
monkeypatch.delenv("PAPER_BANKROLL", raising=False)
return asyncio.run(api_main.get_summary())
def test_api_insufficient_sample_returns_null_with_explanation(monkeypatch):
"""Current prod situation: 1 resolved, ~40 days → null Sharpe, not ready."""
db = FakeDB(daily_closes=[0.0] * 35 + [299.06] * 5, resolved_count=1)
s = _summary(db, monkeypatch)
assert s["sharpe_ratio"] is None
assert s["sharpe_status"] == SHARPE_INSUFFICIENT
assert s["resolved_count"] == 1
assert s["min_resolved_required"] == MIN_RESOLVED_TRADES == 10
assert s["days_observed"] == 40
assert s["min_days_required"] == MIN_DAYS_OBSERVED == 30
# One lucky resolved trade must never promote, even with perfect
# win_rate/calibration and 50+ trades.
assert s["promotion_ready"] is False
def test_api_sharpe_appears_with_sufficient_sample(monkeypatch):
deltas = [10.0, 14.0, 8.0, 12.0, 6.0] * 8
db = FakeDB(daily_closes=_closes_from_deltas(deltas), resolved_count=12)
s = _summary(db, monkeypatch)
assert s["sharpe_status"] == SHARPE_OK
assert s["sharpe_ratio"] == pytest.approx(
compute_sharpe(_closes_from_deltas(deltas), BANKROLL)
)
assert s["sharpe_ratio"] >= 0.5
assert s["promotion_ready"] is True
def test_api_not_ready_when_sharpe_below_threshold(monkeypatch):
# Zero-drift curve: mean return ~0 → Sharpe ≈ 0 < 0.5
deltas = [50.0, -50.0] * 20
db = FakeDB(daily_closes=_closes_from_deltas(deltas), resolved_count=12)
s = _summary(db, monkeypatch)
assert s["sharpe_status"] == SHARPE_OK
assert s["sharpe_ratio"] < 0.5
assert s["promotion_ready"] is False
def test_api_not_ready_when_metrics_null(monkeypatch):
db = FakeDB(
daily_closes=_closes_from_deltas([10.0, 14.0, 8.0, 12.0, 6.0] * 8),
resolved_count=12,
win_rate=None,
calibration=None,
)
s = _summary(db, monkeypatch)
assert s["sharpe_status"] == SHARPE_OK
assert s["promotion_ready"] is False
# ── MetricsTracker: no hardcoded 0.0 in the snapshot ─────────────────────────
class FakeTrackerDB:
def __init__(self, daily_closes, resolved_count):
self._closes = daily_closes
self._resolved = resolved_count
self.saved = None
async def compute_metrics_from_db(self):
return {
"total_trades": 60,
"open_count": 40,
"closed_count": 20,
"resolved_count": self._resolved,
"wins_realized": self._resolved,
"unrealized_pnl_est": 0.0,
"realized_pnl": 100.0,
"total_deployed": 1000.0,
"total_fees": 20.0,
"calibration_score": 0.8,
}
async def get_daily_pnl_closes(self):
return list(self._closes)
async def save_daily_metrics(self, metrics):
self.saved = metrics
def test_tracker_stores_null_sharpe_below_gate(monkeypatch):
monkeypatch.delenv("PAPER_BANKROLL", raising=False)
db = FakeTrackerDB(daily_closes=[0.0] * 35 + [299.06] * 5, resolved_count=1)
asyncio.run(MetricsTracker(db).update_daily_summary())
assert db.saved is not None
assert db.saved["sharpe_ratio"] is None
def test_tracker_stores_real_sharpe_above_gate(monkeypatch):
monkeypatch.delenv("PAPER_BANKROLL", raising=False)
closes = _closes_from_deltas([10.0, 14.0, 8.0, 12.0, 6.0] * 8)
db = FakeTrackerDB(daily_closes=closes, resolved_count=12)
asyncio.run(MetricsTracker(db).update_daily_summary())
assert db.saved["sharpe_ratio"] == pytest.approx(
compute_sharpe(closes, BANKROLL)
)
assert db.saved["sharpe_ratio"] != 0.0