Behavior-preserving cleanup (345 tests pass, ruff clean):
- scheduler: replace 62 inline logger.x(json.dumps({...})) calls with a
_log_event helper, and collapse 11 identical _job_runtime dicts into an
_idle_runtime() factory over _JOB_NAMES.
- settings: add app/services/settings_store.py (get_setting/get_value/get_map/
upsert_setting) and route ~13 hand-rolled SystemSetting queries + two
identical _settings_map helpers through it.
- scoring.get_rankings: collapse the per-ticker N+1 (3-4 queries + a commit each)
into 2 bulk reads + a single conditional commit; drop the redundant re-fetch.
Lazy recompute-on-read is preserved. Adds first tests for get_rankings.
Net ~ -245 lines across the touched modules.
min_target_probability is gone: it filtered on the probability model the
calibration has repeatedly shown to be weak and overconfident, it was redundant
with the momentum gate, and as an off-by-default knob it just invited bad tuning.
Removed from the backend gate, activation config/schema, the frontend mirror
(qualifiesSetup / activationSummary), and ActivationSettings. The probability
model stays where it does real work (primary-target selection + display).
Charts: with multi-year history the all-bars default was unreadable. Added
time-range presets (1M / 3M / 6M / YTD / 1Y / 3Y / 5Y / All), defaulting to 1Y;
clicking a preset always re-applies (snaps back after a manual zoom). Y-axis
autoscale and wheel-zoom / drag-pan were already there.
339 backend tests pass; frontend build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 5-year backtest confirmed the EV gate adds negative value (high threshold =
worst expectancy) and that 12-1 month momentum is the one price signal with a
plausible, right-signed cross-sectional IC (~0.05). So "qualified" now means:
clears the R:R + confidence floors AND the ticker ranks in the top
`min_momentum_percentile` of the universe by 12-1 momentum that week.
- qualification.py: drop expected_value_r / the EV gate; add a momentum-percentile
gate (duck-typed `momentum_percentile`, only enforced when attached + threshold
set, else defers to floors). Mirrored in frontend qualification.ts.
- activation config/schema: min_expected_value -> min_momentum_percentile
(default 80 = top quintile). ActivationSettings, DashboardPage (ranks/【shows】
momentum instead of EV), and the BacktestPanel sweep follow.
- backtest: rank each ISO week's universe by 12-1 momentum, assign a percentile,
and qualify the top slice; the sweep now sweeps the percentile cutoff.
Also offload the backtest's per-ticker compute to a worker thread so the heavy
~5y run no longer blocks the API event loop (the "backend offline" flicker).
Production setups don't carry momentum_percentile yet — wiring the scanner to
attach it (a universe momentum-rank step) is the next step; until then the live
gate defers to floors while the backtest measures the momentum selection. 330
backend tests pass; frontend build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two changes so the cross-sectional signal results can actually be trusted.
(a) History depth — the binding constraint. Ingestion defaulted to 365 days, so
long-lookback factors (12-month momentum, 52-week high) were only computable on a
handful of weeks at the tail, and every IC reflected a single market regime.
- New `settings.ohlcv_history_days` (default 1825 ≈ 5y); new tickers backfill this
far instead of 1 year.
- New manual "data_backfill" job (Admin → Jobs) re-fetches the full window for
every ticker, ignoring incremental resume — run once to deepen existing
1-year histories. Idempotent (upsert); resumes after rate limits.
(b) Factor-IC honesty. The IC was averaged over weekly rebalances whose 30-day
forward windows overlap, inflating the t-stat ~sqrt(6)x.
- IC now measured on NON-OVERLAPPING windows (weeks thinned to ~HORIZON apart).
- Each signal carries a `reliable` flag (>= 12 independent windows); BacktestPanel
greys out and de-stars thin signals so a lucky 9-week IC of 0.3 can't masquerade
as an edge.
332 backend tests pass; frontend build clean. No migration (config + job + an
added JSON field on the cached backtest report).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosing "no qualified signals for 5 days": setups were generated but none
qualified. The gate required BOTH a high min_rr (2.0) AND a high
min_target_probability (60), which became contradictory after the Jun-15
probability recalibration — probability already embeds R:R via the 1/(rr+1) ruin
term, so high-R:R targets are inherently low-probability and nothing cleared both.
Gate is now expected value (R): p*rr - (1-p) from the primary target's
probability. R:R and confidence stay as floors; high-conviction / exclude-conflicts
/ min-target-probability become optional tighteners (default off). Defaults:
min_expected_value=0.15, min_rr=1.2, min_confidence=55. EV is only enforced when
computable. Migration 009 clears stored activation_* rows so the new defaults
apply. Backtest sweeps min_expected_value instead of target probability.
Scheduling: pipelines are now cron-configurable in Admin -> Jobs. daily_pipeline
(full, default 0 7 * * *) plus a new light intraday_pipeline (OHLCV + outcome eval,
default hourly US session) that keeps prices/live-R:R current without setup churn.
Fundamentals on its own early weekly cron. Timezone configurable (default
Europe/Berlin). Moving interval->CronTrigger also fixes the restart-deferral bug
where an interval job's countdown resets on every process restart.
319 backend unit tests pass; frontend tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulled the fundamental collector out of the daily pipeline (where it re-fetched
near-identical numbers every day and burned free-tier API quota) and made it an
independent weekly job. P/E/market-cap drift with price but the score buckets
them coarsely; revenue growth and earnings surprise only change at quarterly
earnings. Added "weekly" to the frequency map; fundamental_fetch_frequency now
defaults to weekly (configurable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jobs were independent 24h timers with no ordering, so the scanner could run on
stale OHLCV, and manual runs desynced the offsets. New daily_pipeline job runs
the data→signal flow in dependency order: OHLCV → fundamentals → sentiment →
R:R scan → outcome eval (+paper close) → market regime. Each step keeps its own
enable flag and runtime status; a failing step is logged and the pipeline
continues.
The member jobs are registered PAUSED (no auto-fire) so they only run via the
pipeline — but stay manually triggerable from Admin → Jobs (shown as "runs in
daily pipeline"). Alerts (hourly), ticker universe sync, and backtest keep their
own independent cadence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replays the price-derived engine over stored OHLCV: at each weekly as-of date,
rebuild the setup from bars <= D (no lookahead) and walk the actual forward bars
for the realized outcome. Reports realized hit-rate/expectancy of qualified
setups (and all setups, by direction) plus a probability calibration curve
(predicted target prob vs realized hit rate).
Reuses pure functions throughout; extracted compute_technical_from_arrays /
compute_momentum_from_closes from scoring_service so live and backtest stay in
sync. Runs as a weekly/triggerable 'backtest' job caching the report in a
SystemSetting; GET /backtest/report serves it. Sentiment/fundamentals held
neutral (no point-in-time history) — calibrates the price/S-R/probability machinery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New market_regime_service computes a benchmark (SPY) trend from its 50/200-day
SMAs, cached in a SystemSetting and refreshed by a nightly job; GET /market/regime
exposes it. Dashboard shows a regime banner; setup cards flag a counter-trend
caution when a setup fights the regime (LONG in a bearish market / SHORT in a
bullish one). Informational only — nothing is suppressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the action loop — instead of polling the dashboard, the platform pushes
actionable signals to Telegram. New hourly 'alerts' job dispatches four
toggleable triggers, deduped via a new alert_log table (cooldown-based for
qualified/S-R/digest, watermark-based for score deterioration). Admin → Settings
gains a Telegram panel (write-only bot token, chat ID, per-trigger toggles, Send
Test). Credentials follow DB > env precedence (TELEGRAM_BOT_TOKEN / _CHAT_ID).
Backend: alert_service + AlertLog model + migration 005, scheduler job, admin
endpoints/schema. Frontend: AlertSettings panel, hooks, api, types.
Deploy: run alembic upgrade (new alert_log table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track Record: new "Reset" action (POST /admin/track-record/reset) deletes all
trade setups so stats start fresh after material scoring/setup changes — live
setups regenerate on the next scan. Guarded by a confirm dialog.
Recommendation config: remove distance_penalty_factor, which was exposed in the
admin UI but consumed nowhere (the touch-probability model superseded it). A
knob that silently does nothing is worse than no knob. Remaining defaults are
left as-is — they're reasonable, and the honest way to tune them is backtesting
against accumulated outcomes, not invented "researched" numbers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make "qualified" mean an edge candidate, not just R:R + confidence.
The gate now also requires (all admin-configurable, defaults on):
- high conviction: recommended_action LONG_HIGH / SHORT_HIGH only
- clean read: risk_level Low (no contradicting signals)
- probable primary target: best target probability >= min (default 60)
- Shared predicate: app/services/qualification.py +
frontend/src/lib/qualification.ts (mirrored)
- Activation config extended (min_target_probability,
require_high_conviction, exclude_conflicts) with bool-aware
get/update + validation
- /trades/performance switched to ?qualified_only=true, applying
the full gate server-side; confidence breakdown stays unfiltered
- Dashboard "Qualified", Signals "Qualified only" toggle, and
Track Record all use the one gate; Admin gains the new controls
Sentiment provider runtime config (prior change) included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Admin-configurable thresholds (min R:R, default 2.0; min confidence,
default 70%) defining what counts as an actionable signal:
- Admin Settings: new Activation Thresholds panel
(GET/PUT /admin/settings/activation)
- GET /trades/activation exposes values to all users with access
- Signals/Setups: filters initialize from activation values
- Track Record: "Qualified signals only" toggle (default on) via
min_rr/min_confidence params on /trades/performance; the
confidence breakdown always covers the full population so the
thresholds can be validated against outcomes
- Dashboard: "Qualified" metric and qualified-first Top Setups
- Outcome evaluator unchanged: every setup is still evaluated
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the feedback loop on R:R scanner signals:
- Nightly outcome_evaluator job replays unresolved setups against daily
OHLCV bars: target_hit / stop_hit / ambiguous (same-bar, counted as
loss) / expired after OUTCOME_EVALUATION_MAX_BARS (default 30)
- Migration 004: evaluated_at + outcome_date on trade_setups
- GET /trades/performance: hit rate, expectancy (avg R), total R with
breakdowns by direction, recommended action, and confidence bucket
- New Performance page (stat cards, breakdown tables, Evaluate Now,
methodology disclosure) wired into sidebar and mobile nav
- 17 new unit tests for evaluation logic and stats aggregation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dennisthiessen
Dennis Thiessen