Two independent bugs left the near-close scan running on the previous
session's close, silently degrading live execution to the stale_close
floor (~1.57 Sharpe) instead of the intended ~1.77 close-fill case.
1. OHLCV window never covered the current day. Daily bars are stamped at
session start (04:00Z under EDT), so an end of midnight-on-end_date
landed before that day's bar and dropped it. Widening the window alone
fails the whole request with 'subscription does not permit querying
recent SIP data', so end is also clamped to now-20min. Today's bar is
now returned, roughly 20 minutes behind live -- within the staleness
the near-close design already assumed.
Intraday runs therefore store a partial bar and ingestion progress
reaches today, which made incremental resume skip the after-close
refresh entirely. collect_ohlcv_final() re-pulls the last sessions so
the consolidated bar overwrites the partial one before outcome eval.
2. APScheduler's from_crontab() passes day-of-week to its own field where
0=Monday, so '1-5' meant Tue-Sat: every Monday was skipped and the
scanner ran Saturdays on stale data. Weekday schedules now use names.
Stored settings already corrected via Admin; this fixes the defaults.
Tests cover both: today's bar inside the window, the delayed-data clamp,
historical windows untruncated, and a week of fire times asserting Monday
is present and weekends are not.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regime is computed at 02:00 ET; without a morning alert pass, quadrant
changes waited until 15:30. Dispatcher is change-driven so quiet days stay
quiet. Drop unused alerts_frequency config.
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts,
put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day
requalify semantics, stamp paper trades fill_mode=near_close, and migrate
stored schedule_* keys to America/New_York.
Persist job and ingestion warnings/errors for 7 days, surface a dismissible top-nav badge, treat stale OHLCV as a warning (e.g. ticker renames), and show market bar age on the ticker freshness chip.
Three usability fixes:
1. Global ticker search in the sidebar (TickerSearch) — typeahead over the
tracked universe that opens a ticker's detail page without adding it to the
watchlist. Also wired into the mobile nav.
2. Watchlist table shows the ticker's 12-1 momentum percentile (the top-pick
selector) instead of the noisy full S/R-level list. Enriched from the setup
already loaded in watchlist_service._enrich_entry — no extra query.
3. Alpha vs the S&P 500 on paper trades (open + closed). New benchmark_prices
table + benchmark_service store SPY daily closes (a standalone series, not a
Ticker, so it never enters the scanner / momentum ranking / rankings) via a
new daily-pipeline step. paper_trade_service computes per-trade
benchmark_return / alpha_pct / alpha_usd over each holding period; the open-
trades table, dashboard, and closed-trades panel surface per-trade and total
alpha. The list read path never makes a provider call.
Deploy: alembic upgrade head, then run the benchmark/daily job once to populate
SPY closes (alpha shows "—" until then).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tiered, uncapped sentiment scope so the names that matter are never shown
without sentiment.
- Priority (always fully refreshed): top-pick feeders — momentum leaders with a
tradeable long setup over the R:R floor (the tickers that are, or could become
with positive sentiment, the dashboard top pick) — plus the curated watchlist
and open paper trades.
- Filler: top-N by composite, a discovery net, fetched after the priority set so
a mid-run rate limit lands the important names first.
- Removed the per-run cap (sentiment_max_per_run): the relevant set is naturally
bounded (watchlist <= 20, composite <= top_composite), so a full refresh stays
inside the free tier. extra="ignore" keeps a stale env var from breaking startup.
- Refresh window 72h -> 120h (5 days): sentiment shifts slowly, score window is 7d.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a leading-by-construction candidate and the harness to measure whether it
actually leads regime breaks, before any of it earns weight in the live index.
- breadth_service: % of the stored universe above its own 200-DMA + a divergence
score (benchmark price up while breadth falls, nudged by low breadth). Genuinely
leading because it keys on divergence, not level. Not wired into the live score.
- event_study_service: detect drawdown events on the benchmark, then measure each
indicator's median lead time (event-centered) and precision/recall vs. the base
rate (signal-centered). Compares breadth-divergence against the deterministic
coincident price composite (reuses the regime price sub-scores). Price/breadth
only — reproducible, no LLM/FRED.
- Manual "Event Study" job (Admin → Jobs), GET /regime/event-study, and an
inline early-warning panel on the Regime tab with an honest small-sample caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new /regime tab scoring how far the AI/Tech bull regime has deteriorated
toward a re-rating as a single 0-100 index with per-signal breakdown and a
7/30-day trend. Intentionally decoupled: nothing reads its output to gate or
score trades — the daily-pipeline membership is scheduling only.
- regime_monitor_service: price sub-scores (P1-P6 via Alpaca, like
market_regime), VIX + HY credit spreads via a small FRED helper, weighted
aggregation over available signals (missing source -> n/a, dropped from the
denominator), one snapshot row/day, and a ~90-day history backfill by
replaying the already-fetched series as-of each past day.
- F1/F3 fundamentals proposed by the configured grounded LLM (reuses
sentiment_provider_service config resolution), with a manual override + lock.
- regime_snapshots table (migration 011); endpoints on the existing market
router; admin-editable weights/threshold; standalone /regime page.
Data needs: prices via Alpaca, VIX/credit via FRED (optional key — signals show
n/a without it). No LLM needed for history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Qualified setups could carry no sentiment because the sentiment job scoped
its relevant-set to watchlist + open trades + top-N composite score, while
the activation gate qualifies on 12-1 momentum percentile — a different axis.
A top-momentum ticker outside the composite top-N never got sentiment, so the
R:R scan enhanced it as neutral.
Add the gate's momentum leaders (percentile >= activation min_momentum_percentile)
to the sentiment relevant-set so scope tracks the gate. Best-effort: a momentum
or config failure falls back to the base set rather than aborting collection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Behavior-preserving cleanup (345 tests pass, ruff clean):
- scheduler: replace 62 inline logger.x(json.dumps({...})) calls with a
_log_event helper, and collapse 11 identical _job_runtime dicts into an
_idle_runtime() factory over _JOB_NAMES.
- settings: add app/services/settings_store.py (get_setting/get_value/get_map/
upsert_setting) and route ~13 hand-rolled SystemSetting queries + two
identical _settings_map helpers through it.
- scoring.get_rankings: collapse the per-ticker N+1 (3-4 queries + a commit each)
into 2 bulk reads + a single conditional commit; drop the redundant re-fetch.
Lazy recompute-on-read is preserved. Adds first tests for get_rankings.
Net ~ -245 lines across the touched modules.
Two changes so the cross-sectional signal results can actually be trusted.
(a) History depth — the binding constraint. Ingestion defaulted to 365 days, so
long-lookback factors (12-month momentum, 52-week high) were only computable on a
handful of weeks at the tail, and every IC reflected a single market regime.
- New `settings.ohlcv_history_days` (default 1825 ≈ 5y); new tickers backfill this
far instead of 1 year.
- New manual "data_backfill" job (Admin → Jobs) re-fetches the full window for
every ticker, ignoring incremental resume — run once to deepen existing
1-year histories. Idempotent (upsert); resumes after rate limits.
(b) Factor-IC honesty. The IC was averaged over weekly rebalances whose 30-day
forward windows overlap, inflating the t-stat ~sqrt(6)x.
- IC now measured on NON-OVERLAPPING windows (weeks thinned to ~HORIZON apart).
- Each signal carries a `reliable` flag (>= 12 independent windows); BacktestPanel
greys out and de-stars thin signals so a lucky 9-week IC of 0.3 can't masquerade
as an edge.
332 backend tests pass; frontend build clean. No migration (config + job + an
added JSON field on the cached backtest report).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosing "no qualified signals for 5 days": setups were generated but none
qualified. The gate required BOTH a high min_rr (2.0) AND a high
min_target_probability (60), which became contradictory after the Jun-15
probability recalibration — probability already embeds R:R via the 1/(rr+1) ruin
term, so high-R:R targets are inherently low-probability and nothing cleared both.
Gate is now expected value (R): p*rr - (1-p) from the primary target's
probability. R:R and confidence stay as floors; high-conviction / exclude-conflicts
/ min-target-probability become optional tighteners (default off). Defaults:
min_expected_value=0.15, min_rr=1.2, min_confidence=55. EV is only enforced when
computable. Migration 009 clears stored activation_* rows so the new defaults
apply. Backtest sweeps min_expected_value instead of target probability.
Scheduling: pipelines are now cron-configurable in Admin -> Jobs. daily_pipeline
(full, default 0 7 * * *) plus a new light intraday_pipeline (OHLCV + outcome eval,
default hourly US session) that keeps prices/live-R:R current without setup churn.
Fundamentals on its own early weekly cron. Timezone configurable (default
Europe/Berlin). Moving interval->CronTrigger also fixes the restart-deferral bug
where an interval job's countdown resets on every process restart.
319 backend unit tests pass; frontend tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulled the fundamental collector out of the daily pipeline (where it re-fetched
near-identical numbers every day and burned free-tier API quota) and made it an
independent weekly job. P/E/market-cap drift with price but the score buckets
them coarsely; revenue growth and earnings surprise only change at quarterly
earnings. Added "weekly" to the frequency map; fundamental_fetch_frequency now
defaults to weekly (configurable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jobs were independent 24h timers with no ordering, so the scanner could run on
stale OHLCV, and manual runs desynced the offsets. New daily_pipeline job runs
the data→signal flow in dependency order: OHLCV → fundamentals → sentiment →
R:R scan → outcome eval (+paper close) → market regime. Each step keeps its own
enable flag and runtime status; a failing step is logged and the pipeline
continues.
The member jobs are registered PAUSED (no auto-fire) so they only run via the
pipeline — but stay manually triggerable from Admin → Jobs (shown as "runs in
daily pipeline"). Alerts (hourly), ticker universe sync, and backtest keep their
own independent cadence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resolve_open_trades walks the daily bars after each open trade and closes it at
the target (target hit) or stop (stop/ambiguous), leaving undecided trades open.
Runs nightly inside the outcome evaluator (so it's coordinated with fresh OHLCV)
and on its manual trigger. New "My Trades" section at the top of Signals → Track
Record shows realized hit-rate, expectancy (avg R), total R, total P&L, and a
closed-trades table — your actual results, separate from the theoretical signal
record below it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Richer LLM output (same grounded call, ~no extra cost):
- All providers now also return a recommendation (buy/hold/avoid) and a thorough
reasoning paragraph; Gemini now actually captures reasoning + grounding
citations (it was dropping them). Stored on sentiment_scores (migration 008),
exposed in the API; display-only — NOT fed into the composite/EV.
- Ticker Sentiment panel shows an "LLM view" badge and a "Full analysis & sources"
expander with the complete reasoning + citations.
Search-budget scoping (Gemini grounding free tier = 5000/mo):
- collect_sentiment now targets only watchlist + open paper trades + top-N by
composite, skips tickers refreshed within sentiment_fresh_hours (72h), and caps
per run (sentiment_max_per_run). Once the relevant set is fresh, runs spend 0
searches until it ages out — bounding monthly usage well under the free tier.
- Widened sentiment lookback to 7d (scoring + display) so sparser collection
still feeds the dimension score.
Deploy: alembic upgrade (sentiment_scores.recommendation). Switch provider to
Gemini Flash in Admin for the cost win (grounded, cheapest).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replays the price-derived engine over stored OHLCV: at each weekly as-of date,
rebuild the setup from bars <= D (no lookahead) and walk the actual forward bars
for the realized outcome. Reports realized hit-rate/expectancy of qualified
setups (and all setups, by direction) plus a probability calibration curve
(predicted target prob vs realized hit rate).
Reuses pure functions throughout; extracted compute_technical_from_arrays /
compute_momentum_from_closes from scoring_service so live and backtest stay in
sync. Runs as a weekly/triggerable 'backtest' job caching the report in a
SystemSetting; GET /backtest/report serves it. Sentiment/fundamentals held
neutral (no point-in-time history) — calibrates the price/S-R/probability machinery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Finnhub's earnings calendar now supplies next_earnings_date through the
fundamentals chain; persisted on fundamental_data (migration 006) and exposed in
the fundamentals API. The recommendation panel warns when earnings fall within
the ~30-day target horizon (a report can gap price through stop/target) and
otherwise shows the next date. Informational only.
Deploy: run alembic upgrade (new fundamental_data.next_earnings_date column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New market_regime_service computes a benchmark (SPY) trend from its 50/200-day
SMAs, cached in a SystemSetting and refreshed by a nightly job; GET /market/regime
exposes it. Dashboard shows a regime banner; setup cards flag a counter-trend
caution when a setup fights the regime (LONG in a bearish market / SHORT in a
bullish one). Informational only — nothing is suppressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "price plan needed in bulk but fine on manual reload": on free
tiers FMP returns only market cap (others 402) and the chain merged that as a
partial success — so when the Finnhub/Alpha Vantage fallbacks were rate-limited
during a bulk run, the chain silently returned market-cap-only and the
collector's backoff never engaged. Manual single fetches worked because the
fallbacks weren't throttled at that moment.
Fixes:
- Chain distinguishes RateLimitError from other failures: if a fallback is
rate-limited and fields are still missing, raise RateLimitError (unless
allow_partial=True) so the collector backs off and retries.
- Bulk job paces requests (fundamental_request_spacing_seconds, default 3s) to
stay under Finnhub's ~60/min, and on retry-exhaustion stores partial data and
continues instead of aborting the whole run.
- Manual fetch passes allow_partial=True so a lone 429 doesn't fail the refresh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the action loop — instead of polling the dashboard, the platform pushes
actionable signals to Telegram. New hourly 'alerts' job dispatches four
toggleable triggers, deduped via a new alert_log table (cooldown-based for
qualified/S-R/digest, watermark-based for score deterioration). Admin → Settings
gains a Telegram panel (write-only bot token, chat ID, per-trigger toggles, Send
Test). Credentials follow DB > env precedence (TELEGRAM_BOT_TOKEN / _CHAT_ID).
Backend: alert_service + AlertLog model + migration 005, scheduler job, admin
endpoints/schema. Frontend: AlertSettings panel, hooks, api, types.
Deploy: run alembic upgrade (new alert_log table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AsyncIOScheduler was constructed with no job_defaults, so APScheduler's default
misfire_grace_time of 1s applied. In this single-process app the scheduler shares
one event loop with the API and all other jobs, so when a daily job came due
while the loop was busy (e.g. the scanner mid-run), the fire was processed >1s
late, flagged a misfire, and skipped — while next_run still advanced 24h, making
the job look healthy though it never ran. Set a generous grace window (1h),
coalesce missed runs into a single catch-up, and cap concurrency at 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scan_rr set the total then called scan_all_tickers as one opaque await, so the
runtime snapshot's processed count stayed 0 until the whole scan finished and
jumped straight to 100%. scan_all_tickers now takes an optional progress_callback
invoked per ticker; the scheduler wires it to _runtime_progress so the sidebar's
live indicator advances as tickers are scanned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make "qualified" mean an edge candidate, not just R:R + confidence.
The gate now also requires (all admin-configurable, defaults on):
- high conviction: recommended_action LONG_HIGH / SHORT_HIGH only
- clean read: risk_level Low (no contradicting signals)
- probable primary target: best target probability >= min (default 60)
- Shared predicate: app/services/qualification.py +
frontend/src/lib/qualification.ts (mirrored)
- Activation config extended (min_target_probability,
require_high_conviction, exclude_conflicts) with bool-aware
get/update + validation
- /trades/performance switched to ?qualified_only=true, applying
the full gate server-side; confidence breakdown stays unfiltered
- Dashboard "Qualified", Signals "Qualified only" toggle, and
Track Record all use the one gate; Admin gains the new controls
Sentiment provider runtime config (prior change) included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the feedback loop on R:R scanner signals:
- Nightly outcome_evaluator job replays unresolved setups against daily
OHLCV bars: target_hit / stop_hit / ambiguous (same-bar, counted as
loss) / expired after OUTCOME_EVALUATION_MAX_BARS (default 30)
- Migration 004: evaluated_at + outcome_date on trade_setups
- GET /trades/performance: hit rate, expectancy (avg R), total R with
breakdowns by direction, recommended action, and confidence bucket
- New Performance page (stat cards, breakdown tables, Evaluate Now,
methodology disclosure) wired into sidebar and mobile nav
- 17 new unit tests for evaluation logic and stats aggregation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
