Three follow-ups from review of the A6 commits.
The cache-summary re-finalize raised a second durable system event on a failed
run: _runtime_finish emits for `error`/`rate_limited`, and the dedup key
includes the message, so "SEC unavailable" and "SEC unavailable · cache 511 · 2
score inputs changed" landed as two unacknowledged Admin events. Adds
`emit_event` so a re-finalize that only rewords an outcome stays silent, with a
regression test asserting exactly one event.
The rollback section still claimed disabling the SEC job freezes the cache — the
opposite of what the same page says two lines earlier, and of what the code now
does. Rewritten: there is no Admin cache-off switch, restoring `fundamental_data`
alone is temporary because the next run rebuilds it from the same snapshots and
code, and a real freeze means stopping the service.
Remaining "shadow" wording: the two import jobs have never been shadow since
activation, so `_run_shadow_import` -> `_run_source_import`, its section heading,
the deployment doc's job label, and the plan doc's "production switch remains"
handoff paragraph are all brought up to date.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runtime message only appended the cache summary when the import itself
completed. On a deferred, failed or source-locked run Admin → Jobs showed just
the import outcome, so an operator had no signal that `fundamental_data` had
advanced — contradicting the claim now made in the docstring, the schedule hint
and the deployment doc.
The import status still varies and stays the headline; the cache summary is
appended to all of them. Adds a source-locked regression test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The A5 cutover has been on and observed in production, so SEC Company Facts +
DoltHub earnings are already the live source for `fundamental_data`. This
removes everything the legacy path still occupied.
Gone: the three providers and their config/env keys; the weekly
`fundamental_collector` job; the cutover toggle (SEC + Dolt is now the
unconditional path, so `off` can no longer silently freeze scoring inputs); the
A5 parity report, whose deltas became structurally zero once the candidate
builder started writing the table it compared against; and the FMP tier of
universe bootstrap.
Two behavioral notes:
- Disabling **SEC Fundamentals Import** now stops the SEC network fetch only.
The local cache refresh moved outside the job-enable check, because candidates
also derive from daily closes and earnings events — freezing those on an
ingestion pause would stale scoring with no fallback left to recover from.
- `/ingestion/fetch?sources=fundamentals` still accepts the key and reports
`skipped`; there is no per-ticker fetch any more.
Migration 029 does not blanket-delete the leftover settings rows. Migrations run
before the service restart, and pre-A6 code reads an absent `job_*_enabled` row
as *enabled* — so the two behavior-bearing keys become tombstones pinned to safe
values (hidden in Admin) and only the inert three are deleted. Removing the
provider keys from the production `.env` is the matching rollout step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review of the shadow book found seven ways the two books could leak into
each other; all are fixed here. The most serious silently invalidated the
comparison the shadow book exists to make.
- Shadow holdings no longer suppress the manual candidate list. The
open-trade exclusion filtered on any book, so shadow taking the
top-ranked names removed exactly those from the user's list and alerts,
confining the discretionary book to leftovers. Scoped to the manual
book. Closed-trade alerts and paper-book equity were leaking the same
way and are likewise scoped.
- Shadow sizing now matches _simulate_portfolio: min(1% risk, 20% notional
cap, available cash) from marked equity, plus the sub- dust guard.
Previously risk-only from realized equity, so a tight stop produced a
multiples-of-equity leveraged position the strategy would never take.
- Shadow only trades setups from the scan that just ran (<6h old) with one
setup per ticker. A failed or disabled scan step could otherwise open
positions from a prior session at stale prices.
- Gate-reset transitions are observed for both books, so a shadow stop-out
completes fail -> requalify instead of staying locked forever.
- Manual list/close endpoints default to the manual book and reject
hand-closing shadow trades; the performance endpoint is scoped to the
caller so 'your picks' is not every user's book.
- run_shadow_book is registered as a paused job so Admin can trigger it.
Also anchors three pre-existing paper-trade tests (and the new alpaca
window test) on the UTC date. They build fixtures from the local date but
the service stamps opened_at in UTC, so they failed only between 00:00 and
02:00 in a UTC+hh timezone -- latent on ba2df8b, exposed by the clock.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts,
put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day
requalify semantics, stamp paper trades fill_mode=near_close, and migrate
stored schedule_* keys to America/New_York.
Three usability fixes:
1. Global ticker search in the sidebar (TickerSearch) — typeahead over the
tracked universe that opens a ticker's detail page without adding it to the
watchlist. Also wired into the mobile nav.
2. Watchlist table shows the ticker's 12-1 momentum percentile (the top-pick
selector) instead of the noisy full S/R-level list. Enriched from the setup
already loaded in watchlist_service._enrich_entry — no extra query.
3. Alpha vs the S&P 500 on paper trades (open + closed). New benchmark_prices
table + benchmark_service store SPY daily closes (a standalone series, not a
Ticker, so it never enters the scanner / momentum ranking / rankings) via a
new daily-pipeline step. paper_trade_service computes per-trade
benchmark_return / alpha_pct / alpha_usd over each holding period; the open-
trades table, dashboard, and closed-trades panel surface per-trade and total
alpha. The list read path never makes a provider call.
Deploy: alembic upgrade head, then run the benchmark/daily job once to populate
SPY closes (alpha shows "—" until then).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a leading-by-construction candidate and the harness to measure whether it
actually leads regime breaks, before any of it earns weight in the live index.
- breadth_service: % of the stored universe above its own 200-DMA + a divergence
score (benchmark price up while breadth falls, nudged by low breadth). Genuinely
leading because it keys on divergence, not level. Not wired into the live score.
- event_study_service: detect drawdown events on the benchmark, then measure each
indicator's median lead time (event-centered) and precision/recall vs. the base
rate (signal-centered). Compares breadth-divergence against the deterministic
coincident price composite (reuses the regime price sub-scores). Price/breadth
only — reproducible, no LLM/FRED.
- Manual "Event Study" job (Admin → Jobs), GET /regime/event-study, and an
inline early-warning panel on the Regime tab with an honest small-sample caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new /regime tab scoring how far the AI/Tech bull regime has deteriorated
toward a re-rating as a single 0-100 index with per-signal breakdown and a
7/30-day trend. Intentionally decoupled: nothing reads its output to gate or
score trades — the daily-pipeline membership is scheduling only.
- regime_monitor_service: price sub-scores (P1-P6 via Alpaca, like
market_regime), VIX + HY credit spreads via a small FRED helper, weighted
aggregation over available signals (missing source -> n/a, dropped from the
denominator), one snapshot row/day, and a ~90-day history backfill by
replaying the already-fetched series as-of each past day.
- F1/F3 fundamentals proposed by the configured grounded LLM (reuses
sentiment_provider_service config resolution), with a manual override + lock.
- regime_snapshots table (migration 011); endpoints on the existing market
router; admin-editable weights/threshold; standalone /regime page.
Data needs: prices via Alpaca, VIX/credit via FRED (optional key — signals show
n/a without it). No LLM needed for history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two changes so the cross-sectional signal results can actually be trusted.
(a) History depth — the binding constraint. Ingestion defaulted to 365 days, so
long-lookback factors (12-month momentum, 52-week high) were only computable on a
handful of weeks at the tail, and every IC reflected a single market regime.
- New `settings.ohlcv_history_days` (default 1825 ≈ 5y); new tickers backfill this
far instead of 1 year.
- New manual "data_backfill" job (Admin → Jobs) re-fetches the full window for
every ticker, ignoring incremental resume — run once to deepen existing
1-year histories. Idempotent (upsert); resumes after rate limits.
(b) Factor-IC honesty. The IC was averaged over weekly rebalances whose 30-day
forward windows overlap, inflating the t-stat ~sqrt(6)x.
- IC now measured on NON-OVERLAPPING windows (weeks thinned to ~HORIZON apart).
- Each signal carries a `reliable` flag (>= 12 independent windows); BacktestPanel
greys out and de-stars thin signals so a lucky 9-week IC of 0.3 can't masquerade
as an edge.
332 backend tests pass; frontend build clean. No migration (config + job + an
added JSON field on the cached backtest report).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosing "no qualified signals for 5 days": setups were generated but none
qualified. The gate required BOTH a high min_rr (2.0) AND a high
min_target_probability (60), which became contradictory after the Jun-15
probability recalibration — probability already embeds R:R via the 1/(rr+1) ruin
term, so high-R:R targets are inherently low-probability and nothing cleared both.
Gate is now expected value (R): p*rr - (1-p) from the primary target's
probability. R:R and confidence stay as floors; high-conviction / exclude-conflicts
/ min-target-probability become optional tighteners (default off). Defaults:
min_expected_value=0.15, min_rr=1.2, min_confidence=55. EV is only enforced when
computable. Migration 009 clears stored activation_* rows so the new defaults
apply. Backtest sweeps min_expected_value instead of target probability.
Scheduling: pipelines are now cron-configurable in Admin -> Jobs. daily_pipeline
(full, default 0 7 * * *) plus a new light intraday_pipeline (OHLCV + outcome eval,
default hourly US session) that keeps prices/live-R:R current without setup churn.
Fundamentals on its own early weekly cron. Timezone configurable (default
Europe/Berlin). Moving interval->CronTrigger also fixes the restart-deferral bug
where an interval job's countdown resets on every process restart.
319 backend unit tests pass; frontend tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulled the fundamental collector out of the daily pipeline (where it re-fetched
near-identical numbers every day and burned free-tier API quota) and made it an
independent weekly job. P/E/market-cap drift with price but the score buckets
them coarsely; revenue growth and earnings surprise only change at quarterly
earnings. Added "weekly" to the frequency map; fundamental_fetch_frequency now
defaults to weekly (configurable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jobs were independent 24h timers with no ordering, so the scanner could run on
stale OHLCV, and manual runs desynced the offsets. New daily_pipeline job runs
the data→signal flow in dependency order: OHLCV → fundamentals → sentiment →
R:R scan → outcome eval (+paper close) → market regime. Each step keeps its own
enable flag and runtime status; a failing step is logged and the pipeline
continues.
The member jobs are registered PAUSED (no auto-fire) so they only run via the
pipeline — but stay manually triggerable from Admin → Jobs (shown as "runs in
daily pipeline"). Alerts (hourly), ticker universe sync, and backtest keep their
own independent cadence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replays the price-derived engine over stored OHLCV: at each weekly as-of date,
rebuild the setup from bars <= D (no lookahead) and walk the actual forward bars
for the realized outcome. Reports realized hit-rate/expectancy of qualified
setups (and all setups, by direction) plus a probability calibration curve
(predicted target prob vs realized hit rate).
Reuses pure functions throughout; extracted compute_technical_from_arrays /
compute_momentum_from_closes from scoring_service so live and backtest stay in
sync. Runs as a weekly/triggerable 'backtest' job caching the report in a
SystemSetting; GET /backtest/report serves it. Sentiment/fundamentals held
neutral (no point-in-time history) — calibrates the price/S-R/probability machinery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New market_regime_service computes a benchmark (SPY) trend from its 50/200-day
SMAs, cached in a SystemSetting and refreshed by a nightly job; GET /market/regime
exposes it. Dashboard shows a regime banner; setup cards flag a counter-trend
caution when a setup fights the regime (LONG in a bearish market / SHORT in a
bullish one). Informational only — nothing is suppressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the action loop — instead of polling the dashboard, the platform pushes
actionable signals to Telegram. New hourly 'alerts' job dispatches four
toggleable triggers, deduped via a new alert_log table (cooldown-based for
qualified/S-R/digest, watermark-based for score deterioration). Admin → Settings
gains a Telegram panel (write-only bot token, chat ID, per-trigger toggles, Send
Test). Credentials follow DB > env precedence (TELEGRAM_BOT_TOKEN / _CHAT_ID).
Backend: alert_service + AlertLog model + migration 005, scheduler job, admin
endpoints/schema. Frontend: AlertSettings panel, hooks, api, types.
Deploy: run alembic upgrade (new alert_log table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the feedback loop on R:R scanner signals:
- Nightly outcome_evaluator job replays unresolved setups against daily
OHLCV bars: target_hit / stop_hit / ambiguous (same-bar, counted as
loss) / expired after OUTCOME_EVALUATION_MAX_BARS (default 30)
- Migration 004: evaluated_at + outcome_date on trade_setups
- GET /trades/performance: hit rate, expectancy (avg R), total R with
breakdowns by direction, recommended action, and confidence bucket
- New Performance page (stat cards, breakdown tables, Evaluate Now,
methodology disclosure) wired into sidebar and mobile nav
- 17 new unit tests for evaluation logic and stats aggregation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
