The ablation judged floors under the target/stop model, but the exit
sweeps point at replacing that exit with a fixed hold — under which the
R:R floor's rationale (bigger payoff at the target) may not apply. Each
ablation row now also carries hold_avg_r / hold_net_avg_r / hold_total_r
(30d hold, initial stop only), so the Phase 3 gate decision can be read
under the exit policy that would actually be used.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_window_setups computed them but _replay_ticker dropped them, so the
ablation's NEUTRAL/tightener checks saw None for every candidate and the
'without confidence floor' / 'without R:R floor' rows collapsed to 0
setups (impossible — removing a floor can only add setups). Regression
test now goes through the real _replay_ticker path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 1 of the strategy-measurement plan — report-only, no production
trading behavior changes:
- Cost haircut: every bucket/sweep now reports net_avg_r/net_total_r
alongside gross (COST_PER_SIDE=0.1% of notional, converted to R via
each setup's stop distance); params carry cost_per_side_pct.
- Gate ablation table: re-qualifies candidates at the current momentum
cutoff with one floor removed per row (confidence / R:R / NEUTRAL /
momentum-only) to show which floors earn their keep.
- Time-based exit sweep: hold 5/10/21/30 days with the initial ATR stop,
exit at the day-N close — the classic momentum implementation, to
disambiguate the wide-trailing result.
- TP sweep extended to +40/+50%, trailing to 25/30% so the optima are
interior instead of starred at the sweep edge.
- BacktestPanel: Net Avg R columns everywhere, gate-ablation and
time-exit tables, stars now mark best net avg R; stale cached reports
still render (all new fields optional/guarded).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Store an optional company name on Ticker (migration 014) and backfill it from
Alpaca's asset list in a single Trading-API call for the whole universe — no
per-ticker fetch. Runs automatically at the end of universe bootstrap and via a
manual "Backfill Names" button (admin) / POST /admin/tickers/backfill-names.
The name ships on /tickers; a shared symbol→name map (useTickerNames) lets any view
show it without its own request. Displayed subtly next to the symbol — in the global
search, the ticker header, and as a small muted line under the symbol in Top Setups
and Open Trades (no extra column, truncated so it never widens the table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_score computed base_score / sentiment_score / sentiment_adjustment /
max_sentiment_adjustment, but the router's _map_composite_breakdown built the
response model from only the five original keys and silently dropped the rest — so
the API always returned null for them. That's why the ticker page showed neither the
"Composite = Base + Sentiment" caption nor the ± marker on the sentiment row despite
the frontend and scoring service both supporting it. Pass the fields through, with a
guard test so they can't be dropped again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Going from no sentiment to a bullish read used to be able to *lower* the composite:
sentiment was blended into the weighted average as an absolute level, so a bullish
75 diluted a ticker already scoring 78. That's backwards for a directional signal.
Now the non-sentiment dimensions form a re-normalized weighted-average base, and
sentiment is applied as a signed adjustment around neutral (50):
composite = clamp(base + MAX_ADJ * (sentiment - 50) / 50)
MAX_ADJ = sentiment weight * 100 (default weight 0.10 → ±10)
Neutral leaves the base unchanged, bullish adds and bearish subtracts (scaled by
confidence, since a 50%-confidence call maps to 50 → no effect), and no sentiment
never penalises. Default sentiment weight 0.15 → 0.10; the weight now means "max ±
points." Composite breakdown exposes base_score/sentiment_score/sentiment_adjustment,
and the ScoreCard shows "Base 78 · sentiment +5.0" plus the per-dimension adjustment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Applies the backtest-validated trailing stop to live paper trading, and surfaces
it transparently.
Exit (A):
- New paper-trade exit policy (paper_exit_mode=trailing, paper_trailing_pct=12),
tunable in Admin → Paper-Trade Exit. resolve_open_trades runs a trailing stop
(initial stop as floor, ratchets up from the peak; target ignored — the
validated rule) and records close_reason (trailing|stop|target|manual; +migration
013).
- list_trades enriches open trades with the live trailing-stop level + distance %.
Open Trades panel shows the active tactic and a Trail Stop column.
Alerts (B):
- Daily digest now lists open trades with unrealized gain, trailing stop, and how
far away it is.
- New "trade closed" alert: one summary per auto-close (trailing/target/stop, not
manual) — direction, reason, days held, P&L abs+%/R — covering wins AND
stop-loss losses. Deduped by trade id; toggle in Admin alerts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third exit model alongside target-vs-stop and the fixed take-profit. The TP sweep
showed the edge lives in the fat tail (avg R keeps rising as you let winners run),
but a fixed wide target is win-rate-brutal and gives everything back on a reversal.
A trailing stop harvests the tail while protecting gains.
Per setup the replay computes the realized R for several trail widths (3/5/7/10/
15/20%) in a single conservative pass — stop ratchets up via max(initial_stop,
peak*(1-trail)), exit on the pullback or at the horizon close, R vs the initial
risk. Aggregated into a trailing sweep (win rate = share closed in profit, avg R,
total R) over the qualified set and shown as a new table in the Backtest panel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Avg R was still rising at the previous top level (+15%), so the optimum was off
the table. Extend TP_LEVELS to 20/25/30% to reveal where letting winners run
stops paying (it plateaus toward "just hold to the horizon close").
Also clarify in the panel that the take-profit model deliberately does NOT use
the setup's S/R target — it's a standalone fixed-% exit; exiting at the target is
the target-vs-stop model above. The two are complementary ends, not in conflict.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The target-vs-stop model counts a near-miss of a far S/R target as a full loss
and ignores the partial gains you actually bank — so it measures a different
strategy than "scalp the early pop, take +8%". Add a realistic take-profit exit
model next to it (original untouched).
Per setup the replay now also records risk%, whether the stop was hit, the
favourable excursion reachable before the stop (MFE), and the horizon-close move.
From those a fixed-take-profit sweep (4/6/8/10/12/15%) is scored in R: bank +X%
if reached before the stop, else -1R, else the horizon close. Hit rate = how
often +X% was banked (the MFE CDF), so you can pick the EV-optimal TP without
top-ticking fantasy. Shown as a new table in the Backtest panel; the IC,
calibration and momentum sweep are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A NEUTRAL ("No Clear Setup") recommendation means the engine found no clear
directional trade, yet such setups could still qualify and even be crowned the
top pick purely on momentum rank (e.g. an extended momentum leader with a far,
5%-probability target). A NEUTRAL signal isn't actionable, so it shouldn't
qualify.
New `exclude_neutral` activation flag (default on): setup_qualifies drops setups
whose recommended_action is NEUTRAL. It lives in the shared gate, so it flows
through the dashboard's qualified/top-pick selection, the track record's
qualified stats, and the backtest (which computes recommended_action and gates on
meets_core). Toggleable in Admin → Settings → Activation; the frontend mirror and
activationSummary ("directional") match.
Re-run the backtest after enabling to confirm it holds/improves expectancy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fetching a symbol the provider doesn't cover (e.g. RHM/Rheinmetall — Alpaca
serves US listings only) returned 0 bars but reported "complete · Successfully
ingested 0 records", which the UI showed as green success.
fetch_and_ingest now returns a distinct `no_data` status when the provider
returns nothing AND the ticker has no history (vs. "already up to date" when bars
exist). The fetch endpoint maps it to a `warning` source status, and the fetch
toast renders it as ⚠ with the provider message instead of success.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The outcome stats were dominated by quick stop-outs: near stops resolve as losses
within days while far targets take weeks, so a young sample (mostly pending,
0 expired) skewed sharply negative (e.g. 13.8% hit / -0.46R vs the backtest's
35.8% / +0.18R) — a maturation artifact, not a real result.
get_performance_stats now counts only setups whose full ~30-day window has
elapsed (_MATURITY_DAYS), so winners had as long as losers (unbiased, and
comparable to the backtest). A new `maturing` count reports the younger setups
held back. The Track Record UI relabels "Evaluated" -> "Matured", shows the
maturing count, and explains the window in the empty state + methodology note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- admin_service: register benchmark_collector in VALID_JOB_NAMES, JOB_LABELS and
PIPELINE_MEMBERS. The Admin → Jobs list is built from these hardcoded sets, not
the scheduler, so the job was registered but invisible/untriggerable.
- deploy.yml:
- SSH: verify the host key (StrictHostKeyChecking=yes) now that known_hosts is
supplied; move private-key cleanup to an `if: always()` step.
- Add a concurrency guard so deploys serialize.
- Health-check the service after restart (127.0.0.1:8998/api/v1/health).
- Align CI Python to 3.12 (matches prod); pip + npm caching.
- Clarify the Postgres service only validates migrations (tests use SQLite);
drop the redundant DATABASE_URL from the pytest step.
- Split the monolithic "Deploy to server" step into named steps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three usability fixes:
1. Global ticker search in the sidebar (TickerSearch) — typeahead over the
tracked universe that opens a ticker's detail page without adding it to the
watchlist. Also wired into the mobile nav.
2. Watchlist table shows the ticker's 12-1 momentum percentile (the top-pick
selector) instead of the noisy full S/R-level list. Enriched from the setup
already loaded in watchlist_service._enrich_entry — no extra query.
3. Alpha vs the S&P 500 on paper trades (open + closed). New benchmark_prices
table + benchmark_service store SPY daily closes (a standalone series, not a
Ticker, so it never enters the scanner / momentum ranking / rankings) via a
new daily-pipeline step. paper_trade_service computes per-trade
benchmark_return / alpha_pct / alpha_usd over each holding period; the open-
trades table, dashboard, and closed-trades panel surface per-trade and total
alpha. The list read path never makes a provider call.
Deploy: alembic upgrade head, then run the benchmark/daily job once to populate
SPY closes (alpha shows "—" until then).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tiered, uncapped sentiment scope so the names that matter are never shown
without sentiment.
- Priority (always fully refreshed): top-pick feeders — momentum leaders with a
tradeable long setup over the R:R floor (the tickers that are, or could become
with positive sentiment, the dashboard top pick) — plus the curated watchlist
and open paper trades.
- Filler: top-N by composite, a discovery net, fetched after the priority set so
a mid-run rate limit lands the important names first.
- Removed the per-run cap (sentiment_max_per_run): the relevant set is naturally
bounded (watchlist <= 20, composite <= top_composite), so a full refresh stays
inside the free tier. extra="ignore" keeps a stale env var from breaking startup.
- Refresh window 72h -> 120h (5 days): sentiment shifts slowly, score window is 7d.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fires once when the regime monitor shifts quadrant (regime index x early
warning), so you don't have to watch the tab. Two guards against spam:
- Hysteresis: each axis only flips once the value crosses its divider by a
margin, so a point parked on a boundary keeps its quadrant instead of
flip-flopping day to day.
- Cooldown: a genuine change stays quiet for a few days after the last alert.
Seeds the baseline silently on first run; reuses the existing Telegram dispatch
+ AlertLog. New per-trigger toggle in Admin → Alerts (on by default).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plots the index, early-warning, and combined scores over time beneath the live
gauges, with a 1M/3M/6M/All range toggle and band reference lines — so the trend
and any divergence between the scores is visible, not just today's snapshot.
- Backend: GET /regime/history + get_regime_history (the three scores per
snapshot date from regime_snapshots).
- Frontend: recharts line chart, lazy-loaded so recharts ships in its own
regime-tab chunk instead of nearly doubling the main bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The early-warning score showed n/a because it required an exact date match
between the live benchmark (Alpaca, may have today's bar) and the stored
universe breadth (DB, often a day behind), which blanked the newest snapshot —
the one the UI displays.
- Look up the divergence as-of the snapshot date (newest value within a 7-day
lag) instead of requiring an exact match.
- Backfill early_warning + combined onto recent existing snapshots (the index
history predates this signal) so the 7/30-day trends populate on the first run
rather than only filling in over the coming weeks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The event study showed the breadth-divergence signal genuinely leads (warned
before 7/11 drawdowns, ~6 weeks median, where the coincident baseline almost
never did). Surface it live to observe before deciding how to embed it — kept
separate from the index, not folded into its weights.
- regime_monitor daily job now computes breadth-divergence live and attaches a
separate early_warning score plus a combined blend (weighted mean, default
0.6/0.4, configurable via combined_weights) to each snapshot, including the
backfill so the 7/30-day trends populate immediately. Stored in breakdown_json
— no schema change. Best-effort: a breadth failure can't break the index.
- get_regime_monitor returns the index, early_warning, and combined scores each
with 7/30-day deltas.
- Regime tab shows three gauges (generalized ScoreGauge): coincident index,
early warning, and a compact combined blend. Stale snapshots render "—".
Note: the daily regime job now also does a universe-wide breadth scan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first run gave only 2 events (N=2 is anecdote, not evidence) and an unfairly
weak coincident baseline, so the +42d lead couldn't be trusted. This makes the
measurement meaningful:
- More, cleaner events: default drawdown threshold 15%→10%, and dedup switched
from "recover to the high" to a rising-edge + cooldown (40d), so distinct
drawdowns each register instead of merging.
- Fair comparison: each indicator now warns at its OWN 80th percentile instead of
a shared absolute 60, removing the artifact that muted the coincident baseline.
- Per-event breakdown (date · depth · breadth lead · coincident lead) so a median
over a tiny sample can't hide an apples-to-oranges comparison — you see whether
both warned on the same drawdown.
- Surface precision/recall (best row) + base rate per indicator — the honest edge
read, not just lead time.
Re-run the Event Study job to regenerate the cached report in the new shape.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a leading-by-construction candidate and the harness to measure whether it
actually leads regime breaks, before any of it earns weight in the live index.
- breadth_service: % of the stored universe above its own 200-DMA + a divergence
score (benchmark price up while breadth falls, nudged by low breadth). Genuinely
leading because it keys on divergence, not level. Not wired into the live score.
- event_study_service: detect drawdown events on the benchmark, then measure each
indicator's median lead time (event-centered) and precision/recall vs. the base
rate (signal-centered). Compares breadth-divergence against the deterministic
coincident price composite (reuses the regime price sub-scores). Price/breadth
only — reproducible, no LLM/FRED.
- Manual "Event Study" job (Admin → Jobs), GET /regime/event-study, and an
inline early-warning panel on the Regime tab with an honest small-sample caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new /regime tab scoring how far the AI/Tech bull regime has deteriorated
toward a re-rating as a single 0-100 index with per-signal breakdown and a
7/30-day trend. Intentionally decoupled: nothing reads its output to gate or
score trades — the daily-pipeline membership is scheduling only.
- regime_monitor_service: price sub-scores (P1-P6 via Alpaca, like
market_regime), VIX + HY credit spreads via a small FRED helper, weighted
aggregation over available signals (missing source -> n/a, dropped from the
denominator), one snapshot row/day, and a ~90-day history backfill by
replaying the already-fetched series as-of each past day.
- F1/F3 fundamentals proposed by the configured grounded LLM (reuses
sentiment_provider_service config resolution), with a manual override + lock.
- regime_snapshots table (migration 011); endpoints on the existing market
router; admin-editable weights/threshold; standalone /regime page.
Data needs: prices via Alpaca, VIX/credit via FRED (optional key — signals show
n/a without it). No LLM needed for history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Qualified setups could carry no sentiment because the sentiment job scoped
its relevant-set to watchlist + open trades + top-N composite score, while
the activation gate qualifies on 12-1 momentum percentile — a different axis.
A top-momentum ticker outside the composite top-N never got sentiment, so the
R:R scan enhanced it as neutral.
Add the gate's momentum leaders (percentile >= activation min_momentum_percentile)
to the sentiment relevant-set so scope tracks the gate. Best-effort: a momentum
or config failure falls back to the base set rather than aborting collection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Behavior-preserving cleanup (345 tests pass, ruff clean):
- scheduler: replace 62 inline logger.x(json.dumps({...})) calls with a
_log_event helper, and collapse 11 identical _job_runtime dicts into an
_idle_runtime() factory over _JOB_NAMES.
- settings: add app/services/settings_store.py (get_setting/get_value/get_map/
upsert_setting) and route ~13 hand-rolled SystemSetting queries + two
identical _settings_map helpers through it.
- scoring.get_rankings: collapse the per-ticker N+1 (3-4 queries + a commit each)
into 2 bulk reads + a single conditional commit; drop the redundant re-fetch.
Lazy recompute-on-read is preserved. Adds first tests for get_rankings.
Net ~ -245 lines across the touched modules.
min_target_probability is gone: it filtered on the probability model the
calibration has repeatedly shown to be weak and overconfident, it was redundant
with the momentum gate, and as an off-by-default knob it just invited bad tuning.
Removed from the backend gate, activation config/schema, the frontend mirror
(qualifiesSetup / activationSummary), and ActivationSettings. The probability
model stays where it does real work (primary-target selection + display).
Charts: with multi-year history the all-bars default was unreadable. Added
time-range presets (1M / 3M / 6M / YTD / 1Y / 3Y / 5Y / All), defaulting to 1Y;
clicking a preset always re-applies (snaps back after a manual zoom). Y-axis
autoscale and wheel-zoom / drag-pan were already there.
339 backend tests pass; frontend build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Part 1 — long-only. The momentum edge is long top-momentum; the gate was
qualifying shorts on high-momentum names (fighting the trend), which showed as
the -0.13R Short(qual.) drag. While the gate is active, shorts no longer qualify
(backend qualification, backtest _momentum_qualifies, and the frontend mirror).
Part 2 — production wiring. Live setups now carry a real momentum rank, so the
dashboard, the Track Record's qualified stats, and outcome evaluation all gate on
the same value instead of deferring to floors:
- new momentum_service.compute_momentum_percentiles: 12-1 momentum per ticker,
ranked across the universe into a {symbol: percentile} map.
- the daily R:R scan ranks the universe up front and stores each setup's
percentile (new trade_setups.momentum_percentile column, migration 010).
- enhance_trade_setup mutates the same row, so the percentile is preserved;
_trade_setup_to_dict + TradeSetupResponse expose it to the API.
Until a fresh scan runs, pre-existing setups have a null percentile and the gate
falls back to floors for them (longs) / excludes them (shorts) — they fill in on
the next scan. 341 backend tests pass; frontend build clean.
Needs the alembic upgrade (migration 010) on deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The replay was CPU-bound and single-core: the earlier asyncio.to_thread offload
kept the API responsive but, because of the GIL, ran on one core. Per-ticker
replay is independent, so fan it out across worker processes (which sidestep the
GIL) for real multi-core speedup.
- New `settings.backtest_workers` (default 4), capped to cpu_count-1 so a core
stays free for the web server.
- Uses a `forkserver` context (workers forked from a clean single-threaded
server — avoids the fork-with-threads deadlock); falls back to `fork`. On
spawn-only platforms (Windows) and for 1-ticker runs it uses the thread path,
so dev/tests are unaffected.
- Worker takes primitive column arrays (cheap to pickle), rebuilds bars, and
returns (candidates, plain-dict signal series) — both picklable across the
process boundary. Bars are still fetched in the event loop (ORM-safe).
- Pool creation is guarded: if the pool can't start, the job falls back to the
sequential thread path instead of failing.
334 backend tests pass (parallel path is POSIX/server-only, so it's covered by
construction + the picklability/worker-count tests; the thread fallback is
exercised by the run_backtest smoke test).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 5-year backtest confirmed the EV gate adds negative value (high threshold =
worst expectancy) and that 12-1 month momentum is the one price signal with a
plausible, right-signed cross-sectional IC (~0.05). So "qualified" now means:
clears the R:R + confidence floors AND the ticker ranks in the top
`min_momentum_percentile` of the universe by 12-1 momentum that week.
- qualification.py: drop expected_value_r / the EV gate; add a momentum-percentile
gate (duck-typed `momentum_percentile`, only enforced when attached + threshold
set, else defers to floors). Mirrored in frontend qualification.ts.
- activation config/schema: min_expected_value -> min_momentum_percentile
(default 80 = top quintile). ActivationSettings, DashboardPage (ranks/【shows】
momentum instead of EV), and the BacktestPanel sweep follow.
- backtest: rank each ISO week's universe by 12-1 momentum, assign a percentile,
and qualify the top slice; the sweep now sweeps the percentile cutoff.
Also offload the backtest's per-ticker compute to a worker thread so the heavy
~5y run no longer blocks the API event loop (the "backend offline" flicker).
Production setups don't carry momentum_percentile yet — wiring the scanner to
attach it (a universe momentum-rank step) is the next step; until then the live
gate defers to floors while the backtest measures the momentum selection. 330
backend tests pass; frontend build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two changes so the cross-sectional signal results can actually be trusted.
(a) History depth — the binding constraint. Ingestion defaulted to 365 days, so
long-lookback factors (12-month momentum, 52-week high) were only computable on a
handful of weeks at the tail, and every IC reflected a single market regime.
- New `settings.ohlcv_history_days` (default 1825 ≈ 5y); new tickers backfill this
far instead of 1 year.
- New manual "data_backfill" job (Admin → Jobs) re-fetches the full window for
every ticker, ignoring incremental resume — run once to deepen existing
1-year histories. Idempotent (upsert); resumes after rate limits.
(b) Factor-IC honesty. The IC was averaged over weekly rebalances whose 30-day
forward windows overlap, inflating the t-stat ~sqrt(6)x.
- IC now measured on NON-OVERLAPPING windows (weeks thinned to ~HORIZON apart).
- Each signal carries a `reliable` flag (>= 12 independent windows); BacktestPanel
greys out and de-stars thin signals so a lucky 9-week IC of 0.3 can't masquerade
as an edge.
332 backend tests pass; frontend build clean. No migration (config + job + an
added JSON field on the cached backtest report).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-setup hit-rate report can't tell whether a signal predicts returns —
only how a target/stop structure built on one performs. This adds a
cross-sectional factor-IC pass: each week the universe is ranked by a price-only
signal and graded by its rank correlation (Spearman IC) and top-minus-bottom-
quintile spread against the forward 30-day return.
Candidate signals (point-in-time from price; sentiment/fundamentals have no
history in the replay): 12-1/6-1/3-1 month momentum, 1-month reversal,
price-vs-200d SMA, proximity to the 52-week high (George/Hwang), and 126-day
realized volatility (low-vol anomaly).
Reuses the existing per-ticker replay loop (no new data, no second DB pass);
results land in the cached backtest_report as `signal_eval` and render as a
"Signal edge" table in BacktestPanel beside the calibration curve.
330 backend tests pass (10 new in test_signal_eval); frontend build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnosing "no qualified signals for 5 days": setups were generated but none
qualified. The gate required BOTH a high min_rr (2.0) AND a high
min_target_probability (60), which became contradictory after the Jun-15
probability recalibration — probability already embeds R:R via the 1/(rr+1) ruin
term, so high-R:R targets are inherently low-probability and nothing cleared both.
Gate is now expected value (R): p*rr - (1-p) from the primary target's
probability. R:R and confidence stay as floors; high-conviction / exclude-conflicts
/ min-target-probability become optional tighteners (default off). Defaults:
min_expected_value=0.15, min_rr=1.2, min_confidence=55. EV is only enforced when
computable. Migration 009 clears stored activation_* rows so the new defaults
apply. Backtest sweeps min_expected_value instead of target probability.
Scheduling: pipelines are now cron-configurable in Admin -> Jobs. daily_pipeline
(full, default 0 7 * * *) plus a new light intraday_pipeline (OHLCV + outcome eval,
default hourly US session) that keeps prices/live-R:R current without setup churn.
Fundamentals on its own early weekly cron. Timezone configurable (default
Europe/Berlin). Moving interval->CronTrigger also fixes the restart-deferral bug
where an interval job's countdown resets on every process restart.
319 backend unit tests pass; frontend tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulled the fundamental collector out of the daily pipeline (where it re-fetched
near-identical numbers every day and burned free-tier API quota) and made it an
independent weekly job. P/E/market-cap drift with price but the score buckets
them coarsely; revenue growth and earnings surprise only change at quarterly
earnings. Added "weekly" to the frequency map; fundamental_fetch_frequency now
defaults to weekly (configurable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jobs were independent 24h timers with no ordering, so the scanner could run on
stale OHLCV, and manual runs desynced the offsets. New daily_pipeline job runs
the data→signal flow in dependency order: OHLCV → fundamentals → sentiment →
R:R scan → outcome eval (+paper close) → market regime. Each step keeps its own
enable flag and runtime status; a failing step is logged and the pipeline
continues.
The member jobs are registered PAUSED (no auto-fire) so they only run via the
pipeline — but stay manually triggerable from Admin → Jobs (shown as "runs in
daily pipeline"). Alerts (hourly), ticker universe sync, and backtest keep their
own independent cadence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resolve_open_trades walks the daily bars after each open trade and closes it at
the target (target hit) or stop (stop/ambiguous), leaving undecided trades open.
Runs nightly inside the outcome evaluator (so it's coordinated with fresh OHLCV)
and on its manual trigger. New "My Trades" section at the top of Signals → Track
Record shows realized hit-rate, expectancy (avg R), total R, total P&L, and a
closed-trades table — your actual results, separate from the theoretical signal
record below it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Richer LLM output (same grounded call, ~no extra cost):
- All providers now also return a recommendation (buy/hold/avoid) and a thorough
reasoning paragraph; Gemini now actually captures reasoning + grounding
citations (it was dropping them). Stored on sentiment_scores (migration 008),
exposed in the API; display-only — NOT fed into the composite/EV.
- Ticker Sentiment panel shows an "LLM view" badge and a "Full analysis & sources"
expander with the complete reasoning + citations.
Search-budget scoping (Gemini grounding free tier = 5000/mo):
- collect_sentiment now targets only watchlist + open paper trades + top-N by
composite, skips tickers refreshed within sentiment_fresh_hours (72h), and caps
per run (sentiment_max_per_run). Once the relevant set is fresh, runs spend 0
searches until it ages out — bounding monthly usage well under the free tier.
- Widened sentiment lookback to 7d (scoring + display) so sparser collection
still feeds the dimension score.
Deploy: alembic upgrade (sentiment_scores.recommendation). Switch provider to
Gemini Flash in Admin for the cost win (grounded, cheapest).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New paper_trades table (migration 007) + service/router. "Mark as taken" on each
setup card (shares prefilled from position sizing, entry from current price, both
editable) records a simulated trade. Overview gains an Open Trades table that
marks each position to the latest close — P&L in $, %, and R-multiples — with a
total unrealized P&L footer and a Sell button to close at the current price.
Closed trades are retained for future realized-P&L reporting.
Deploy: alembic upgrade (new paper_trades table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-applies the activation gate at several min_target_probability thresholds
(60→30, other conditions fixed) over the already-replayed candidates, so the
trade-off between how many setups qualify and their expectancy is visible in one
table — the cheap "optimize" half of Phase 2. Candidates now carry meets_core +
best_prob so the sweep needs no re-replay. New sweep table in BacktestPanel with
the current threshold starred.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backtest (32k setups) showed the touch-only probability model was ~2x
over-confident — predicted 70% hit 39%, predicted 88% hit 46% — because it
ignored the competing stop. estimate_probability now multiplies the reach
probability (touch within horizon) by the two-barrier gambler's-ruin ratio
1/(R:R+1) = P(target before stop). A 3:1 setup now reads ~25% base, not ~70%,
which lines up with realized rates. Strength/alignment modulation unchanged.
Recalibrates every probability and the EV ranking; the min_target_probability
gate threshold now means roughly what it says. Re-run the backtest to confirm
the calibration table flattens toward the diagonal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replays the price-derived engine over stored OHLCV: at each weekly as-of date,
rebuild the setup from bars <= D (no lookahead) and walk the actual forward bars
for the realized outcome. Reports realized hit-rate/expectancy of qualified
setups (and all setups, by direction) plus a probability calibration curve
(predicted target prob vs realized hit rate).
Reuses pure functions throughout; extracted compute_technical_from_arrays /
compute_momentum_from_closes from scoring_service so live and backtest stay in
sync. Runs as a weekly/triggerable 'backtest' job caching the report in a
SystemSetting; GET /backtest/report serves it. Sentiment/fundamentals held
neutral (no point-in-time history) — calibrates the price/S-R/probability machinery.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Finnhub's earnings calendar now supplies next_earnings_date through the
fundamentals chain; persisted on fundamental_data (migration 006) and exposed in
the fundamentals API. The recommendation panel warns when earnings fall within
the ~30-day target horizon (a report can gap price through stop/target) and
otherwise shows the next date. Informational only.
Deploy: run alembic upgrade (new fundamental_data.next_earnings_date column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New market_regime_service computes a benchmark (SPY) trend from its 50/200-day
SMAs, cached in a SystemSetting and refreshed by a nightly job; GET /market/regime
exposes it. Dashboard shows a regime banner; setup cards flag a counter-trend
caution when a setup fights the regime (LONG in a bearish market / SHORT in a
bullish one). Informational only — nothing is suppressed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Qualified tickers already get their own "qualified setup" alert, so an S/R
proximity ping on them is redundant noise. Drop the watchlist ∪ qualified scope
(remove now-unused _alert_scope_tickers) and alert only on watchlist tickers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Trade-setup targets now pre-merge near-duplicate S/R levels into zone
representatives (same 2% clusterer as chart + alerts) before generate_targets
runs. A clustered wall (e.g. 183 + 185) becomes one target carrying the zone's
COMBINED strength (capped 100) instead of two near-identical targets that each
undervalue the wall — which also feeds a more honest reach-probability via the
S/R-strength magnet. Representative price is the zone's near edge; the strongest
constituent's id is retained. Singleton levels pass through unchanged, so the
downstream band-spreading / probability / primary-selection pipeline and its
tests are untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes to over-firing S/R proximity alerts:
- Route through cluster_sr_zones (the same merger the chart uses) instead of raw
SRLevel rows, so near-duplicate levels (e.g. CVX 183 + 185) collapse into one
zone and one alert.
- Alert only the single NEAREST strong zone per ticker, not every nearby level.
- Scope to watchlist + qualified-setup tickers via _alert_scope_tickers (was
iterating all watchlist entries only; qualified setups are now included too).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "price plan needed in bulk but fine on manual reload": on free
tiers FMP returns only market cap (others 402) and the chain merged that as a
partial success — so when the Finnhub/Alpha Vantage fallbacks were rate-limited
during a bulk run, the chain silently returned market-cap-only and the
collector's backoff never engaged. Manual single fetches worked because the
fallbacks weren't throttled at that moment.
Fixes:
- Chain distinguishes RateLimitError from other failures: if a fallback is
rate-limited and fields are still missing, raise RateLimitError (unless
allow_partial=True) so the collector backs off and retries.
- Bulk job paces requests (fundamental_request_spacing_seconds, default 3s) to
stay under Finnhub's ~60/min, and on retry-exhaustion stores partial data and
continues instead of aborting the whole run.
- Manual fetch passes allow_partial=True so a lone 429 doesn't fail the refresh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the action loop — instead of polling the dashboard, the platform pushes
actionable signals to Telegram. New hourly 'alerts' job dispatches four
toggleable triggers, deduped via a new alert_log table (cooldown-based for
qualified/S-R/digest, watermark-based for score deterioration). Admin → Settings
gains a Telegram panel (write-only bot token, chat ID, per-trigger toggles, Send
Test). Credentials follow DB > env precedence (TELEGRAM_BOT_TOKEN / _CHAT_ID).
Backend: alert_service + AlertLog model + migration 005, scheduler job, admin
endpoints/schema. Frontend: AlertSettings panel, hooks, api, types.
Deploy: run alembic upgrade (new alert_log table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AsyncIOScheduler was constructed with no job_defaults, so APScheduler's default
misfire_grace_time of 1s applied. In this single-process app the scheduler shares
one event loop with the API and all other jobs, so when a daily job came due
while the loop was busy (e.g. the scanner mid-run), the fire was processed >1s
late, flagged a misfire, and skipped — while next_run still advanced 24h, making
the job look healthy though it never ran. Set a generous grace window (1h),
coalesce missed runs into a single catch-up, and cap concurrency at 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scan_rr set the total then called scan_all_tickers as one opaque await, so the
runtime snapshot's processed count stayed 0 until the whole scan finished and
jumped straight to 100%. scan_all_tickers now takes an optional progress_callback
invoked per ticker; the scheduler wires it to _runtime_progress so the sidebar's
live indicator advances as tickers are scanned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track Record: new "Reset" action (POST /admin/track-record/reset) deletes all
trade setups so stats start fresh after material scoring/setup changes — live
setups regenerate on the next scan. Guarded by a confirm dialog.
Recommendation config: remove distance_penalty_factor, which was exposed in the
admin UI but consumed nowhere (the touch-probability model superseded it). A
knob that silently does nothing is worse than no knob. Remaining defaults are
left as-is — they're reasonable, and the honest way to tune them is backtesting
against accumulated outcomes, not invented "researched" numbers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dennisthiessen