The run-id marker proved which scan wrote last, but the shadow book still
selected setups by detected_at >= scan_start. An overlapping manual scan
could insert rows in that same window; if the pipeline's scan wrote the
marker last its id matched and the shadow book proceeded, then swept in --
or ranked highest -- a manual-scan row. The identity check gated entry but
selection did not.
Carry the run id onto the rows. Migration 025 adds an indexed
trade_setups.scan_run_id. scan_all_tickers computes one id per run
(pipeline's when a step, else fresh), passes it to scan_ticker which stamps
every row after enhancement, and writes the same id to the completion
marker. The shadow book selects WHERE scan_run_id == the matched id, so a
concurrent scan's rows are excluded by identity regardless of their
detected_at. The now-unused STARTED marker is dropped; COMPLETED
(freshness) and RUN_ID (identity) remain.
Decisive test: the pipeline's id matches, but a same-window manual row with
a higher rank is present and is excluded -- only the pipeline's own row is
traded. A time-window select would have swept it in and ranked it first.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A manually triggered rr_scanner and the scheduled near-close pipeline are
separate APScheduler jobs; max_instances=1 serialises a job only against
itself, so they can overlap. A manual scan starting just before the
pipeline can finish just after it began and overwrite the scan markers.
Its completion timestamp is then later than the pipeline start, so the
previous 'completed >= pipeline_start' check accepted its batch as though
it were the pipeline's own -- exactly when the pipeline's scan may have
failed.
Replace the timestamp comparison with an exact run-id match. A new
pipeline_run module holds a per-task run-id contextvar (separate module so
the scanner and scheduler import it without a cycle). _run_pipeline binds a
fresh id per invocation; scan_all_tickers stamps that id -- or a fresh one
when run standalone -- into the scan markers, written with started/completed
in a single commit. The shadow step requires the stored run id to equal its
pipeline's id exactly, so a concurrent manual scan (its own id) or a failed
pipeline scan (a prior run's id) can never be mistaken for it. Direct Admin
triggers have no pipeline context and keep the freshness fallback.
Known residual: the id match governs whether shadow proceeds; setup
selection remains detected_at >= scan start, so a fully per-run setup
isolation would need a run_id column on trade_setups (not required here).
Tests cover the reported race (manual scan finishing last is refused), a
failed pipeline scan, the id-match accept path, and contextvar propagation
and non-leakage across tasks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second review round on the shadow book; all three findings were real.
- Scan freshness is now proven, not assumed. Pipeline steps run and fail
independently, so a disabled or failed scan step still let the shadow
step run on the newest *stored* setups -- a prior session's picks at
stale prices. scan_all_tickers now records a run boundary
(last_scan_run_started_at / _completed_at) only on successful
completion; the shadow book refuses to trade unless COMPLETED is fresh
and selects only setups with detected_at >= the run start. Deduplication
to the latest row per ticker now happens BEFORE qualification, so a newer
unqualified row suppresses an older qualified one rather than the reverse.
- Shadow selection is hard long-only. setup_qualifies only enforces
long-only when min_momentum_percentile > 0, but 0 is a legal admin
setting, and the cash accounting assumes long positions -- so the
constraint is enforced in shadow selection regardless of gate config.
- The personal setup list excludes only the caller's own open positions.
get_trade_setups gained exclude_open_trade_user_id; the trades route
passes the authenticated user, while the Telegram broadcast stays global
since it has no single owner.
New tests cover stale/absent scan markers, prior-run exclusion, newer
unqualified suppressing older qualified, long-only under a disabled gate,
and both sides of the user-scoped exclusion.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review of the shadow book found seven ways the two books could leak into
each other; all are fixed here. The most serious silently invalidated the
comparison the shadow book exists to make.
- Shadow holdings no longer suppress the manual candidate list. The
open-trade exclusion filtered on any book, so shadow taking the
top-ranked names removed exactly those from the user's list and alerts,
confining the discretionary book to leftovers. Scoped to the manual
book. Closed-trade alerts and paper-book equity were leaking the same
way and are likewise scoped.
- Shadow sizing now matches _simulate_portfolio: min(1% risk, 20% notional
cap, available cash) from marked equity, plus the sub- dust guard.
Previously risk-only from realized equity, so a tight stop produced a
multiples-of-equity leveraged position the strategy would never take.
- Shadow only trades setups from the scan that just ran (<6h old) with one
setup per ticker. A failed or disabled scan step could otherwise open
positions from a prior session at stale prices.
- Gate-reset transitions are observed for both books, so a shadow stop-out
completes fail -> requalify instead of staying locked forever.
- Manual list/close endpoints default to the manual book and reject
hand-closing shadow trades; the performance endpoint is scoped to the
caller so 'your picks' is not every user's book.
- run_shadow_book is registered as a paused job so Admin can trigger it.
Also anchors three pre-existing paper-trade tests (and the new alpaca
window test) on the UTC date. They build fixtures from the local date but
the service stamps opened_at in UTC, so they failed only between 00:00 and
02:00 in a UTC+hh timezone -- latent on ba2df8b, exposed by the clock.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ship greenfield min_rr=2.0 and conf=0, read-only Structural S/R, indicator
cache invalidation, and UI/gate language that treats GTL as screening not exit.
Align strategy_rank missing-vol fallback live vs backtest, single-source
PRIMARY_TARGET_MIN_RR, expand prod parity tests, and drop dead FE clients.
Single-ticker fetch now attaches residual-momentum ranks so setups do not silently fail the activation gate. Exit plan is a timeline, chart labels move left of the price scale, and missing ranks surface explicitly.
A scoring error no longer skips setup detection for the ticker: the
rollback already restores a clean transaction, and qualification
re-gates on live scores at alert time, so a stale score is recoverable
but a skipped scan is not.
Iterating symbol strings instead of Ticker instances fixes a latent
crash the new regression test caught: rollback() expires ORM objects
regardless of expire_on_commit, so touching ticker.symbol in the except
handler triggered sync lazy-loading, which raises on an AsyncSession
and killed the whole scan on the first per-ticker error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three follow-ups to the gate probability floor (8f41143):
- Signals table shows the starred primary target (shared primaryTarget
helper) instead of an independently computed max-probability best,
so Overview, Signals and ticker details agree by construction.
- Targets pinned at the 3% probability clamp floor collapse to the
nearest one (enhance_trade_setup + backtest candidates in parity):
floor-pinned levels are indistinguishable to the model, so farther
ones were duplicate 3% rows inviting lottery headlines.
- get_trade_setups only returns setups re-emitted within
LIVE_SETUP_MAX_AGE_DAYS (3): an older latest row means the daily
scan no longer confirms the setup, and such rows otherwise surface
forever on Overview/Signals/ticker/alerts. History endpoints keep
full history.
Backtest on the Jul-3 snapshot is metric-identical to the gate-floor
run on all qualified stats (1089 qualified, Sharpe 2.02, CAGR +49.6%,
DD -15.8%): the prune only removes noise the gate already rejected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A local backtest (offline prod snapshot, 506 tickers) evaluated blue-sky
projected targets under the PRODUCTION exit (3x ATR trailing + 30d max hold,
paper_trade_service DEFAULT_EXIT_MODE="atr_trailing"). Blue-sky setups are
dilutive: the qualified book scored 328% return / Sharpe 1.84 / DD -21.0%
WITHOUT them vs 300% / 1.58 / -18.7% WITH them. They rank high on momentum by
construction, so they grab slots from S/R setups that catch bigger runs under
a trailing-stop exit (only ~2pp worse drawdown doesn't justify the lost return
and Sharpe).
Reverts the scanner/TargetGenerator measured-move projection, the stricter
projected activation gate, the frontend qualification mirror, the `projected`
type field, and the projected tests -- all backend files are now byte-identical
to the pre-blue-sky commit.
Keeps the played-out "No current setup" UX (RecommendationPanel): when price
has run past the target (played out) or through the stop (invalidated), the
panel shows a plain no-setup state instead of a stale actionable card. This is
frontend-only (reads last close + existing setup fields) and is what actually
fixes the reported stale-below-price bug -- no backend change or rescan needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes stale below-price setups showing as current recommendations. Three
distinct causes share the symptom (get_trade_setups returns the latest stored
setup per direction and never expires it):
- Genuine blue-sky (no overhead S/R): scanner + TargetGenerator now project a
measured-move target (entry +/- 3*ATR, ~2:1 R:R), flagged projected with a
low sr_strength probability haircut. Overhead check keys on level tag OR price
so it never projects through a straddling resistance cluster.
- Projected targets clear a stricter activation bar (long-only, momentum >= 90,
confidence >= min+10), independent of the general momentum gate. Mirrored in
frontend qualification.ts.
- Played-out UX (fixes the reported TTWO case, which is R:R-starved under a
resistance cluster, not blue-sky): when price is at/past target or through the
stop, RecommendationPanel shows a "No current setup" state and softens the
stale ticker-level header/reasoning, instead of a stale actionable card.
No migration: the projected flag rides in existing targets_json. 504 backend
unit tests pass; frontend typechecks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stored TradeSetup rows are point-in-time snapshots from the RR scan, so
the ticker page could show stale confidence/reasoning/composite (e.g.
sentiment=neutral in the setup card while the sentiment panel showed
bullish). Overlay current score/sentiment context onto the API payload
for GET /trades and GET /trades/{symbol}, gate and format Telegram
qualified-setup alerts on the same live values, and apply the
min_confidence/recommended_action filters after the overlay so they
judge what the caller actually sees. Stored setups stay frozen for
outcome analysis and backtests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backtest report now includes research-only hold-to-horizon portfolio variants comparing raw vs residual 12-1 momentum, cutoff 80 vs 90, max 10 vs 15 positions, and SPY-200 risk scaling. A dynamic research recommendation panel flags residual momentum, cutoff 90, or regime scaling only when transparent promotion rules pass.
Adds signal_context_snapshots with migration 016 and captures one point-in-time context row per newly generated TradeSetup: setup fields, composite/dimensions, latest sentiment, latest fundamentals, and strategy_version=momentum_12_1_rr_time_v1. This is forward-only; no historical sentiment/fundamental backfill is attempted.
No live gate, paper-trade exit, or production ranking behavior changes.
Verification: 458 backend tests pass, ruff check app/ clean, frontend npm run build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part 1 — long-only. The momentum edge is long top-momentum; the gate was
qualifying shorts on high-momentum names (fighting the trend), which showed as
the -0.13R Short(qual.) drag. While the gate is active, shorts no longer qualify
(backend qualification, backtest _momentum_qualifies, and the frontend mirror).
Part 2 — production wiring. Live setups now carry a real momentum rank, so the
dashboard, the Track Record's qualified stats, and outcome evaluation all gate on
the same value instead of deferring to floors:
- new momentum_service.compute_momentum_percentiles: 12-1 momentum per ticker,
ranked across the universe into a {symbol: percentile} map.
- the daily R:R scan ranks the universe up front and stores each setup's
percentile (new trade_setups.momentum_percentile column, migration 010).
- enhance_trade_setup mutates the same row, so the percentile is preserved;
_trade_setup_to_dict + TradeSetupResponse expose it to the API.
Until a fresh scan runs, pre-existing setups have a null percentile and the gate
falls back to floors for them (longs) / excludes them (shorts) — they fill in on
the next scan. 341 backend tests pass; frontend build clean.
Needs the alembic upgrade (migration 010) on deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scan_rr set the total then called scan_all_tickers as one opaque await, so the
runtime snapshot's processed count stayed 0 until the whole scan finished and
jumped straight to 100%. scan_all_tickers now takes an optional progress_callback
invoked per ticker; the scheduler wires it to _runtime_progress so the sidebar's
live indicator advances as tickers are scanned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Answers "why does a too-far-progressed setup still show": setups are only
recalculated by the scheduled R:R scan and manual fetch; at creation
entry == current price (0% progress), so over-progression is a
between-scans drift effect and must be judged at read time.
- /trades now attaches current_price (latest close per ticker).
- Qualification drops setups whose R:R recomputed from the current price
falls below min_rr — i.e. price already ran toward target (reward
consumed) or through the stop. Reuses the existing min_rr threshold
instead of a separate progress %; far cleaner (a 3:1 is already ~1:1
by 33% progress). Skipped for historical setups (no current_price).
- Fix: useFetchSymbolData now invalidates the trades queries, so a fetch/
recompute actually refreshes confidence/setups in the UI (was the cause
of the stale 100% confidence lingering after recompute).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scores never updated ("101d ago"): get_score only recomputes stale/
missing dimensions, but nothing marked them stale on new data, and there
was no scheduled scoring job.
- Fetch endpoint force-recomputes dimensions + composite.
- Scheduled scan (scan_all_tickers) refreshes scores per ticker, so
scores stay current globally, not just on manual fetch.
Granular fetch: /ingestion/fetch accepts a sources filter; the freshness
bar gets a per-row refresh button (OHLCV/Sentiment/Fundamentals fetch
that provider only — marked paid; S/R/Scores recompute for free). Header
button is now "Fetch All".
Job visibility: GET /jobs/running (any user) + sidebar live indicator
showing running scheduled jobs with progress, polled every 10s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Triggered by CNC showing "LONG (High Confidence)" with SHORT reasoning
and no long setup.
- A: recommendation action + reasoning are ticker-level and identical
on both setups; reasoning always matches the shown action
- B: recommended_action only picks a direction with a tradeable setup;
strong bias with no setup (e.g. price at ATH) → NEUTRAL with an
explanatory reason instead of a fake LONG_HIGH
- C: confidence is a directional-agreement model — opposing signals push
it below 50 (SHORT on a 92-technical/99-momentum stock ~0%, not 55%)
- D: fundamental score requires >=2 real metrics (market-cap-only no
longer yields a high score)
- E: RSI score peaks at healthy momentum (~60) and penalizes
overbought/oversold extremes instead of treating RSI 90 as maximal
- F: fundamentals chain merges fields across providers (FMP market cap
+ Finnhub P/E) instead of stopping at the first with any field
- NEUTRAL label: "No Clear Setup" (covers untradeable-bias case)
Scores recompute on next scan/scoring run; C and E shift score
distributions intentionally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the feedback loop on R:R scanner signals:
- Nightly outcome_evaluator job replays unresolved setups against daily
OHLCV bars: target_hit / stop_hit / ambiguous (same-bar, counted as
loss) / expired after OUTCOME_EVALUATION_MAX_BARS (default 30)
- Migration 004: evaluated_at + outcome_date on trade_setups
- GET /trades/performance: hit rate, expectancy (avg R), total R with
breakdowns by direction, recommended action, and confidence bucket
- New Performance page (stat cards, breakdown tables, Evaluate Now,
methodology disclosure) wired into sidebar and mobile nav
- 17 new unit tests for evaluation logic and stats aggregation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dennisthiessen
