The A5 parity report surfaced coverage gaps and wrong values that all traced
to the SEC facts parser and read-time derivation rather than to bad source
data. Fixes, each validated by replaying the production parser + derivation
against live company facts:
- Period identity is derived from period_end against the issuer's fiscal
calendar, not SEC's fy/fp fields, which collide (two period ends on one key,
one silently discarded) and invert (a period sorting before one that precedes
it) often enough to break the quarter chain. Recovers BXP, CRM, CRWD, FRT,
MTD, NTAP, PPL, STX, WDAY. Fixed labels are internal ordering keys only (not
in any API schema), so a filer whose year ends in early January shifting by
one is harmless.
- Revenue concept list gains RevenuesNetOfInterestExpense (banks) and the
IncludingAssessedTax variant (REITs/consumer); EPS gains the continuing-ops
variant (REG/FCX) and, last, basic EPS for a period tagging no diluted
variant at all (PPL). All appended, so any issuer that already resolved keeps
its concept.
- YTD span tolerance 20 -> 25 days, covering 4-4-5 retail calendars whose
36-week YTD-Q3 (251-252d) previously missed by ~2 (COST, PEP, DPZ).
- Amendment resolution is per field: a partial 10-K/A (Part III only, no
financial facts) no longer blanks the period (DVN).
- TTM diluted EPS is suppressed when a split contaminates the trailing window
(BKNG's mixed-unit sum produced a P/E of 1.10 that clamped to a perfect
fundamental sub-score). A post-filing split with no share-count evidence
(KLAC) remains undetectable from this data.
- Multi-class share fallback: weighted_avg_diluted_shares is captured and used
for market cap when the cover-page count is absent (dimensional, so missing
from company facts for META/CMCSA/CHTR/FOXA/NWSA/LEN). Within ~0.6% of the
true count on controls; flagged shares_estimated in the API. BRK-B has no
weighted-average fact either and stays unavailable.
820 unit tests pass; new tests confirmed to fail against the pre-fix code.
Effect is inert until existing rows are reparsed (see reparse path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. Removed the 45-day index-walk cap: it discarded the older part of a long
outage while still advancing source_max_date, permanently losing filings.
The walk now covers every unprocessed date (a large gap is one-time cost).
2. Discrepancy detection meets the immutability contract: it compares ALL source
snapshot fields (not five), read-only during stage/validate, reports the
differing accessions + fields in validation_json, and promote emits a warning
system event (in-transaction) — never mutating the stored row.
3. Malformed companyfacts (missing facts/units structure) are recorded separately
and FAIL validation, instead of silently degrading to skipped rows that the
50% backfill coverage floor could still pass.
Also corrected the stale "sum share classes" / DEI-only wording in the snapshot
model docstring and the A3 design doc to describe the us-gaap fallback.
Tests: +4 regressions (>45-day gap loses nothing, newly-added issuer backfills
without filing, malformed payload fails, shares discrepancy detected + evented).
23 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A1 carry-forward. The SEC cover-page share count
(dei:EntityCommonStockSharesOutstanding) is reported "as of" its own date, which
can differ from the fiscal period_end — store that date so market cap uses the
right point-in-time count. Migration 026 edited in place (never run with data).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First reviewable slice of workstream A: schema only, no importers, no data.
- data_import_runs: lean batch-import audit (source/revision/status,
row_counts_json + validation_json as Text-holding-JSON per repo convention).
- fundamental_snapshots: CIK-keyed, one immutable row per accession; stores
per-period raw facts (duration = cumulative YTD/FY, balance-sheet =
period-end) plus period_start/period_end/fiscal_year/fiscal_period so
discrete quarters, Q4, TTM and YoY are derived at read time.
- earnings_events: Dolt-sourced calendar + surprise history, unique
(ticker_id, announce_date).
- tickers: nullable cik/sic/sic_description — the ticker<->issuer join point.
fundamental_data is left untouched (cutover gated separately at A5). Models
registered in app/models/__init__.py; Ticker gains an earnings_events
relationship. Verified: create_all builds the tables, mappers configure, and
migration 026 renders valid Postgres DDL up and down.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The run-id marker proved which scan wrote last, but the shadow book still
selected setups by detected_at >= scan_start. An overlapping manual scan
could insert rows in that same window; if the pipeline's scan wrote the
marker last its id matched and the shadow book proceeded, then swept in --
or ranked highest -- a manual-scan row. The identity check gated entry but
selection did not.
Carry the run id onto the rows. Migration 025 adds an indexed
trade_setups.scan_run_id. scan_all_tickers computes one id per run
(pipeline's when a step, else fresh), passes it to scan_ticker which stamps
every row after enhancement, and writes the same id to the completion
marker. The shadow book selects WHERE scan_run_id == the matched id, so a
concurrent scan's rows are excluded by identity regardless of their
detected_at. The now-unused STARTED marker is dropped; COMPLETED
(freshness) and RUN_ID (identity) remain.
Decisive test: the pipeline's id matches, but a same-window manual row with
a higher rank is present and is excluded -- only the pipeline's own row is
traded. A time-window select would have swept it in and ranked it first.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The manual paper book only contains trades taken by hand, inside a 20
minute window, on days someone was available. The backtest that validated
this strategy auto-takes the top-ranked qualified setups up to capacity
every session. The forward record was therefore measuring strategy plus
discretion plus availability -- and degrading silently on busy days.
The shadow book closes that gap: it mirrors the backtest's selection rule
(top strategy_rank qualified, up to capacity, 1% fixed-fractional risk)
and shares the manual book's exit policy, so the only difference between
the two books is which setups get taken. Selection ordering reuses the
strategy_rank the scanner already stores rather than recomputing it, so
the two cannot drift apart. It runs as a near-close pipeline step right
after the scan, marking entries at the same prices a human would see.
Gate-reset re-entry state is now scoped per book -- the books diverge as
soon as their entries differ, and each must see only its own stops.
Performance view rewritten around the comparison:
- three series (shadow, manual, SPY) from a new endpoint
- SPY changes from a per-trade cost-basis counterfactual to plain
buy-and-hold %, since one line has to serve two books
- headline stats are R-multiples, not currency: the books size
differently, so only R compares across them
- configurable start date, because the strategy has been revised
repeatedly and pre-cutover trades ran under rules that no longer
exist
Migration 024 also repairs the numeric weekday crons written by 023,
rewriting only rows still holding the broken form so hand-corrected
settings survive. Its literals are inlined because bound parameters
render as NULL under 'alembic upgrade --sql'.
The shadow book is opt-in and writes nothing until enabled. Verify its
first selections match a backtest of that day's cross-section before
trusting any point on the curve.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts,
put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day
requalify semantics, stamp paper trades fill_mode=near_close, and migrate
stored schedule_* keys to America/New_York.
Persist job and ingestion warnings/errors for 7 days, surface a dismissible top-nav badge, treat stale OHLCV as a warning (e.g. ticker renames), and show market bar age on the ticker freshness chip.
Backtest report now includes research-only hold-to-horizon portfolio variants comparing raw vs residual 12-1 momentum, cutoff 80 vs 90, max 10 vs 15 positions, and SPY-200 risk scaling. A dynamic research recommendation panel flags residual momentum, cutoff 90, or regime scaling only when transparent promotion rules pass.
Adds signal_context_snapshots with migration 016 and captures one point-in-time context row per newly generated TradeSetup: setup fields, composite/dimensions, latest sentiment, latest fundamentals, and strategy_version=momentum_12_1_rr_time_v1. This is forward-only; no historical sentiment/fundamental backfill is attempted.
No live gate, paper-trade exit, or production ranking behavior changes.
Verification: 458 backend tests pass, ruff check app/ clean, frontend npm run build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Store an optional company name on Ticker (migration 014) and backfill it from
Alpaca's asset list in a single Trading-API call for the whole universe — no
per-ticker fetch. Runs automatically at the end of universe bootstrap and via a
manual "Backfill Names" button (admin) / POST /admin/tickers/backfill-names.
The name ships on /tickers; a shared symbol→name map (useTickerNames) lets any view
show it without its own request. Displayed subtly next to the symbol — in the global
search, the ticker header, and as a small muted line under the symbol in Top Setups
and Open Trades (no extra column, truncated so it never widens the table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Applies the backtest-validated trailing stop to live paper trading, and surfaces
it transparently.
Exit (A):
- New paper-trade exit policy (paper_exit_mode=trailing, paper_trailing_pct=12),
tunable in Admin → Paper-Trade Exit. resolve_open_trades runs a trailing stop
(initial stop as floor, ratchets up from the peak; target ignored — the
validated rule) and records close_reason (trailing|stop|target|manual; +migration
013).
- list_trades enriches open trades with the live trailing-stop level + distance %.
Open Trades panel shows the active tactic and a Trail Stop column.
Alerts (B):
- Daily digest now lists open trades with unrealized gain, trailing stop, and how
far away it is.
- New "trade closed" alert: one summary per auto-close (trailing/target/stop, not
manual) — direction, reason, days held, P&L abs+%/R — covering wins AND
stop-loss losses. Deduped by trade id; toggle in Admin alerts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three usability fixes:
1. Global ticker search in the sidebar (TickerSearch) — typeahead over the
tracked universe that opens a ticker's detail page without adding it to the
watchlist. Also wired into the mobile nav.
2. Watchlist table shows the ticker's 12-1 momentum percentile (the top-pick
selector) instead of the noisy full S/R-level list. Enriched from the setup
already loaded in watchlist_service._enrich_entry — no extra query.
3. Alpha vs the S&P 500 on paper trades (open + closed). New benchmark_prices
table + benchmark_service store SPY daily closes (a standalone series, not a
Ticker, so it never enters the scanner / momentum ranking / rankings) via a
new daily-pipeline step. paper_trade_service computes per-trade
benchmark_return / alpha_pct / alpha_usd over each holding period; the open-
trades table, dashboard, and closed-trades panel surface per-trade and total
alpha. The list read path never makes a provider call.
Deploy: alembic upgrade head, then run the benchmark/daily job once to populate
SPY closes (alpha shows "—" until then).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A new /regime tab scoring how far the AI/Tech bull regime has deteriorated
toward a re-rating as a single 0-100 index with per-signal breakdown and a
7/30-day trend. Intentionally decoupled: nothing reads its output to gate or
score trades — the daily-pipeline membership is scheduling only.
- regime_monitor_service: price sub-scores (P1-P6 via Alpaca, like
market_regime), VIX + HY credit spreads via a small FRED helper, weighted
aggregation over available signals (missing source -> n/a, dropped from the
denominator), one snapshot row/day, and a ~90-day history backfill by
replaying the already-fetched series as-of each past day.
- F1/F3 fundamentals proposed by the configured grounded LLM (reuses
sentiment_provider_service config resolution), with a manual override + lock.
- regime_snapshots table (migration 011); endpoints on the existing market
router; admin-editable weights/threshold; standalone /regime page.
Data needs: prices via Alpaca, VIX/credit via FRED (optional key — signals show
n/a without it). No LLM needed for history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Part 1 — long-only. The momentum edge is long top-momentum; the gate was
qualifying shorts on high-momentum names (fighting the trend), which showed as
the -0.13R Short(qual.) drag. While the gate is active, shorts no longer qualify
(backend qualification, backtest _momentum_qualifies, and the frontend mirror).
Part 2 — production wiring. Live setups now carry a real momentum rank, so the
dashboard, the Track Record's qualified stats, and outcome evaluation all gate on
the same value instead of deferring to floors:
- new momentum_service.compute_momentum_percentiles: 12-1 momentum per ticker,
ranked across the universe into a {symbol: percentile} map.
- the daily R:R scan ranks the universe up front and stores each setup's
percentile (new trade_setups.momentum_percentile column, migration 010).
- enhance_trade_setup mutates the same row, so the percentile is preserved;
_trade_setup_to_dict + TradeSetupResponse expose it to the API.
Until a fresh scan runs, pre-existing setups have a null percentile and the gate
falls back to floors for them (longs) / excludes them (shorts) — they fill in on
the next scan. 341 backend tests pass; frontend build clean.
Needs the alembic upgrade (migration 010) on deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Richer LLM output (same grounded call, ~no extra cost):
- All providers now also return a recommendation (buy/hold/avoid) and a thorough
reasoning paragraph; Gemini now actually captures reasoning + grounding
citations (it was dropping them). Stored on sentiment_scores (migration 008),
exposed in the API; display-only — NOT fed into the composite/EV.
- Ticker Sentiment panel shows an "LLM view" badge and a "Full analysis & sources"
expander with the complete reasoning + citations.
Search-budget scoping (Gemini grounding free tier = 5000/mo):
- collect_sentiment now targets only watchlist + open paper trades + top-N by
composite, skips tickers refreshed within sentiment_fresh_hours (72h), and caps
per run (sentiment_max_per_run). Once the relevant set is fresh, runs spend 0
searches until it ages out — bounding monthly usage well under the free tier.
- Widened sentiment lookback to 7d (scoring + display) so sparser collection
still feeds the dimension score.
Deploy: alembic upgrade (sentiment_scores.recommendation). Switch provider to
Gemini Flash in Admin for the cost win (grounded, cheapest).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New paper_trades table (migration 007) + service/router. "Mark as taken" on each
setup card (shares prefilled from position sizing, entry from current price, both
editable) records a simulated trade. Overview gains an Open Trades table that
marks each position to the latest close — P&L in $, %, and R-multiples — with a
total unrealized P&L footer and a Sell button to close at the current price.
Closed trades are retained for future realized-P&L reporting.
Deploy: alembic upgrade (new paper_trades table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Finnhub's earnings calendar now supplies next_earnings_date through the
fundamentals chain; persisted on fundamental_data (migration 006) and exposed in
the fundamentals API. The recommendation panel warns when earnings fall within
the ~30-day target horizon (a report can gap price through stop/target) and
otherwise shows the next date. Informational only.
Deploy: run alembic upgrade (new fundamental_data.next_earnings_date column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the action loop — instead of polling the dashboard, the platform pushes
actionable signals to Telegram. New hourly 'alerts' job dispatches four
toggleable triggers, deduped via a new alert_log table (cooldown-based for
qualified/S-R/digest, watermark-based for score deterioration). Admin → Settings
gains a Telegram panel (write-only bot token, chat ID, per-trigger toggles, Send
Test). Credentials follow DB > env precedence (TELEGRAM_BOT_TOKEN / _CHAT_ID).
Backend: alert_service + AlertLog model + migration 005, scheduler job, admin
endpoints/schema. Frontend: AlertSettings panel, hooks, api, types.
Deploy: run alembic upgrade (new alert_log table).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the feedback loop on R:R scanner signals:
- Nightly outcome_evaluator job replays unresolved setups against daily
OHLCV bars: target_hit / stop_hit / ambiguous (same-bar, counted as
loss) / expired after OUTCOME_EVALUATION_MAX_BARS (default 30)
- Migration 004: evaluated_at + outcome_date on trade_setups
- GET /trades/performance: hit rate, expectancy (avg R), total R with
breakdowns by direction, recommended action, and confidence bucket
- New Performance page (stat cards, breakdown tables, Evaluate Now,
methodology disclosure) wired into sidebar and mobile nav
- 17 new unit tests for evaluation logic and stats aggregation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
