dennisthiessenandClaude Opus 5 a13dbc9710 fix(sec): stop an unrecoverable filing gap pausing setups forever
A filing gap pauses its issuer until the filing is ingested or a later one
supersedes it, which assumes the gap is temporary. It is not always: SEC's
per-company Company-Facts files can go stale indefinitely — 43 large caps
whose Q2 10-Qs the frames API carries but whose companyfacts files never
received (Abbott's newest fact was 2026-04-29 in late August) — and because
the supersede rule needs a *successfully ingested* later filing, a stale file
swallows the next quarter too. The pause was open-ended, not seasonal.

So the pause hands off to the alert: once filing_gap_aged has escalated a gap,
it stops gating if the issuer's newest stored 10-K/10-Q is under 180 days old.
An issuer with nothing that recent has no usable fundamentals at all and stays
paused, which is the case the gate was built for.

Applied in the gate service only. active_gaps is deliberately untouched so
_retry_backlog keeps retrying and a recovered filing still resolves normally,
and the bound covers both gate paths — the queue and the validation_json
summary that mirrors the same filings — since bounding one leaves production
behaviour unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Lo99z3jWqu9X9ueBU3Z3D
2026-08-21 16:46:28 +02:00
2026-07-12 21:58:10 +02:00
2026-02-20 17:31:01 +01:00

Signal Dashboard

Investing-signal platform for US equities. It runs one strategy, and it is a boring one:

A long-only cross-sectional momentum book. Buy the top quintile by beta-adjusted 12-1 month momentum, tilt toward higher volatility, hold at most 15 names, cut at 1.5× ATR, then trail at 3× ATR for up to 30 trading days. After an initial-stop exit, re-enter only after the gate has failed and subsequently qualified again.

Philosophy: don't predict price — rank it. The edge is relative strength across the universe, and the discipline is in the exit: cut losers fast, let winners run until the trail catches them.

What is NOT the edge — read this before trusting a number on screen. The composite score, the 5 dimensions, sentiment, fundamentals, and Structural S/R are display context, not validated predictors. The Gate Target Ladder is screening machinery that preserves the production setup population; it is not a claim about true market structure. In particular:

  • The headline "target" is not an exit. It comes from the internal Gate Target Ladder and exists only to compute the R:R and reach-probability used by the activation gate. Human-facing chart S/R is a separate model. The live exit reads neither. Across 472 trades in the current daily gate-reset replay, the exit reasons were 229 initial stop, 147 trailing stop, 96 max hold — and 0 targets. Honoring the target as a take-profit was tested and halves CAGR (research).
  • The composite score does not select trades. Residual momentum does.

Full experiment log — everything tested, kept, and rejected: docs/research/.

The strategy, end to end

flowchart TD
    U["Universe — ~500 tickers<br/>daily OHLCV"] --> M["Residual 12-1 momentum<br/><i>12-month return, skip last month,<br/>beta-adjusted vs SPY</i>"]
    M --> R["Rank cross-sectionally<br/>into percentiles"]
    R --> G1{"Top 20%?<br/>percentile ≥ 80"}
    G1 -->|no| SKIP["Not traded<br/><i>(still scored — the control group)</i>"]
    G1 -->|yes| S["Build the setup<br/>entry = last close<br/><b>stop = entry  1.5 × ATR</b><br/>primary = Gate Target Ladder proposal"]

    S --> G2{"Activation gate"}
    G2 --> G2a["R:R ≥ 2.0 <i>(to the primary gate target)</i>"]
    G2 --> G2b["touch odds ≥ 20%"]
    G2 --> G2c["action not NEUTRAL<br/>and matches direction"]
    G2a & G2b & G2c --> Q{"qualified?"}
    Q -->|no| SKIP
    Q -->|yes| RANK["Rank by production score<br/>80% momentum %ile<br/>+ 20% volatility %ile"]

    RANK --> BOOK{"Room in the book?<br/>max 15 positions"}
    BOOK -->|no| WAIT["Wait for a slot"]
    BOOK -->|yes| OPEN["OPEN — size at 1% account risk"]

    OPEN --> EXIT{"Exit — whichever comes first"}
    EXIT --> E1["Initial stop hit<br/>entry  1.5 × ATR → 1R<br/><b>49% of trades</b>"]
    EXIT --> E2["Trailing stop hit<br/>highest close  3 × ATR<br/><i>only binds once price is ~1R up</i><br/><b>31% of trades</b>"]
    EXIT --> E3["Max hold reached<br/>30 trading days<br/><b>20% of trades</b>"]
    EXIT -.->|"NEVER"| E4["Gate Target Ladder target<br/><b>0% of trades</b>"]

    E1 --> LOCK["Re-entry locked"]
    LOCK --> GF{"Later daily scan<br/>fails the gate?"}
    GF -->|no| LOCK
    GF -->|yes| GQ{"A subsequent daily scan<br/>qualifies again?"}
    GQ -->|no| GQ
    GQ -->|yes| RANK

    style M fill:#1e3a5f,color:#fff
    style OPEN fill:#1e4d2b,color:#fff
    style E4 fill:#2a2a2a,color:#888
    style E1 fill:#4a1f1f,color:#fff
    style E2 fill:#1e4d2b,color:#fff
    style LOCK fill:#4a351f,color:#fff

How to read the exit box. The initial stop is tight (1.5× ATR) and the trail is wide (3× ATR), so the trail sits below the initial stop at entry and only takes over once price has advanced roughly 1R. Cut fast when wrong; give room once right. That asymmetry is what produces the right-tailed return profile the strategy depends on — most trades lose a little (win rate 36.2%), a few win big (best trade +12.0R), and that is why there is no take-profit.

What happens after an initial stop. The stop always closes the trade and realizes its costs. The ticker is then locked until a successful daily full-universe scan first observes it outside the production gate and a later scan observes a fresh qualification. A continuously qualified ticker therefore cannot generate an immediate duplicate entry. Other exit reasons do not start this reset. See the daily post-stop re-entry study.

Live timing matters. The only full-universe R:R scan runs near the US close (~15:30 ET), then Telegram alerts fire immediately so manual fills can still hit MOC. Outcome eval runs later (~16:45 ET) after a fresh OHLCV fetch of the final bar. Morning jobs refresh data/sentiment/regime without scanning. Stops closed by earlier same-day intraday outcome evals can get a same-day fail observation at the near-close scan — closer to the promoted research gate_reset arm than the old morning-scan strict_gate_reset analogue. Stops after the bell still need a later day. Same-day fail+qualify cannot unlock: trade_policy requires the failure to fall on an earlier America/New_York trading date.

How It Works

Scheduled pipelines turn raw prices into a ranked, gated list of tradeable setups. Everything downstream of OHLCV is recomputed from stored data, so each refresh is cheap and idempotent. Job timing is cron-based and configurable in Admin → Jobs (default timezone America/New_York so the near-close scan tracks the cash close through DST).

Price-level architecture: two different jobs

The platform deliberately has two price-level components. Calling both of them "S/R" hid an important distinction, so the internal screening component is now named the Gate Target Ladder (GTL).

Component Purpose Lifetime Consumed by
Structural S/R A small set of meaningful support/resistance zones for humans Persisted as SRLevel Charts and alerts
Gate Target Ladder A broad set of price proposals that preserves the validated setup screen Built transiently per scan; never persisted as S/R Target table, headline R:R and activation gate
flowchart TD
    O["Ticker OHLCV history"] --> SR["Structural S/R detector<br/>volume peaks + prominent pivots + round numbers<br/>rejection and recency strength"]
    SR --> DB[("Persisted SRLevel rows")]
    DB --> UI["Charts and alerts"]

    O --> GTL["Gate Target Ladder<br/>20 price-range centers + 5-bar pivots<br/>no volume calculation"]
    GTL --> TRAFFIC["Score historical price traffic<br/>merge nearby proposals and tag side"]
    TRAFFIC --> HAS{"Any directional proposal<br/>with R:R ≥ 1.5?"}
    HAS -->|no| NONE["No setup for that direction"]
    HAS -->|yes| TARGETS["Build up to 5 target candidates<br/>estimate reach-probability"]
    TARGETS --> PRIMARY["Headline target<br/>most likely candidate clearing<br/>R:R ≥ 1.5 and probability ≥ 20%"]
    PRIMARY --> GATE{"Live activation gate<br/>headline R:R ≥ 2.0<br/>probability ≥ 20%<br/>momentum and direction pass?"}
    GATE -->|no| OBS["Keep as unqualified observation"]
    GATE -->|yes| BOOK["Eligible for production ranking/book"]
    BOOK --> EXIT["Exit only by ATR stop/trail<br/>or max hold — never by target"]

The Gate Target Ladder works step by step:

  1. Build 20 evenly spaced centers over the ticker's observed high/low range and add unfiltered five-bar swing pivots. Volume is not used.
  2. Count how often historical bars pass through each proposal, convert that traffic to strength, merge proposals within 0.5%, and label them above/below spot.
  3. For each direction, require at least one proposal with scanner R:R ≥ 1.5 against the 1.5× ATR initial stop.
  4. Collapse nearby proposals into target zones, discard unsuitable ATR distances, and retain up to five candidates spanning near to far.
  5. Estimate each candidate's probability of reaching the target before the stop. The headline target is the most likely candidate with R:R ≥ 1.5 and probability ≥ 20%; if none clears both, the most likely candidate remains headline so a distant lottery target cannot game the gate.
  6. Apply the separate live activation floor to that headline target: production requires R:R ≥ 2.0 and probability ≥ 20%, plus the momentum/direction rules.
  7. If traded, ignore the target for exits. The initial ATR stop, 3× ATR trail and maximum hold remain authoritative.

The ladder is intentionally broad and mechanical. It is not presented as market structure, and its transient negative level IDs must never be stored as chart S/R. The full-period parity run reproduced all 202,765 backtest setup candidates, all 1,086 qualified setups, and the production book exactly (Sharpe 2.03, CAGR 50.0%, max drawdown 21.4%, 321 trades). See the S/R and Gate Target Ladder research.

Ticker-chart diagnostic. The optional GTL traffic toggle draws a right-edge horizontal profile aligned to the price axis. It borrows the visual grammar of a volume profile, but not its meaning: bar width is relative historical OHLCV-bar crossings at each GTL proposal, not traded volume at that price. Hover a bar to inspect its price, crossing count, strength and source. The violet profile is deliberately distinct from the Structural S/R lines and is off by default; it is a research aid, not another trade overlay.

Below the chart, the Production rank strip makes the current 80/20 ordering snapshot explicit: a blue residual-momentum contribution and amber realized- volatility contribution add to the stored strategy rank, while separate percentile rails show each input. Only momentum carries the live activation- gate marker. These are cross-sectional scan percentiles, not historical chart indicators.

Pipelines (America/New_York)

Morning (~02:00 ET) — data and display only, no qualifying R:R scan:

  1. OHLCV — latest daily bars (Alpaca) plus the SPY benchmark; new tickers backfill ~5 years. A symbol whose bars have been stale for 3 days is probed against SEC for a Form 25/25-NSE/15 and retired on a hit (history kept — see Delisting).
  2. Sentiment — stale names that matter (top-pick feeders, watchlist, open paper, discovery net). Display context only; the activation gate is price-only.
  3. Market Trend (SPY) + AI/Tech Risk Monitor — the SPY trend guard and the v4 risk thermometer; feed no trades.
  4. Telegram alerts — change-driven (risk-quadrant etc.); quiet days stay quiet. Setup alerts still fire on the near-close pipeline after the scan.

Near-close (~15:30 ET MonFri) — the only full-universe qualifying observation:

  1. OHLCV fetch — refresh the in-progress day-t bar (same path as intraday).
  2. R:R Scan — Structural S/R, scores, Gate Target Ladder setups, residual 121 + 80/20 rank. Advances post-stop gate-reset transitions; failed scans never count.
  3. Shadow book — opt-in automated book; opens top-ranked qualified setups up to capacity at the same near-close prices. Only accepts a scan from this same pipeline run.
  4. Telegram alerts — chained immediately so manual MOC fills can still hit ~15:50/15:55.

After close (~16:45 ET MonFri):

  1. OHLCV fetch — final bar (not the partial near-close bar).
  2. Outcome Eval — resolve setups and auto-close paper trades (default 3× ATR trail, 30-day max hold).

A failing step is logged; the pipeline continues with the next. Near-close duration is logged; warn if > 10 minutes.

Intraday — light refresh

Hourly mid-session (MonFri ~10:0015:00 ET): only OHLCV → Outcome Eval, to keep prices current and close paper trades intraday. No scan/sentiment — the dashboard recomputes live R:R from the latest price.

Other jobs

Dolt earnings import (daily 02:30 ET) · SEC fundamentals import (daily 04:00 ET, also refreshes the fundamentals cache scoring reads) · Backtest (weekly) · Ticker-universe sync (daily). Alerts auto-fire only via the near-close pipeline (still manually triggerable). Deep history backfill and event study are manual-only (Admin → Jobs).

The SEC import defers a run rather than writing partial data when a filing's XBRL hasn't landed. Two bounds keep that from compounding: MISSING_XBRL_RETRY_DAYS caps how long one filing blocks promotion, and PROMOTION_CEILING_DAYS (7) caps how long the import as a whole can stay deferred — past the ceiling every unresolved filing is aged out in place so promote() queues it as a gap row, source_max_date advances, and the import self-heals. A deferred_stale alert inside that window is normal and clears on its own; check source_max_date in data_import_runs before diagnosing a wedge.

Delisting, not deletion

Retiring a symbol used to mean delete_ticker or a pruning universe bootstrap, both of which cascade through OHLCV, setups and scores. That destroys exactly the history four research documents apologise for: today's tracked universe projected backward is survivorship-biased, and hard-deleting every delisted name is what causes it. Keeping the rows preserves the option to fix that later (it does not fix it — the replay still has to model a delisting as an exit event).

tickers therefore carries delisted_on / delisted_reason (migration 032); NULL means actively traded. The filter is opt-in via ticker_service.active_only, applied to the live path only — scanner, momentum ranking, scoring, breadth, fundamentals candidates, SEC universe, earnings import, ingestion. The registry and admin views deliberately keep delisted rows visible, and run_backtest keeps them on purpose. Detection runs off OHLCV staleness (not the SEC fundamentals import, which stalls for days on unrelated Company-Facts gaps) and retires only on a Form 25/25-NSE/15 hit, so a halt or a rename keeps the existing warning instead. delisted_on is the effective date — Rule 12d2-2 makes a Form 25 removal take effect ten days after filing, so a symbol filed today keeps trading (and keeps qualifying) until that date. It is safe to automate because it is reversible: clear_delisted un-retires a false positive, where a delete had already taken the history.

From score to "top pick"

  1. Composite score — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0100) combine into a weighted composite (weights configurable; missing dimensions re-normalize). Display and ranking only — it does not select trades.
  2. Setups — the scanner builds long/short setups with a 1.5× ATR stop, generates up to five candidates from the transient Gate Target Ladder, and makes the most likely worthwhile candidate the headline target. It then adds confidence and conflict context plus a per-target reach-probability.
  3. Activation gate — a setup qualifies only if it ranks in the top residual-momentum percentile of the universe (the actual selection, long-only), its headline target clears the live R:R floor, and that target carries at least a 20% reach-probability. The confidence floor was ablated to zero effect and defaults off.
  4. Top pick — qualified setups are ordered by the production rank: 80% residual momentum percentile + 20% 6-month realized-volatility percentile. The #1 is highlighted on the Dashboard and labelled on the ticker page.

What the R:R and reach-probability in step 3 actually are. They are gate inputs, computed from a Gate Target Ladder proposal the trade will never exit at — they exist to filter setups, not to forecast the trade you're about to take. A setup with "R:R 2.4:1, 34% reach probability" is not a claim that you'll make 2.4R with 34% probability; it's a claim that this setup cleared the screen. What actually happens to a trade is in the exit box of the diagram above, and on the "what usually happens" panel in the UI. Conflating the two is the single easiest way to misread this app.

Two books: shadow (automated) and discretionary (manual)

The platform keeps two paper books, and the difference between them is the whole point.

Book Who selects What it measures
Shadow book (app/services/shadow_book_service.py) The machine — top-ranked qualified setups up to capacity, every near-close scan The strategy, faithfully
Discretionary book You, by clicking "paper trade" on a setup The strategy plus your discretion and availability

The manual book only ever contains trades the user chose to take, inside a ~20 minute window, on days they were around. The backtest that validated this strategy does none of that, which makes the manual record unusable on its own as out-of-sample evidence. The shadow book closes that gap: it mirrors _simulate_portfolio's selection rule exactly, orders on the stored strategy_rank the scanner already wrote (so the two cannot drift apart) and shares the manual book's exit policy — the only difference between the books is which qualified setups get taken.

It runs as a step of the near-close pipeline, straight after the scan so entries mark at the same near-close prices, and it only accepts a scan from the same pipeline run. It is opt-in (shadow_book_enabled, with capacity, risk % and starting equity under Admin → Settings → Performance & Shadow Book) because it writes live trades. The Dashboard's performance chart plots shadow vs discretionary vs SPY; Signals → Paper Trades still shows the discretionary book only.

Strategy Status — What's Validated and What Isn't

Read this before touching scoring, gating, or setup logic. The platform measures itself — a weekly-replay backtest plus a factor rank-IC harness (app/services/backtest_service.py) — and the verdicts below come from those reports (latest run July 2026, ~5 years of OHLCV), not from opinion.

The full experiment log lives in docs/research/ — every strategy we've tested, the result, and the decision. Check it before proposing an idea; most of the obvious ones have already been run and rejected.

Component Verdict Evidence
Residual 12-1 cross-sectional momentum (the activation gate, long-only) Production gate — in-sample edge Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable
3× ATR trailing exit (+ 1.5× ATR initial stop, 30-day max hold) Production exit — best Sharpe of every exit tested Beat hold / SMA50 / 20-day-low / technical-40 and both take-profit variants (July 2026)
Post-stop gate reset Production re-entry policy The initial stop always closes; the ticker must later fail the daily gate and subsequently qualify again. At the then-production capacity of 10: Sharpe 1.67 → 1.77, CAGR 45.2% → 48.3%, DD 24.3% → 21.6% versus immediate re-entry. Capacity has since been raised to 15 — see the open question under the re-entry section. Full study
Book capacity 15 (raised from 10, 2026-08-05) Production sizing The focused daily capacity bracket found the count cap was binding and cost real compounding: +1.075pp CAGR paired, 51 paths better / 2 worse, drawdown unchanged. Cash plus the 20% notional cap saturates the book near 12, so the cap no longer binds. Findings
Structural S/R Human-facing context only — not a gate and not an exit Clean, capped zones are persisted for charts and alerts. The scanner deliberately does not read them.
Gate Target Ladder Gate input only — not market structure and not an exit Volume-free range grid + pivots preserves the useful legacy screening behavior exactly: 1,086/1,086 qualified setups retained and identical Sharpe 2.03 / CAGR 50.0% / DD 21.4% / 321 trades. The exit never reads its target. Full write-up
Composite score + 5 dimensions Display/ranking only Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" dimension is 5/20-day ROC — NOT the validated 12-1 factor (that lives in momentum_service)
LLM sentiment Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) Deliberately kept out of the setup engine; no point-in-time history to validate against yet
Fundamentals Feeds composite + confidence only Latest values only, no history — same limitation
Short setups Excluded while the momentum gate is active Backtest showed shorts fight the trend and drag expectancy
Expected-value gate (removed June 2026) Degenerate — do not resurrect Structurally favored distant lottery targets; selected worse-than-random setups. Orphaned settings dropped in migration 020
Gate target as a take-profit (tested July 2026) Rejected Sharpe 2.04 → 1.47, CAGR halved. Win rate rose — it truncates the right tail where the edge lives
"Clear-air" gate relaxation (tested July 2026) Rejected — failed out-of-sample Strictly better in-sample (Sharpe 2.07 / CAGR 62.3% / DD 20.1%), then lost on a real train/test split (Sharpe 2.78 → 2.45). A cautionary tale: nested lookbacks are not OOS

Caveats on the momentum result: in-sample, roughly one market regime, costs/slippage approximated at 0.1% per side, and residual momentum still needs SPY benchmark history to compute. The out-of-sample proof is the forward record of the shadow book — the automated twin that takes every top-ranked qualified setup, with no discretion or availability mixed in. The Dashboard chart tracks it against the discretionary book and SPY; Signals → Backtest is what it is being compared against.

Daily post-stop re-entry decision (2026-07-17)

The production policy is normal gate reset, evaluated with daily setup opportunities and live-like full-universe ranking. An initial stop always closes. Re-entry unlocks only after a later successful daily scan observes the ticker failing the gate and a subsequent scan observes it qualifying again. The study replayed 1,011,248 point-in-time candidate observations across 505 tickers from 2022-06-24 through 2026-07-02, with the production GTL gate, 80/20 rank, exit, fees, sizing, and 10-position capacity.

Re-entry policy Total return CAGR Max DD Sharpe Trades
Immediate 348.4% 45.2% -24.3% 1.67 489
Gate reset (selected study arm) 388.1% 48.3% -21.6% 1.77 472
Strict gate reset (live timing analogue) 342.7% 44.8% -23.4% 1.68 471
Fixed five-session cooldown 250.8% 36.6% -22.2% 1.47 473

In the disjoint 2025+ book, gate reset also beat immediate re-entry (Sharpe 1.66 vs 1.55; CAGR 41.8% vs 39.3%) and the fixed five-session rule (Sharpe 1.43; CAGR 32.7%). Its lead over both survived costs of 0.2% and 0.3% per side. The result is capacity-specific: cooldown 5 won at capacity 5, while immediate had slightly higher return and Sharpe at capacity 15.

Open question (since 2026-08-05). This study was run — and gate reset promoted — at capacity 10. Production capacity was subsequently raised to 15, which is the one capacity in the matrix where immediate re-entry edged ahead. The re-entry policy is therefore currently running outside the portfolio it was validated on. Nothing else changed, and the two arms differed only modestly, but the matrix should be rerun at capacity 15 before treating gate reset as settled. Until then, keep gate reset (the incumbent) rather than switching on an untested read.

Those promotion numbers belong to the selected normal-reset study arm. Under the pre-cutover morning-scan scheduler (scan always before any outcome eval), live first-observation timing matched the stricter strict_gate_reset analogue (full-period Sharpe 1.68 / CAGR 44.8% / DD 23.4%). After the near-close cutover (2026-07), stops closed by earlier same-day intraday evals can receive a same-day fail observation at ~15:30 ET — moving live behavior toward the promoted gate_reset arm. Requalification still requires a later America/New_York trading date than the failure (trade_policy distinct-day guard). Full definitions and all nine policy arms: docs/research/post-stop-reentry.md; execution evidence: docs/research/execution-recovery.md.

gate_reset and a simple next_session block happened to produce the same executed live-universe portfolio in this sample. Their rules are still different: this establishes that same-day re-entry was harmful here, but does not isolate a separate historical return premium from the reset condition. Gate reset was promoted because it represents a genuinely new signal episode and did not sacrifice results in the production book. Full definitions, all nine policy arms, cost/capacity sensitivity, and legacy-rank results are in docs/research/post-stop-reentry.md; source report: reports/daily_reentry_matrix.json.

Historical weekly production baseline (pre gate-reset)

Use this as the historical ranking/exit regression guardrail, not as a return promise or the current re-entry-policy result. This run predates the post-stop gate reset and the 2026-08-05 capacity raise to 15, and uses weekly entry replay, so its portfolio headline is not directly comparable with the daily matrix above. Backtest run: local production SQLite snapshot, 506 tickers, weekly cadence, 30-trading-day horizon, 2022-06-28 → 2026-07-02, 0.1% per-side costs, price-only SPY benchmark. Numbers below are the 2026-07-11 run (reports/backtest-20260711-prod-baseline.json) — measured after the primary-target probability floor shipped, which pruned lottery-target setups (1,428 → 1,089 qualified) and lifted Sharpe on all three promotion contenders.

Item Historical weekly baseline
Strategy version residual_highvol_80_20_atr_trail3_v1
Production gate Long-only, residual 12-1 momentum percentile >= 80, headline gate-target R:R >= 2.0 (live activation_min_rr; code default 2.0), primary-target reach-probability >= 20%, NEUTRAL excluded, confidence floor off (0)
Production rank 80% residual momentum percentile + 20% 6-month realized-volatility percentile
Exit Initial ATR stop plus 3x ATR trailing stop, max 30 trading days
Portfolio CAGR +50.4%
Portfolio total return +413.8% vs SPY +95.7%
Max drawdown -21.4%
Sharpe 2.04 daily, annualized
Trades 320
Win rate 37.5%
Average hold 15.3 trading days
Best / worst trade +12.9R / -3.3R
How trades actually ended initial stop 144 (45%) · trailing stop 98 (31%) · max hold 78 (24%) · target 0 (0%)

That last row is the strategy in one line: a 37.5% win rate is fine because the +12.9R tail pays for every 1R stop. It is also why no take-profit exists — and why the S/R "target" shown in the UI is a screening artifact, not a plan.

Promotion evidence from the same snapshot:

Candidate CAGR Max DD Sharpe Trades Read
Legacy residual 80 + 30d hold +49.6% -15.8% 2.02 300 Previous production baseline. Still the shallowest drawdown of the three
Residual/high-vol 80/20 + 30d hold +51.9% -22.2% 2.00 303 The vol tilt buys CAGR and pays for it in drawdown
Residual/high-vol 80/20 + 3x ATR trail +50.4% -21.4% 2.04 320 Promoted: best Sharpe. The ATR trail recovers part of the drawdown the vol tilt costs
Pure high-vol 80 + 30d hold +31.6% -34.8% 1.12 476 Rejected: standalone volatility was too volatile
Low-vol 80 + 30d hold +2.7% -19.5% 0.29 240 Rejected: no useful edge

Read the top three honestly: the production book wins on Sharpe, not on every axis. The 80/20 vol tilt buys ~2pp of CAGR over the legacy residual-only book but costs ~6pp of drawdown, and the ATR trail hands part of that drawdown back. If drawdown ever matters more than risk-adjusted return here, legacy residual 80 + hold is the row to revisit.

The conclusion is not "trade high volatility alone." Keep residual momentum as the entry gate, use realized volatility only as a small ranking tilt, and add the ATR trail as defensive exit discipline.

Live-ranking note: the backtest ranks residual momentum and volatility inside each weekly setup-candidate cross-section. The live scanner computes the same 80/20 formula across the current ticker universe before scanning so every generated setup carries a stable ticker-level rank. That is the production approximation; reconcile it later only if candidate-only post-scan ranking proves materially different.

Parity guard (July 2026): the portfolio monitor's Production row replays the runtime configuration — the live activation gate (qualified flag) and the Admin exit policy (mode / ATR multiplier / hold days) — so tuning the strategy in Admin is reflected in the next backtest run instead of silently diverging. Constants defined on both sides (exit defaults, trail width, the 80/20 ordering weights, the promoted cutoff) are pinned by tests/unit/test_prod_strategy_parity.py, and the ordering weights are single-sourced from momentum_service.

Tuned and confirmed — do not retest without new data (July 2026)

A systematic single-variable sweep (offline prod snapshot, production gate/rank/exit, 2022-06 → 2026-07 plus disjoint 202223 / 202426 folds) confirmed every production setting except book size, which a later focused bracket reversed (see the row below). Retesting these against the same ~4-year snapshot is wasted compute and invites overfitting; revisit only with meaningfully new data (longer history or broader universe) — or, as with capacity, a demonstrably better measurement lens.

Knob tested Verdict Evidence
ATR trail multiple {1.54.0} Keep 3.0 Return+Sharpe peak; ≤2.0 whipsaws out the momentum right tail; ≥2.5 is a plateau
SPY 200d-MA regime overlay (block entries / go flat) Reject Halves return (315%→138%) with zero drawdown benefit — the ATR trail already manages downside, and the filter blocks the recovery-phase entries that make the money
Momentum lookback: 6-1, 3-1, 12-7 (Novy-Marx), composites Keep residual 12-1 6-1/3-1 rank-IC ≈ 0; 12-7 IC 0.045 / t 1.58 — weaker than residual 12-1 (0.055 / t 1.98)
Selection cutoff {70, 75, 85, 90} × book size {10, 15, 20} Keep cutoff 80; book size raised to 15 (2026-08-05) The cutoff is monotonically worse in both directions from 80. The book-size half of this row was reversed: the weekly replay's "the 10-slot cap never binds" read came from EV per trade, which is the wrong lens for anything that changes trade count. The focused daily bracket found cap 10 was binding and cost +1.075pp CAGR; at 15 the cap never bound in any cell (max observed 12 concurrent, zero full-book skips)
Position sizing: equal-weight, inverse-vol, risk-% sweep Keep 1% fixed-fractional See the inverse-vol warning below
Post-stop re-entry: immediate, fixed 25 sessions, gate resets, confirmation filters Keep normal gate reset — but measured at capacity 10, and capacity is now 15 Sharpe 1.77 vs 1.67 immediate and 1.47 cooldown 5. The "rerun before changing portfolio capacity" caveat is now outstanding — see the open question above
FIP path-smoothness as an in-book tie-breaker/filter Reject (but see the lead below) Non-monotonic across FIP quintiles within the qualified set; either half of a median split underperforms the full book — thinning the entry stream costs more compounding than the tilt returns

Two findings future sessions must not re-litigate:

  • The "inverse-vol sizing win" (July 2026) was mis-attributed — do not resurrect. The diagnostic sized notional = equity × 1% / vol_6m, and the 20% notional cap bound on 95% of entries, so it actually measured "~5 positions × 20% notional each" — a concentration/risk-appetite bump economically equivalent to raising risk to 1.5%, not vol-managed sizing. Genuine inverse-vol sizing (risk budget × median-vol/vol) cuts max drawdown to 18.2% but costs ~58pp total return at flat Sharpe: a risk-preference trade, not edge.
  • fip_id — Da/Gurun/Warachka information discreteness over the 12-1 formation window — is the strongest cross-sectional signal on the production universe: IC 0.045, t = 2.91, correct sign (continuous-information winners outperform). It clears the iron-rule bar in isolation but does not improve this book (the momentum gate already captures the effect in-sample). Phase B (liquid-1500, research branch only): unconditional fip fails iron rule (0.017 / t 1.85); mom-conditional fip (0.088 / t 4.58) is a book-tilt candidate only after a baseline breadth mom book is proven. Do not cite the orphaned 21:14 row (+0.0575) — it raced a partial research.sqlite. See docs/research/fip-breadth-ic.md.

The iron rule for strategy changes

A signal earns its way into selection only through the factor harness:

  1. Add it as a point-in-time function of past bars in _signal_values() (backtest_service.py).
  2. Run the backtest (Admin → Jobs, or the weekly run) and read the report's signal_eval section. This one is local-report only — the deployed Backtest tab does not render it (see Reading a local backtest report below).
  3. Wire it into the gate or ranking only if |mean IC| ≳ 0.03 with a consistent sign and reliable: true (≥ 12 non-overlapping windows).

Corollaries: never let an unvalidated score gate setups; the outcome evaluator must keep scoring all setups (unqualified ones are the control group); LLM output stays display-only in the quant path.

Highest-value next experiments (in order)

Check docs/research/ first — 13 strategy ideas have already been tested and rejected, including the obvious ones (take-profit exits, regime overlays, inverse-vol sizing, shorts, sector-residual momentum).

  1. Forward monitor the promoted strategySignals → Backtest behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison, and the Dashboard chart carries the forward record. Forward months of the shadow book are the only evidence the snapshot cannot provide; the July 2026 tuning pass closed every in-sample lead. (Trailing-stop sensitivity and the capacity bracket are done — capacity was raised to 15.)
  2. Signal context snapshots — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only.
  3. Breadth is no longer free leverage — Phase B found residual-mom t-stat fell on liquid-1500 vs the 505-name fingerprint (0.055/1.98 → 0.029/1.33). Any breadth book must clear a pre-registered baseline arm before fip tilts mean anything. (Deeper history was considered and declined.)

Key Use Cases

  • Find today's best long setup. On the Dashboard, the Top Setups table lists residual-gated qualified setups ranked by the production 80/20 residual/high-vol score, with the #1 flagged "Top pick". Each row opens the ticker page for its chart, Structural S/R, Gate Target Ladder targets and entry/stop.
  • Track a trade you took. Mark a setup as a paper trade: it's marked-to-market against the latest close, auto-closed by the active exit policy (default: 3x ATR trail with a 30-trading-day max hold), and its sentiment stays fresh while open. Signals → Paper Trades shows the realized edge of your discretionary book; the Dashboard chart puts it next to the automated shadow book and SPY.
  • Ask whether the strategy is worth trading at all. Signals → Backtest replays the promoted strategy over history — portfolio monitor vs SPY over selectable lookbacks, headline risk-adjusted metrics (Sharpe, Sortino, Gain-to-Pain, dollar profit factor) and the report's own recommendation — with the live-outcome evaluation panel underneath it.

Stack

Layer Tech
Backend Python 3.12+, FastAPI, Uvicorn, async SQLAlchemy, Alembic
Database PostgreSQL (asyncpg)
Scheduler APScheduler — daily & intraday pipelines, fundamentals, alerts, regime, backtest
Frontend React 18, TypeScript, Vite 5
Styling Tailwind CSS 3 with custom glassmorphism design system
State TanStack React Query v5 (server), Zustand (client/auth)
Charts Canvas 2D candlestick chart with S/R overlays
Routing React Router v6 (SPA)
HTTP Axios with JWT interceptor
Data providers Alpaca (OHLCV); OpenAI / Gemini / DeepSeek / xAI (sentiment, pluggable); SEC EDGAR Company Facts + DoltHub earnings (fundamentals, bulk import); FRED (regime); Telegram (alerts)

Features

Backend

  • Ticker registry with reversible delisting (history preserved) plus an explicit cascade delete
  • Universe bootstrap for sp500, nasdaq100, nasdaq_all via admin endpoint — free public sources (Wikipedia / NASDAQ Trader), then the cached snapshot, then a built-in seed list. The seeds are representative, not complete, so a fresh install bootstrapped while the public source is unreachable gets a partial universe; a warm instance falls through to its cache.
  • OHLCV price storage with upsert and validation
  • Technical indicators: ADX, EMA, RSI, ATR, Volume Profile, Pivot Points, EMA Cross
  • Structural Support/Resistance detection with rejection/recency strength, ATR-adaptive merging and a hard cap; persisted for charts and alerts
  • Transient Gate Target Ladder — volume-free range grid plus pivots, used only for nominal targets, reach-probability and gate R:R
  • Sentiment analysis with time-decay weighted scoring
  • Fundamental data tracking (P/E, revenue growth, earnings surprise, market cap)
  • 5-dimension scoring engine (technical, S/R quality, sentiment, fundamental, momentum) with configurable weights
  • Risk:Reward scanner — long and short setups, 1.5x ATR stops, Gate Target Ladder nominal targets, configurable scan R:R threshold (default 1.5:1 — distinct from the activation floor below)
  • Activation gate — qualifies setups on a residual-momentum percentile floor (the actual selection), a headline gate-target R:R floor (prod: 2.0) and a 20% primary-target reach-probability floor (validated long-only edge)
  • Recommendation layer — directional confidence, conflict detection, per-target reach-probability
  • Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: 3x ATR trail with a 30-trading-day max hold; time / percent-trailing / target-stop selectable), realized track record + outcome evaluation
  • Shadow book — opt-in automated twin of the backtest's selection rule (top-ranked qualified setups up to capacity, every near-close scan), sharing the manual book's exit policy; the honest forward out-of-sample record
  • System events — structured job/import/data warnings with acknowledgement, surfaced in Admin and deduplicated for alerting
  • Market-regime guard + observational State/Warning monitor (fixed-basket breadth, VIX, credit level + impulse) with a manual chronological correction study
  • Telegram alerts (e.g. regime-quadrant changes)
  • User-curated watchlist (cap: 20), enriched with composite score, R:R and S/R summary
  • JWT auth with admin role, configurable registration, user access control
  • Cron-scheduled pipelines (admin-configurable) with per-job enable/disable and live status monitoring
  • Admin panel: user management, data cleanup, job control, system settings

Frontend

  • Glassmorphism UI with frosted glass panels, gradient text, ambient glow effects, mesh gradient background
  • Interactive candlestick chart (Canvas 2D) with hover tooltips showing OHLCV values
  • Support/Resistance level overlays on chart (top 6 by strength, dashed lines with labels)
  • Optional GTL price-traffic profile on the ticker chart (right-edge diagnostic; explicitly not volume)
  • Production-rank strip below the ticker chart (80/20 contribution ledger plus separate momentum and volatility percentiles)
  • Data freshness bar showing availability and recency of each data source
  • Watchlist with composite scores, R:R ratios, and S/R summaries
  • Ticker detail page: chart, scores, sentiment breakdown, fundamentals, technical indicators, S/R table
  • Rankings table with configurable dimension weights
  • Trade scanner showing detected R:R setups
  • Backtest tab: portfolio monitor vs SPY over selectable lookbacks, headline risk-adjusted tiles (Sharpe, Sortino, Gain-to-Pain, dollar profit factor), the report's recommendation card, and a live-outcome evaluation panel
  • Dashboard performance chart: cumulative shadow book vs discretionary book vs SPY since the configured start date
  • Paper Trades tab: open/closed discretionary trades with realized R and P&L tiles
  • Admin page: user management, job status with live indicators, enable/disable toggles, pipeline readiness, system-event log, ticker management, data cleanup, system settings
  • Protected routes with JWT auth, admin-only sections
  • Responsive layout with mobile navigation
  • Toast notifications for async operations

Pages

Route Page Access
/login Login Public
/register Register Public (when enabled)
/ Dashboard — top setups, open trades, regime, shadow-vs-manual-vs-SPY performance chart (default) Authenticated
/market Market — watchlist + rankings tabs Authenticated
/signals Signals — Setups / Paper Trades / Backtest tabs Authenticated
/regime AI/Tech Risk Monitor Authenticated
/ticker/:symbol Ticker Detail Authenticated
/admin Admin Panel Admin only

Legacy routes redirect: /watchlist/market, /rankings/market?tab=rankings, /scanner/signals, /performance/signals?tab=track (the Paper Trades tab — track stays its slug so the old link keeps working).

API Endpoints

All under /api/v1/. Interactive docs at /docs (Swagger) and /redoc.

Group Endpoints
Health GET /health
Auth POST /auth/register, POST /auth/login
Tickers POST /tickers, GET /tickers, DELETE /tickers/{symbol}, POST /tickers/{symbol}/delisting, DELETE /tickers/{symbol}/delisting
OHLCV POST /ohlcv, GET /ohlcv/{symbol}
Ingestion POST /ingestion/fetch/{symbol}
Indicators GET /indicators/{symbol}/{type}, GET /indicators/{symbol}/ema-cross
S/R Levels GET /sr-levels/{symbol}
Gate Target Ladder GET /gate-target-ladder/{symbol}
Sentiment GET /sentiment/{symbol}
Fundamentals GET /fundamentals/{symbol}
Scores GET /scores/{symbol}, GET /rankings, PUT /scores/weights
Trades GET /trades, GET /trades/{symbol}, GET /trades/{symbol}/history, GET /trades/activation, GET /trades/performance
Paper Trades GET /paper-trades, POST /paper-trades, POST /paper-trades/{id}/close, GET /paper-trades/equity-curve, GET /paper-trades/performance (shadow vs manual vs SPY), GET/PUT /paper-trades/exit-policy
Market / Regime GET /market/regime, GET /regime/monitor, GET/PUT /regime/config, GET /regime/history, GET /regime/event-study, GET/PUT /regime/fundamentals, POST /regime/fundamentals/refresh, GET /backtest/report
Jobs GET /jobs/running
Watchlist GET /watchlist, POST /watchlist/{symbol}, DELETE /watchlist/{symbol}
Admin GET /admin/users, POST /admin/users, PUT /admin/users/{id}/access, PUT /admin/users/{id}/password, PUT /admin/settings/registration, GET /admin/settings, PUT /admin/settings/{key}, GET/PUT /admin/settings/{recommendations,activation,schedule,performance,shadow-book,sentiment,alerts,ticker-universe}, POST /admin/settings/{sentiment,alerts}/test, POST /admin/tickers/bootstrap, POST /admin/tickers/backfill-names, POST /admin/data/cleanup, POST /admin/track-record/reset, GET /admin/jobs, POST /admin/jobs/{name}/trigger, PUT /admin/jobs/{name}/toggle, GET /admin/pipeline/readiness, GET /admin/system-events, GET /admin/system-events/summary, POST /admin/system-events/acknowledge

Development Setup

Prerequisites

  • Python 3.12+
  • PostgreSQL (via Homebrew on macOS: brew install postgresql@17)
  • Node.js 18+ and npm

Backend Setup

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Configure environment
cp .env.example .env
# Edit .env with your values (see Environment Variables below)

# Start PostgreSQL and create database
brew services start postgresql@17
createdb stock_data_backend
createuser stock_backend

# Run migrations
alembic upgrade head

# Start the backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

A default admin/admin account is created on first startup. Open http://localhost:8000/docs for Swagger UI.

Frontend Setup

cd frontend
npm install
npm run dev

Open http://localhost:5173 for the Signal Dashboard. The Vite dev server proxies /api/v1/ requests to the backend at http://127.0.0.1:8000.

Frontend Build

cd frontend
npm run build     # TypeScript check + production build → frontend/dist/
npm run preview   # Preview the production build locally

Tests

# Backend tests (in-memory SQLite — no PostgreSQL needed)
pytest tests/ -v

# Frontend: there is no test suite and no `test` script at all. The frontend
# check is the full TypeScript build:
cd frontend
npm run build

Local Backtest Snapshots

For research loops, run the production backtest locally from a SQLite snapshot instead of deploying and clicking the Admin job. The snapshot contains only the tables needed by run_backtest: tickers, OHLCV bars, SPY benchmark closes, and the activation / recommendation / paper-exit settings. Secrets and cached reports are not copied.

The paper_% settings must be copied: the portfolio monitor's Production row replays the runtime exit policy via get_exit_policy(). Without them a snapshot silently falls back to the code defaults, so a live-tuned exit would not be reflected and the local run would disagree with prod for no visible reason.

  1. Open an SSH tunnel to the production Postgres instance:
ssh -N -L 15432:127.0.0.1:5432 deploy@your-server
  1. In another terminal, create or refresh the snapshot:
# macOS/Linux
python scripts/create_backtest_snapshot.py \
  --database-url "postgresql+asyncpg://stock_backend:PASSWORD@127.0.0.1:15432/stock_data_backend" \
  --output backtest_snapshots/prod.sqlite \
  --force

# Windows PowerShell
.venv\Scripts\python.exe scripts\create_backtest_snapshot.py `
  --database-url "postgresql+asyncpg://stock_backend:PASSWORD@127.0.0.1:15432/stock_data_backend" `
  --output backtest_snapshots\prod.sqlite `
  --force
  1. Run the backtest fully offline from the snapshot:
# macOS/Linux
python scripts/run_backtest_snapshot.py backtest_snapshots/prod.sqlite --workers 6

# Windows PowerShell
.venv\Scripts\python.exe scripts\run_backtest_snapshot.py backtest_snapshots\prod.sqlite --workers 6 --allow-spawn

Weekly remains the resource-safe default. Add --cadence daily for live-like daily entry opportunities; this performs roughly five times as many setup evaluations. To generate the complete weekly/daily × immediate/gate-reset comparison in one invocation, use:

python scripts/run_backtest_cadence_comparison.py backtest_snapshots/prod.sqlite --workers 7

On Windows, add --allow-spawn. The comparison runner writes the two full cadence reports plus one compact four-arm report. For the larger nine-policy daily research matrix used in the post-stop decision, see scripts/run_daily_reentry_matrix.py and the research record.

On an 8-thread machine, --workers 6 is a good starting point: it leaves a couple of threads for Windows, the shell, and browser/UI work while still using most of the CPU.

The runner writes reports/backtest-<timestamp>.json and prints the headline metrics. Keep the SSH tunnel open only while creating the snapshot; the backtest run itself is local/offline. backtest_snapshots/ and generated backtest reports are git-ignored.

The local runner, scheduled job, and Admin UI all default to production_gtl, matching the live scanner's target path. For a deliberate comparison, select Structural S/R (comparison) in the UI or pass --target-model structural_sr locally. Every report records the selected model and whether it is the production path.

Archived GTL tuning decision

The completed replacement, cohort-composition, and strength-sensitivity matrices found no stable improvement over the frozen Gate Target Ladder. The temporary matrix runners and tuning hooks have been retired; their three compact consolidated report pairs remain in reports/ as the decision audit. Keep the GTL unchanged and evaluate any future challenger only on new forward data. See the full research record.

Reading a local backtest report

The deployed Signals → Backtest tab is deliberately trimmed to validation (portfolio monitor vs SPY, headline metrics, the report's recommendation, and the live-outcome evaluation panel). The strategy-tuning tables that used to live there now live only in the local report — inspect these reports/backtest-<timestamp>.json sections and produce the matching decision. Every change still goes through the factor harness first (see The iron rule for strategy changes above).

Report section What to read Decision it drives
overall_qualified vs overall_all Is qualified net expectancy above the all-setups baseline? Sanity — is the gate adding anything at all
sweep Net avg R and trade count at each residual-momentum cutoff Where to set the momentum percentile (Admin → Settings → Activation)
gate_ablation Net expectancy with each floor removed Drop a floor only if removing it doesn't hurt net expectancy
time_exit_sweep Net avg R / net R-per-day by hold length Whether a fixed time exit beats the promoted ATR trail
portfolio_monitor, portfolio_sim, strategy_variants CAGR, Sharpe, max drawdown, per-year returns Promote a strategy only if it beats the current baseline on CAGR/Sharpe/DD
production_cadence_comparison Immediate vs production gate reset at the selected weekly or daily cadence Isolates the re-entry rule while keeping gate, rank, exit, fees, sizing, and capacity fixed
signal_eval Mean IC, t-stat, IC>0 %, reliable Iron rule: wire a new factor in only if |IC| ≳ 0.03 with a consistent sign and reliable: true
holdout (opt-in) Train vs test books, split by entry date The only honest OOS read. Set BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD
recommendation, research_recommendation The report's own headline read A starting point, not a substitute for the sections above

Out-of-sample validation. The portfolio_monitor lookbacks (6m / 1y / 3y / 5y / all) are nested windows that all end today — every one of them overlaps the data an idea was found on, so none of them is a holdout. A rule that looks good across all five can still be an in-sample artifact (this exact trap ate the clear-air experiment; see the research log). For a real train/test split by entry date:

BACKTEST_HOLDOUT_SPLIT=2024-07-01 python scripts/run_backtest_snapshot.py \
    backtest_snapshots/prod.sqlite --workers 7 --allow-spawn

Research-only flags, all off by default (the default report is byte-identical to the shipped baseline):

Flag What it does
BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD Adds a holdout section: train (entries before) vs test (entries on/after), as disjoint books
BACKTEST_MIN_RR_SWEEP=1 Sweeps the activation R:R floor against portfolio Sharpe. Combine with BACKTEST_HOLDOUT_SPLIT to sweep out-of-sample
BACKTEST_RESEARCH_EXITS=1 Adds the rejected take-profit exit rows to the exit comparison
BACKTEST_ATR_TARGET_FALLBACK=k Synthesizes a k×ATR target where S/R offers none
BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1 Restricts that fallback to setups with genuinely no structure ahead

portfolio_monitor and recommendation are the sections surfaced on the deployed Backtest tab (the monitor chart/tiles and "What this backtest recommends"; the recommendation is rebuilt on read, so it always matches the lookback on screen and flags one it was not computed on). Everything else in this table is intentionally local-only.

Environment Variables

Configure in .env (copy from .env.example):

Variable Required Default Description
DATABASE_URL Yes PostgreSQL connection string (postgresql+asyncpg://...)
JWT_SECRET Yes Random secret for JWT signing
JWT_EXPIRY_MINUTES No 60 JWT token expiry
ALPACA_API_KEY For OHLCV Alpaca Markets API key
ALPACA_API_SECRET For OHLCV Alpaca Markets API secret
GEMINI_API_KEY For sentiment Google Gemini API key
GEMINI_MODEL No gemini-2.0-flash Gemini model name
OPENAI_API_KEY For sentiment (OpenAI path) OpenAI API key
OPENAI_MODEL No gpt-4o-mini OpenAI model name
OPENAI_SENTIMENT_BATCH_SIZE No 5 Micro-batch size for sentiment collector
DEEPSEEK_API_KEY / XAI_API_KEY For sentiment (those paths) Alternative pluggable sentiment providers
SEC_USER_AGENT For fundamentals placeholder SEC EDGAR requires a real name (contact: email) UA — the shipped default is a placeholder and SEC will throttle/refuse it
SEC_REQUEST_SPACING_SECONDS No 0.2 Politeness delay between SEC requests
SEC_MAX_RETRIES / SEC_REQUEST_TIMEOUT_SECONDS No 4 / 30 SEC client retry and timeout budget
DOLT_BINARY For earnings import dolt Path to the dolt executable
DOLT_DATA_DIR / DOLT_EARNINGS_SUBDIR No dolt-data / earnings Local Dolt clone location
DOLT_MIN_FREE_DISK_GB No 5.0 Refuse to clone/pull below this free space
DOLT_COMMAND_TIMEOUT_SECONDS No 600 Per-command Dolt timeout
FRED_API_KEY Optional (risk monitor) FRED key for the AI/Tech risk monitor (VIX, credit spreads)
TELEGRAM_BOT_TOKEN Optional (alerts) Telegram bot token for alerts (can also be set in Admin)
TELEGRAM_CHAT_ID Optional (alerts) Telegram chat id for alerts
DATA_COLLECTOR_FREQUENCY No daily OHLCV collection schedule (legacy — see note below)
SENTIMENT_POLL_INTERVAL_MINUTES No 30 Sentiment polling interval
RR_SCAN_FREQUENCY No daily R:R scanner schedule
DEFAULT_WATCHLIST_AUTO_SIZE No 10 Auto-watchlist size
DEFAULT_RR_THRESHOLD No 1.5 Minimum R:R ratio for setups
OHLCV_HISTORY_DAYS No 1825 Backfill depth for new tickers (~5 years)
OUTCOME_EVALUATION_MAX_BARS No 30 Bars the outcome evaluator resolves a setup over
BACKTEST_WORKERS No 4 Worker processes for the scheduled backtest
DB_POOL_SIZE No 5 Database connection pool size
LOG_LEVEL No INFO Logging level

Note: Pipeline timing (daily / intraday / fundamentals cron, timezone) is configured at runtime in Admin → Jobs and stored in the DB — the *_FREQUENCY env vars are legacy fallbacks for the few jobs still on interval triggers (alerts, universe sync).

Production Deployment (Debian 12)

Ongoing deploys are automated. Every push to main triggers the Gitea Actions pipeline (.gitea/workflows/deploy.yml): lint → test → rsync to the server → pip installalembic upgrade head → restart signalplatform.service → health check. There is no manual deploy step; the steps below are only for provisioning a new server.

1. Install dependencies

sudo apt update && sudo apt install -y python3.12 python3.12-venv postgresql nginx rsync

2. Create the deploy user

The pipeline connects over SSH as this user; it owns the app directory and needs passwordless permission to restart the service:

sudo useradd -m deploy
sudo mkdir -p /opt/signalplatform
sudo chown deploy:deploy /opt/signalplatform
echo 'deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart signalplatform.service' | sudo tee /etc/sudoers.d/deploy-restart

3. Configure the pipeline (Gitea repo settings)

Variables: DEPLOY_HOST, DEPLOY_USER (deploy), DEPLOY_PATH (/opt/signalplatform), SSH_KNOWN_HOSTS (host fingerprint), SSH_PORT. Secret: SSH_PRIVATE_KEY (matching the deploy user's authorized key).

4. Configure the app

# After the first pipeline run has synced the files:
cp /opt/signalplatform/.env.example /opt/signalplatform/.env
# Edit .env with production values (strong JWT_SECRET, real API keys, etc.)

.env is excluded from the rsync, so it survives every deploy.

5. Database

Either trigger the workflow manually (workflow_dispatch) with run_setup_db: true — the deploy then runs deploy/setup_db.sh instead of plain migrations — or run it once by hand:

DB_NAME=stock_data_backend DB_USER=stock_backend DB_PASS=strong_password ./deploy/setup_db.sh

6. Systemd service

sudo cp deploy/signalplatform.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now signalplatform

The unit runs uvicorn on 127.0.0.1:8998 as the deploy user, with WorkingDirectory=/opt/signalplatform.

7. Nginx reverse proxy

sudo cp deploy/nginx.conf /etc/nginx/sites-available/signalplatform
sudo ln -s /etc/nginx/sites-available/signalplatform /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

Nginx serves the frontend static files from frontend/dist/ (built on the CI runner and rsynced) and proxies /api/v1/ to the backend.

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d signal.thiessen.io

Verify

curl https://signal.thiessen.io/api/v1/health

Project Structure

app/
├── main.py              # FastAPI app, lifespan, router wiring
├── config.py            # Pydantic settings from .env
├── database.py          # Async SQLAlchemy engine + session
├── dependencies.py      # DI: DB session, auth guards
├── exceptions.py        # Exception hierarchy
├── middleware.py         # Global error handler → JSON envelope
├── cache.py             # LRU cache with per-ticker invalidation
├── ssl_bootstrap.py     # TLS trust-store bootstrap for outbound calls
├── scheduler.py         # APScheduler job definitions
├── job_catalog.py       # Single source of truth for job names + pipeline step lists
├── models/              # SQLAlchemy ORM models
├── schemas/             # Pydantic request/response schemas
├── services/            # Business logic layer
├── providers/           # External data provider integrations
└── routers/             # FastAPI route handlers

frontend/
├── index.html           # SPA entry point
├── vite.config.ts       # Vite config with API proxy
├── tailwind.config.ts   # Tailwind + glassmorphism theme
├── package.json
└── src/
    ├── App.tsx          # Route definitions
    ├── main.tsx         # React entry point
    ├── api/             # Axios API client modules (one per resource)
    ├── components/
    │   ├── admin/       # User table, job controls, settings, data cleanup
    │   ├── auth/        # Protected route wrapper
    │   ├── charts/      # Canvas candlestick chart
    │   ├── dashboard/   # Top setups, open trades, shadow-vs-manual performance chart
    │   ├── layout/      # App shell, sidebar, mobile nav
    │   ├── rankings/    # Rankings table, weights form
    │   ├── scanner/     # Trade table
    │   ├── signals/     # Setups / Paper Trades / Backtest panels
    │   ├── ticker/      # Sentiment panel, fundamentals, indicators, S/R overlay
    │   ├── ui/          # Badge, toast, skeleton, score card, confirm dialog
    │   └── watchlist/   # Watchlist table, add ticker form
    ├── hooks/           # React Query hooks (one per resource)
    ├── lib/             # Types, formatting utilities
    ├── pages/           # Page components (Login, Register, Dashboard, Market, Signals, Regime, Ticker, Admin)
    ├── stores/          # Zustand auth store
    └── styles/          # Global CSS with glassmorphism classes

docs/
├── dolt-integration-plan.md     # Design record for the Dolt/SEC fundamentals workstream
├── dolt-sec-a3-design.md
├── fundamentals-deployment.md
└── research/            # Experiment log: what was tested, the result, the decision
    ├── README.md        # Overview — start here before proposing a strategy change
    ├── sr-levels-and-exits.md
    ├── post-stop-reentry.md
    ├── portfolio-capacity-bracket*.md
    ├── execution-recovery.md
    ├── fip-breadth-ic.md
    ├── regime-monitor-v3.md / -v4.md
    └── …                # 16 documents total

reports/                 # Committed backtest reports (JSON) + compare_reports.py

deploy/
├── nginx.conf           # Reverse proxy + static file serving
├── setup_db.sh          # Idempotent DB setup script
├── provision_fundamentals.sh   # Server-side Dolt/SEC fundamentals provisioning
└── signalplatform.service      # systemd unit

tests/
├── conftest.py          # Fixtures, strategies, test DB
├── unit/                # Unit tests
└── property/            # Property-based tests (Hypothesis)

Maintainer Guide

Context for whoever — human or AI — continues this work. The owner pushes straight to main on a self-hosted Gitea remote (no PRs); deployment is automated by the Gitea Actions workflow at .gitea/workflows/deploy.yml.

Invariants — do not break these

  • app/services/qualification.py is mirrored in frontend/src/lib/qualification.ts. Any gate change must land in both, or the UI's "qualified" flags silently disagree with the server.
  • Live scan and backtest share the same pure functions. The backtest replays production logic through DB-free functions (compute_technical_from_arrays, compute_momentum_from_closes, detect_sr_levels, detect_gate_target_ladder, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
  • Keep the two price-level models separate. detect_sr_levels produces persisted Structural S/R for charts and alerts. detect_gate_target_ladder produces transient screening proposals and must never be persisted or presented as market structure. The scanner must not read SRLevel rows for target generation.
  • The Gate Target Ladder target is a gate input, never an exit. _atr_trailing_close() does not take it as a parameter, and it must stay that way — take-profit exits were tested and halve CAGR. Any UI or alert that implies the trade exits at the target is a bug (research).
  • The outcome evaluator evaluates ALL setups, not just qualified ones — unqualified setups are the control group that makes the realized-outcome record meaningful.
  • SystemSetting access goes through app/services/settings_store.py — don't query the model directly.
  • Time-series data gets a real table (see benchmark_prices, regime_snapshots); SystemSetting JSON is only for config and cached reports.
  • The shadow book must stay parity-clean. It orders on the stored strategy_rank the scanner wrote and mirrors _simulate_portfolio's selection rule; it accepts only a scan from its own pipeline run. Recomputing its ranking, or letting it consume a stale/manual scan, turns the forward OOS record back into an approximation.
  • Delisted tickers are retired, never deleted. Live paths opt into ticker_service.active_only; the registry, admin views and run_backtest deliberately still see them. Deleting a symbol takes the history that a survivorship-bias fix would need.
  • Discretionary overlay data is forward-only. signal_context_snapshots captures composite/dimension/sentiment/fundamental context for new setups. Do not approximate historical sentiment/fundamental snapshots from today's data.
  • Style: surgical changes, minimal new files; extend existing services rather than adding parallel ones.

Where the strategy lives

Concern File
Composite + 5 dimension scores, weights app/services/scoring_service.py
Residual 12-1 momentum ranking (the validated activation factor) app/services/momentum_service.py
Setup construction (ATR stop, Gate Target Ladder targets) app/services/rr_scanner_service.py
Confidence, targets, reach-probability, action app/services/recommendation_service.py
Activation gate predicate (mirrored in TS) app/services/qualification.py
Gate defaults / admin config app/services/admin_service.py (ACTIVATION_DEFAULTS)
Backtest + factor rank-IC harness ("Signal edge") app/services/backtest_service.py
Outcome resolution (target/stop/expired/ambiguous) app/services/outcome_service.py
Paper trades + time/trailing/target auto-exit app/services/paper_trade_service.py
Shadow book (automated twin of the backtest's selection) app/services/shadow_book_service.py
Re-entry locks / distinct-day guard / book identities app/services/trade_policy.py
Ticker registry, delisting + active_only filter app/services/ticker_service.py
Point-in-time setup context snapshots app/models/signal_context_snapshot.py + app/services/rr_scanner_service.py
Structural S/R detection, Gate Target Ladder & zone clustering app/services/sr_service.py
Research log — what's been tested and rejected docs/research/
SPY benchmark for residual momentum + paper-trade alpha app/services/benchmark_service.py
Pipelines & job registration app/scheduler.py (step lists and job names in app/job_catalog.py)

Verifying changes

pytest tests/ -q                 # backend; in-memory SQLite, no Postgres needed
cd frontend && npm run build     # full tsc check — this IS the frontend "test"
  • There is no npm test in frontend/ — no test script, no test files. npm run build (tsc -b && vite build) is the frontend check.
  • Backend tests that exercise services which commit() need a plain session fixture, not the rolling-back db_session — copy the pattern in tests/unit/test_rr_scanner_integration.py.
  • ruff reports ~11 pre-existing errors in old test files; those are not regressions.

Deploying

Automated by Gitea Actions (.gitea/workflows/deploy.yml) on every push to main: lint (ruff check app/) → test (pytest; alembic upgrade head validated against a real Postgres 16 service; frontend npm ci && npm run build) → deploy (frontend built on the runner, rsync to the server excluding .env, pip install -e ., alembic upgrade head, restart signalplatform.service, health check on 127.0.0.1:8998). Deploys are serialized by a concurrency group so overlapping pushes can't race.

Practical consequences:

  • A ruff error in app/ or any failing backend test blocks the deploy. (CI lints only app/, so the pre-existing ruff noise in old test files doesn't.)
  • Migrations run automatically on deploy — no manual alembic step. A migration that only works on SQLite will fail CI against Postgres, by design.
  • Pushing to main is deploying to production — there is no separate release step.
  • After a gate or scanner change ships, trigger an R:R scan (Admin → Jobs) so live setups pick up new fields.

Roadmap (agreed June 2026)

  1. Forward paper-test the momentum book — the out-of-sample proof the backtest can't give. Watch the Dashboard chart (shadow book vs discretionary vs SPY) against Signals → Backtest.
  2. Full IBKR integration — read real positions, overlay entries/stops on charts, alert on holdings' score deterioration. (Paper trading, the lighter alternative, is done.)
  3. Strategy experiments in the order listed under Strategy Status above — each one goes through the factor harness first.
S
Description
No description provided
Readme
463 MiB
Languages
Python 78.5%
TypeScript 20.8%
CSS 0.4%
Shell 0.3%