dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document Gate Target Ladder architecture

Browse Source
This commit is contained in:
dennisthiessen
2026-07-13 11:47:23 +02:00
parent 3ffd13ce6e
commit dc1570877c
4 changed files with 144 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+69 -21
View File
@@ -6,9 +6,9 @@ Investing-signal platform for US equities. It runs one strategy, and it is a bor
**Philosophy:** don't predict price — rank it. The edge is *relative* strength across the universe, and the discipline is in the exit: cut losers fast, let winners run until the trail catches them.
**What is NOT the edge — read this before trusting a number on screen.** The composite score, the 5 dimensions, sentiment, fundamentals, and the support/resistance engine are **display and screening context**. None has a measured edge. In particular:
**What is NOT the edge — read this before trusting a number on screen.** The composite score, the 5 dimensions, sentiment, fundamentals, and Structural S/R are **display context**, not validated predictors. The Gate Target Ladder is screening machinery that preserves the production setup population; it is not a claim about true market structure. In particular:
- **The S/R "target" is not an exit.** It exists only to compute the R:R and touch-odds that admit a setup through the activation gate. The live exit never reads it. Across 320 backtested production trades the exit reasons were **144 initial stop, 98 trailing stop, 78 max hold — and 0 targets.** Honoring the target as a take-profit was tested and *halves CAGR* ([research](docs/research/sr-levels-and-exits.md)).
- **The headline "target" is not an exit.** It comes from the internal **Gate Target Ladder** and exists only to compute the R:R and reach-probability used by the activation gate. Human-facing chart S/R is a separate model. The live exit reads neither. Across 320 backtested production trades the exit reasons were **144 initial stop, 98 trailing stop, 78 max hold — and 0 targets.** Honoring the target as a take-profit was tested and *halves CAGR* ([research](docs/research/sr-levels-and-exits.md)).
- **The composite score does not select trades.** Residual momentum does.
Full experiment log — everything tested, kept, and rejected: **[docs/research/](docs/research/README.md)**.
@@ -21,10 +21,10 @@ flowchart TD
M --> R["Rank cross-sectionally<br/>into percentiles"]
R --> G1{"Top 20%?<br/>percentile ≥ 80"}
G1 -->|no| SKIP["Not traded<br/><i>(still scored — the control group)</i>"]
G1 -->|yes| S["Build the setup<br/>entry = last close<br/><b>stop = entry − 1.5 × ATR</b><br/>level = nearest S/R above"]
G1 -->|yes| S["Build the setup<br/>entry = last close<br/><b>stop = entry − 1.5 × ATR</b><br/>primary = Gate Target Ladder proposal"]
S --> G2{"Activation gate"}
G2 --> G2a["R:R ≥ 2.0 <i>(to the S/R level)</i>"]
G2 --> G2a["R:R ≥ 2.0 <i>(to the primary gate target)</i>"]
G2 --> G2b["touch odds ≥ 20%"]
G2 --> G2c["action not NEUTRAL<br/>and matches direction"]
G2a & G2b & G2c --> Q{"qualified?"}
@@ -39,7 +39,7 @@ flowchart TD
EXIT --> E1["Initial stop hit<br/>entry − 1.5 × ATR → −1R<br/><b>45% of trades</b>"]
EXIT --> E2["Trailing stop hit<br/>highest close − 3 × ATR<br/><i>only binds once price is ~1R up</i><br/><b>31% of trades</b>"]
EXIT --> E3["Max hold reached<br/>30 trading days<br/><b>24% of trades</b>"]
EXIT -.->|"NEVER"| E4["S/R target<br/><b>0% of trades</b>"]
EXIT -.->|"NEVER"| E4["Gate Target Ladder target<br/><b>0% of trades</b>"]
style M fill:#1e3a5f,color:#fff
style OPEN fill:#1e4d2b,color:#fff
@@ -54,13 +54,59 @@ flowchart TD
Scheduled pipelines turn raw prices into a ranked, gated list of tradeable setups. Everything downstream of OHLCV is recomputed from stored data, so each refresh is cheap and idempotent. Job timing is cron-based and configurable in **Admin → Jobs** (default timezone Europe/Berlin).
### Price-level architecture: two different jobs
The platform deliberately has two price-level components. Calling both of them
"S/R" hid an important distinction, so the internal screening component is now
named the **Gate Target Ladder (GTL)**.
| Component | Purpose | Lifetime | Consumed by |
|---|---|---|---|
| **Structural S/R** | A small set of meaningful support/resistance zones for humans | Persisted as `SRLevel` | Charts and alerts |
| **Gate Target Ladder** | A broad set of price proposals that preserves the validated setup screen | Built transiently per scan; never persisted as S/R | Target table, headline R:R and activation gate |
```mermaid
flowchart TD
O["Ticker OHLCV history"] --> SR["Structural S/R detector<br/>volume peaks + prominent pivots + round numbers<br/>rejection and recency strength"]
SR --> DB[("Persisted SRLevel rows")]
DB --> UI["Charts and alerts"]
O --> GTL["Gate Target Ladder<br/>20 price-range centers + 5-bar pivots<br/>no volume calculation"]
GTL --> TRAFFIC["Score historical price traffic<br/>merge nearby proposals and tag side"]
TRAFFIC --> HAS{"Any directional proposal<br/>with R:R ≥ 1.5?"}
HAS -->|no| NONE["No setup for that direction"]
HAS -->|yes| TARGETS["Build up to 5 target candidates<br/>estimate reach-probability"]
TARGETS --> PRIMARY["Headline target<br/>most likely candidate clearing<br/>R:R ≥ 1.5 and probability ≥ 20%"]
PRIMARY --> GATE{"Live activation gate<br/>headline R:R ≥ 2.0<br/>probability ≥ 20%<br/>momentum and direction pass?"}
GATE -->|no| OBS["Keep as unqualified observation"]
GATE -->|yes| BOOK["Eligible for production ranking/book"]
BOOK --> EXIT["Exit only by ATR stop/trail<br/>or max hold — never by target"]
```
The Gate Target Ladder works step by step:
1. Build 20 evenly spaced centers over the ticker's observed high/low range and add unfiltered five-bar swing pivots. Volume is not used.
2. Count how often historical bars pass through each proposal, convert that traffic to strength, merge proposals within 0.5%, and label them above/below spot.
3. For each direction, require at least one proposal with scanner R:R ≥ 1.5 against the 1.5× ATR initial stop.
4. Collapse nearby proposals into target zones, discard unsuitable ATR distances, and retain up to five candidates spanning near to far.
5. Estimate each candidate's probability of reaching the target before the stop. The headline target is the most likely candidate with R:R ≥ 1.5 and probability ≥ 20%; if none clears both, the most likely candidate remains headline so a distant lottery target cannot game the gate.
6. Apply the separate live activation floor to that headline target: production requires R:R ≥ 2.0 and probability ≥ 20%, plus the momentum/direction rules.
7. If traded, ignore the target for exits. The initial ATR stop, 3× ATR trail and maximum hold remain authoritative.
The ladder is intentionally broad and mechanical. It is not presented as
market structure, and its transient negative level IDs must never be stored as
chart S/R. The full-period parity run reproduced all 202,765 backtest setup
candidates, all 1,086 qualified setups, and the production book exactly
(Sharpe 2.03, CAGR 50.0%, max drawdown 21.4%, 321 trades). See the
[S/R and Gate Target Ladder research](docs/research/sr-levels-and-exits.md#explicit-gate-target-ladder).
### Daily Load — the full refresh
Once a day (default 07:00). Steps run **in dependency order**, each consuming the previous step's fresh output:
1. **OHLCV** — fetch the latest daily bars for every tracked ticker (Alpaca); new tickers backfill ~5 years.
2. **Sentiment** — fetch sentiment for the names that matter and are stale (> 5 days): top-pick feeders (residual-momentum leaders with a tradeable long setup), the watchlist, and open paper trades, plus a top-N-by-composite discovery net. Runs *before* the scan so the scan sees fresh sentiment.
3. **R:R Scan** — recompute S/R zones, the 5-dimension scores and long/short setups (ATR stops, S/R targets) for every ticker, and attach each ticker's residual 12‑1 momentum activation percentile plus the promoted 80/20 production rank.
3. **R:R Scan** — persist clean Structural S/R for charts/alerts, recompute the 5-dimension scores, and build long/short setups from a transient Gate Target Ladder (ATR stops and nominal gate targets) for every ticker. Attach each ticker's residual 12‑1 momentum activation percentile plus the promoted 80/20 production rank.
4. **Outcome Eval** — resolve setups that hit target/stop or expired (default 30 trading days) and auto-close paper trades per the exit policy (default: 3x ATR trail with a 30-trading-day max hold).
5. **Market Regime** — recompute the regime index (breadth/trend).
6. **Regime Monitor** — observational early-warning snapshot (VIX, credit spreads via FRED); feeds nothing else.
@@ -78,11 +124,11 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we
### From score to "top pick"
1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize). **Display and ranking only — it does not select trades.**
2. **Setups** — the scanner builds long/short setups with a 1.5× ATR stop and picks the nearest S/R level as a nominal target, then adds a confidence score, conflict flags and a per-level touch-probability.
3. **Activation gate** — a setup *qualifies* only if it ranks in the top residual-momentum percentile of the universe (**the actual selection**, long-only), clears the R:R floor, **and** its primary level carries at least a 20% touch-probability. The confidence floor was ablated to zero effect and defaults off.
2. **Setups** — the scanner builds long/short setups with a 1.5× ATR stop, generates up to five candidates from the transient Gate Target Ladder, and makes the most likely worthwhile candidate the headline target. It then adds confidence and conflict context plus a per-target reach-probability.
3. **Activation gate** — a setup *qualifies* only if it ranks in the top residual-momentum percentile of the universe (**the actual selection**, long-only), its headline target clears the live R:R floor, **and** that target carries at least a 20% reach-probability. The confidence floor was ablated to zero effect and defaults off.
4. **Top pick** — qualified setups are ordered by the production rank: 80% residual momentum percentile + 20% 6-month realized-volatility percentile. The #1 is highlighted on the Dashboard and labelled on the ticker page.
**What the R:R and touch-probability in step 3 actually are.** They are *gate inputs*, computed from an S/R level the trade will never exit at — they exist to filter setups, not to forecast the trade you're about to take. A setup with "R:R 2.4:1, 34% touch odds" is not a claim that you'll make 2.4R with 34% probability; it's a claim that this setup cleared the screen. What actually happens to a trade is in the exit box of the diagram above, and on the "what usually happens" panel in the UI. Conflating the two is the single easiest way to misread this app.
**What the R:R and reach-probability in step 3 actually are.** They are *gate inputs*, computed from a Gate Target Ladder proposal the trade will never exit at — they exist to filter setups, not to forecast the trade you're about to take. A setup with "R:R 2.4:1, 34% reach probability" is not a claim that you'll make 2.4R with 34% probability; it's a claim that this setup cleared the screen. What actually happens to a trade is in the exit box of the diagram above, and on the "what usually happens" panel in the UI. Conflating the two is the single easiest way to misread this app.
## Strategy Status — What's Validated and What Isn't
@@ -94,13 +140,14 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we
|---|---|---|
| **Residual 12-1 cross-sectional momentum** (the activation gate, long-only) | **Production gate — in-sample edge** | Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable |
| **3× ATR trailing exit** (+ 1.5× ATR initial stop, 30-day max hold) | **Production exit — best Sharpe of every exit tested** | Beat hold / SMA50 / 20-day-low / technical-40 and both take-profit variants (July 2026) |
| S/R setup engine (ATR stops, S/R levels, touch-probability) | **Gate input only — NOT an exit, and not an edge** | The exit never reads the target (0 of 320 trades). Honoring it as a take-profit drops Sharpe 2.04 → 1.47. The R:R floor does real work *as a filter*; the detector itself is methodologically weak. [Full write-up](docs/research/sr-levels-and-exits.md) |
| **Structural S/R** | **Human-facing context only — not a gate and not an exit** | Clean, capped zones are persisted for charts and alerts. The scanner deliberately does not read them. |
| **Gate Target Ladder** | **Gate input only — not market structure and not an exit** | Volume-free range grid + pivots preserves the useful legacy screening behavior exactly: 1,086/1,086 qualified setups retained and identical Sharpe 2.03 / CAGR 50.0% / DD 21.4% / 321 trades. The exit never reads its target. [Full write-up](docs/research/sr-levels-and-exits.md#explicit-gate-target-ladder) |
| Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) |
| LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet |
| Fundamentals | Feeds composite + confidence only | Latest values only, no history — same limitation |
| Short setups | **Excluded while the momentum gate is active** | Backtest showed shorts fight the trend and drag expectancy |
| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups. Orphaned settings dropped in migration 020 |
| S/R target as a take-profit (tested July 2026) | **Rejected** | Sharpe 2.04 → 1.47, CAGR halved. Win rate *rose* — it truncates the right tail where the edge lives |
| Gate target as a take-profit (tested July 2026) | **Rejected** | Sharpe 2.04 → 1.47, CAGR halved. Win rate *rose* — it truncates the right tail where the edge lives |
| "Clear-air" gate relaxation (tested July 2026) | **Rejected — failed out-of-sample** | Strictly better in-sample (Sharpe 2.07 / CAGR 62.3% / DD 20.1%), then lost on a real train/test split (Sharpe 2.78 → 2.45). A cautionary tale: nested lookbacks are not OOS |
Caveats on the momentum result: in-sample, roughly one market regime, costs/slippage approximated at 0.1% per side, and residual momentum still needs SPY benchmark history to compute. The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.
@@ -112,7 +159,7 @@ Use this as a regression guardrail for future strategy changes, not as a return
| Item | Current baseline |
|---|---|
| Strategy version | `residual_highvol_80_20_atr_trail3_v1` |
| Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R >= 2.0 (live `activation_min_rr`; the code default is 1.2), primary-level touch-probability >= 20%, NEUTRAL excluded, confidence floor off (0) |
| Production gate | Long-only, residual 12-1 momentum percentile >= 80, headline gate-target R:R >= 2.0 (live `activation_min_rr`; the code default is 1.2), primary-target reach-probability >= 20%, NEUTRAL excluded, confidence floor off (0) |
| Production rank | 80% residual momentum percentile + 20% 6-month realized-volatility percentile |
| Exit | Initial ATR stop plus 3x ATR trailing stop, max 30 trading days |
| Portfolio CAGR | +50.4% |
@@ -183,7 +230,7 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
## Key Use Cases
- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists residual-gated qualified setups ranked by the production 80/20 residual/high-vol score, with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop.
- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists residual-gated qualified setups ranked by the production 80/20 residual/high-vol score, with the #1 flagged "Top pick". Each row opens the ticker page for its chart, Structural S/R, Gate Target Ladder targets and entry/stop.
- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed by the active exit policy (default: 3x ATR trail with a 30-trading-day max hold), and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge.
## Stack
@@ -208,12 +255,13 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
- Universe bootstrap for `sp500`, `nasdaq100`, `nasdaq_all` via admin endpoint
- OHLCV price storage with upsert and validation
- Technical indicators: ADX, EMA, RSI, ATR, Volume Profile, Pivot Points, EMA Cross
- Support/Resistance detection with strength scoring and merge-within-tolerance
- Structural Support/Resistance detection with rejection/recency strength, ATR-adaptive merging and a hard cap; persisted for charts and alerts
- Transient Gate Target Ladder — volume-free range grid plus pivots, used only for nominal targets, reach-probability and gate R:R
- Sentiment analysis with time-decay weighted scoring
- Fundamental data tracking (P/E, revenue growth, earnings surprise, market cap)
- 5-dimension scoring engine (technical, S/R quality, sentiment, fundamental, momentum) with configurable weights
- Risk:Reward scanner — long and short setups, 1.5x ATR stops, S/R-based nominal targets, configurable scan R:R threshold (default 1.5:1 — distinct from the activation floor below)
- Activation gate — qualifies setups on a residual-momentum percentile floor (the actual selection), an R:R floor (prod: 2.0) and a 20% primary-level touch-probability floor (validated long-only edge)
- Risk:Reward scanner — long and short setups, 1.5x ATR stops, Gate Target Ladder nominal targets, configurable scan R:R threshold (default 1.5:1 — distinct from the activation floor below)
- Activation gate — qualifies setups on a residual-momentum percentile floor (the actual selection), a headline gate-target R:R floor (prod: 2.0) and a 20% primary-target reach-probability floor (validated long-only edge)
- Recommendation layer — directional confidence, conflict detection, per-target reach-probability
- Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: 3x ATR trail with a 30-trading-day max hold; time / percent-trailing / target-stop selectable), realized track record + outcome evaluation
- Market-regime index + FRED early-warning monitor (VIX, credit spreads); weekly backtest + manual event study
@@ -620,9 +668,9 @@ Context for whoever — human or AI — continues this work. The owner pushes st
### Invariants — do not break these
- **`app/services/qualification.py` is mirrored in `frontend/src/lib/qualification.ts`.** Any gate change must land in both, or the UI's "qualified" flags silently disagree with the server.
- **Live scan and backtest share the same pure functions.** The backtest replays production logic through DB-free functions (`compute_technical_from_arrays`, `compute_momentum_from_closes`, `detect_sr_levels`, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
- **One S/R model app-wide:** `sr_service.detect_sr_levels` + `cluster_sr_zones` (2% tolerance) feed the chart, alerts, and target generation identically.
- **The S/R target is a gate input, never an exit.** `_atr_trailing_close()` does not take it as a parameter, and it must stay that way — take-profit exits were tested and halve CAGR. Any UI or alert that implies the trade exits at the target is a bug ([research](docs/research/sr-levels-and-exits.md)).
- **Live scan and backtest share the same pure functions.** The backtest replays production logic through DB-free functions (`compute_technical_from_arrays`, `compute_momentum_from_closes`, `detect_sr_levels`, `detect_gate_target_ladder`, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
- **Keep the two price-level models separate.** `detect_sr_levels` produces persisted Structural S/R for charts and alerts. `detect_gate_target_ladder` produces transient screening proposals and must never be persisted or presented as market structure. The scanner must not read `SRLevel` rows for target generation.
- **The Gate Target Ladder target is a gate input, never an exit.** `_atr_trailing_close()` does not take it as a parameter, and it must stay that way — take-profit exits were tested and halve CAGR. Any UI or alert that implies the trade exits at the target is a bug ([research](docs/research/sr-levels-and-exits.md#explicit-gate-target-ladder)).
- **The outcome evaluator evaluates ALL setups**, not just qualified ones — unqualified setups are the control group that makes the Track Record meaningful.
- **`SystemSetting` access goes through `app/services/settings_store.py`** — don't query the model directly.
- **Time-series data gets a real table** (see `benchmark_prices`, `regime_snapshots`); `SystemSetting` JSON is only for config and cached reports.
@@ -635,7 +683,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
|---|---|
| Composite + 5 dimension scores, weights | `app/services/scoring_service.py` |
| Residual 12-1 momentum ranking (the validated activation factor) | `app/services/momentum_service.py` |
| Setup construction (ATR stop, S/R targets) | `app/services/rr_scanner_service.py` |
| Setup construction (ATR stop, Gate Target Ladder targets) | `app/services/rr_scanner_service.py` |
| Confidence, targets, reach-probability, action | `app/services/recommendation_service.py` |
| Activation gate predicate (mirrored in TS) | `app/services/qualification.py` |
| Gate defaults / admin config | `app/services/admin_service.py` (`ACTIVATION_DEFAULTS`) |
@@ -643,7 +691,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
| Outcome resolution (target/stop/expired/ambiguous) | `app/services/outcome_service.py` |
| Paper trades + time/trailing/target auto-exit | `app/services/paper_trade_service.py` |
| Point-in-time setup context snapshots | `app/models/signal_context_snapshot.py` + `app/services/rr_scanner_service.py` |
| S/R detection & zone clustering | `app/services/sr_service.py` |
| Structural S/R detection, Gate Target Ladder & zone clustering | `app/services/sr_service.py` |
| **Research log — what's been tested and rejected** | **`docs/research/`** |
| SPY benchmark for residual momentum + paper-trade alpha | `app/services/benchmark_service.py` |
| Pipelines & job registration | `app/scheduler.py` |