Docs: state the actual strategy, add an end-to-end diagram, add a research log

The README opened with "find the path of least resistance, key S/R zones, and asymmetric R:R setups" — a description of a strategy we do not run. What we run is a long-only cross-sectional momentum book with a trailing exit. The S/R engine, the composite score, sentiment and fundamentals are screening and display; none has a measured edge. - Rewrites the intro/philosophy around the real strategy, and says plainly what is NOT the edge. - Adds a mermaid decision graph, universe -> qualified -> ranked -> opened -> closed, with the real exit distribution on the terminal nodes: initial stop 45%, trailing stop 31%, max hold 24%, S/R target 0%. Validated against the mermaid parser, not eyeballed. - Documents that the R:R and touch-probability are GATE INPUTS, not forecasts of the trade — the single easiest way to misread this app. - Adds win rate, best/worst R and the exit-reason split to the production baseline table. - New docs/research/README.md: every strategy tested, the result, the decision, and why we stay with the current one. 12 rejected ideas (take-profit exits, clear-air gate relaxation, EV gate, regime overlay, inverse-vol sizing, shorts, standalone vol, FIP, ...), the confirmed tuning knobs, the open leads, and the method rules we learned the hard way (nested lookbacks are not out-of-sample; a rising win rate is a warning, not a win). - Documents the research flags and the holdout harness, and warns that the portfolio_monitor lookbacks are nested windows, NOT a holdout. - Notes the snapshot must copy paper_% settings or it silently diverges from prod. All baseline numbers re-verified against reports/backtest-20260711-prod-baseline.json (506 tickers, 1,089 qualified, CAGR 50.4%, +413.8% vs SPY +95.7%, DD -21.4%, Sharpe 2.04, 320 trades, 15.3d avg hold, and all five promotion contenders). No corrections were needed — the numbers were right, the framing was not. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 15:40:34 +02:00
parent 9789e3d762
commit 906d1db7d1
2 changed files with 214 additions and 11 deletions
@@ -1,8 +1,54 @@
 # Signal Dashboard

-Investing-signal platform for NASDAQ stocks. Surfaces the best trading opportunities through weighted multi-dimensional scoring — technical indicators, support/resistance quality, sentiment, fundamentals, and momentum — with asymmetric risk:reward scanning.
+Investing-signal platform for US equities. It runs one strategy, and it is a boring one:

-**Philosophy:** Don't predict price. Find the path of least resistance, key S/R zones, and asymmetric R:R setups.
+> **A long-only cross-sectional momentum book.** Buy the top quintile by beta-adjusted 12-1 month momentum, tilt toward higher volatility, hold at most 10 names, cut at 1.5× ATR, then trail at 3× ATR for up to 30 trading days.
+
+**Philosophy:** don't predict price — rank it. The edge is *relative* strength across the universe, and the discipline is in the exit: cut losers fast, let winners run until the trail catches them.
+
+**What is NOT the edge — read this before trusting a number on screen.** The composite score, the 5 dimensions, sentiment, fundamentals, and the support/resistance engine are **display and screening context**. None has a measured edge. In particular:
+
+- **The S/R "target" is not an exit.** It exists only to compute the R:R and touch-odds that admit a setup through the activation gate. The live exit never reads it. Across 320 backtested production trades the exit reasons were **144 initial stop, 98 trailing stop, 78 max hold — and 0 targets.** Honoring the target as a take-profit was tested and *halves CAGR* ([research](docs/research/sr-levels-and-exits.md)).
+- **The composite score does not select trades.** Residual momentum does.
+
+Full experiment log — everything tested, kept, and rejected: **[docs/research/](docs/research/README.md)**.
+
+## The strategy, end to end
+
+```mermaid
+flowchart TD
+    U["Universe — ~500 tickers<br/>daily OHLCV"] --> M["Residual 12-1 momentum<br/><i>12-month return, skip last month,<br/>beta-adjusted vs SPY</i>"]
+    M --> R["Rank cross-sectionally<br/>into percentiles"]
+    R --> G1{"Top 20%?<br/>percentile ≥ 80"}
+    G1 -->|no| SKIP["Not traded<br/><i>(still scored — the control group)</i>"]
+    G1 -->|yes| S["Build the setup<br/>entry = last close<br/><b>stop = entry − 1.5 × ATR</b><br/>level = nearest S/R above"]
+
+    S --> G2{"Activation gate"}
+    G2 --> G2a["R:R ≥ 2.0 <i>(to the S/R level)</i>"]
+    G2 --> G2b["touch odds ≥ 20%"]
+    G2 --> G2c["action not NEUTRAL<br/>and matches direction"]
+    G2a & G2b & G2c --> Q{"qualified?"}
+    Q -->|no| SKIP
+    Q -->|yes| RANK["Rank by production score<br/>80% momentum %ile<br/>+ 20% volatility %ile"]
+
+    RANK --> BOOK{"Room in the book?<br/>max 10 positions"}
+    BOOK -->|no| WAIT["Wait for a slot"]
+    BOOK -->|yes| OPEN["OPEN — size at 1% account risk"]
+
+    OPEN --> EXIT{"Exit — whichever comes first"}
+    EXIT --> E1["Initial stop hit<br/>entry − 1.5 × ATR → −1R<br/><b>45% of trades</b>"]
+    EXIT --> E2["Trailing stop hit<br/>highest close − 3 × ATR<br/><i>only binds once price is ~1R up</i><br/><b>31% of trades</b>"]
+    EXIT --> E3["Max hold reached<br/>30 trading days<br/><b>24% of trades</b>"]
+    EXIT -.->|"NEVER"| E4["S/R target<br/><b>0% of trades</b>"]
+
+    style M fill:#1e3a5f,color:#fff
+    style OPEN fill:#1e4d2b,color:#fff
+    style E4 fill:#2a2a2a,color:#888
+    style E1 fill:#4a1f1f,color:#fff
+    style E2 fill:#1e4d2b,color:#fff
+```
+
+**How to read the exit box.** The initial stop is tight (1.5× ATR) and the trail is wide (3× ATR), so the trail sits *below* the initial stop at entry and only takes over once price has advanced roughly 1R. Cut fast when wrong; give room once right. That asymmetry is what produces the right-tailed return profile the strategy depends on — most trades lose a little (win rate ~37.5%), a few win big (best trade +12.9R), and *that is why there is no take-profit*.

 ## How It Works

@@ -31,24 +77,31 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we

 ### From score to "top pick"

-1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize).
-2. **Setups** — the scanner builds long/short setups with ATR stops and S/R targets, then adds a confidence score, conflict flags and a target reach-probability.
-3. **Activation gate** — a setup *qualifies* only if it clears the R:R floor, ranks in the top residual-momentum percentile of the universe, **and** its primary target carries at least a 20% reach-probability (the validated edge is long-only; the confidence floor was ablated to zero effect and defaults off).
+1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize). **Display and ranking only — it does not select trades.**
+2. **Setups** — the scanner builds long/short setups with a 1.5× ATR stop and picks the nearest S/R level as a nominal target, then adds a confidence score, conflict flags and a per-level touch-probability.
+3. **Activation gate** — a setup *qualifies* only if it ranks in the top residual-momentum percentile of the universe (**the actual selection**, long-only), clears the R:R floor, **and** its primary level carries at least a 20% touch-probability. The confidence floor was ablated to zero effect and defaults off.
 4. **Top pick** — qualified setups are ordered by the production rank: 80% residual momentum percentile + 20% 6-month realized-volatility percentile. The #1 is highlighted on the Dashboard and labelled on the ticker page.

+**What the R:R and touch-probability in step 3 actually are.** They are *gate inputs*, computed from an S/R level the trade will never exit at — they exist to filter setups, not to forecast the trade you're about to take. A setup with "R:R 2.4:1, 34% touch odds" is not a claim that you'll make 2.4R with 34% probability; it's a claim that this setup cleared the screen. What actually happens to a trade is in the exit box of the diagram above, and on the "what usually happens" panel in the UI. Conflating the two is the single easiest way to misread this app.
+
 ## Strategy Status — What's Validated and What Isn't

 **Read this before touching scoring, gating, or setup logic.** The platform measures itself — a weekly-replay backtest plus a factor rank-IC harness (`app/services/backtest_service.py`) — and the verdicts below come from those reports (latest run July 2026, ~5 years of OHLCV), not from opinion.

+> **The full experiment log lives in [docs/research/](docs/research/README.md)** — every strategy we've tested, the result, and the decision. Check it before proposing an idea; most of the obvious ones have already been run and rejected.
+
 | Component | Verdict | Evidence |
 |---|---|---|
 | **Residual 12-1 cross-sectional momentum** (the activation gate, long-only) | **Production gate — in-sample edge** | Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable |
-| S/R setup engine (ATR stops, S/R targets, reach-probability) | **Filter/execution context, not the exit** | R:R/room-to-run still earns its keep as a filter, but S/R targets underperform the time exit. The probability model is display-only |
+| **3× ATR trailing exit** (+ 1.5× ATR initial stop, 30-day max hold) | **Production exit — best Sharpe of every exit tested** | Beat hold / SMA50 / 20-day-low / technical-40 and both take-profit variants (July 2026) |
+| S/R setup engine (ATR stops, S/R levels, touch-probability) | **Gate input only — NOT an exit, and not an edge** | The exit never reads the target (0 of 320 trades). Honoring it as a take-profit drops Sharpe 2.04 → 1.47. The R:R floor does real work *as a filter*; the detector itself is methodologically weak. [Full write-up](docs/research/sr-levels-and-exits.md) |
 | Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) |
 | LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet |
 | Fundamentals | Feeds composite + confidence only | Latest values only, no history — same limitation |
 | Short setups | **Excluded while the momentum gate is active** | Backtest showed shorts fight the trend and drag expectancy |
-| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups |
+| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups. Orphaned settings dropped in migration 020 |
+| S/R target as a take-profit (tested July 2026) | **Rejected** | Sharpe 2.04 → 1.47, CAGR halved. Win rate *rose* — it truncates the right tail where the edge lives |
+| "Clear-air" gate relaxation (tested July 2026) | **Rejected — failed out-of-sample** | Strictly better in-sample (Sharpe 2.07 / CAGR 62.3% / DD 20.1%), then lost on a real train/test split (Sharpe 2.78 → 2.45). A cautionary tale: nested lookbacks are not OOS |

 Caveats on the momentum result: in-sample, roughly one market regime, costs/slippage approximated at 0.1% per side, and residual momentum still needs SPY benchmark history to compute. The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.

@@ -59,7 +112,7 @@ Use this as a regression guardrail for future strategy changes, not as a return
 | Item | Current baseline |
 |---|---|
 | Strategy version | `residual_highvol_80_20_atr_trail3_v1` |
-| Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R floor on, primary-target reach-probability >= 20%, NEUTRAL excluded, confidence floor effectively off |
+| Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R >= 2.0 (live `activation_min_rr`; the code default is 1.2), primary-level touch-probability >= 20%, NEUTRAL excluded, confidence floor off (0) |
 | Production rank | 80% residual momentum percentile + 20% 6-month realized-volatility percentile |
 | Exit | Initial ATR stop plus 3x ATR trailing stop, max 30 trading days |
 | Portfolio CAGR | +50.4% |
@@ -67,7 +120,12 @@ Use this as a regression guardrail for future strategy changes, not as a return
 | Max drawdown | -21.4% |
 | Sharpe | 2.04 daily, annualized |
 | Trades | 320 |
+| Win rate | 37.5% |
 | Average hold | 15.3 trading days |
+| Best / worst trade | +12.9R / -3.3R |
+| **How trades actually ended** | **initial stop 144 (45%) · trailing stop 98 (31%) · max hold 78 (24%) · target 0 (0%)** |
+
+That last row is the strategy in one line: a 37.5% win rate is *fine* because the +12.9R tail pays for every −1R stop. It is also why no take-profit exists — and why the S/R "target" shown in the UI is a screening artifact, not a plan.

 Promotion evidence from the same snapshot:

@@ -117,6 +175,8 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m

 ### Highest-value next experiments (in order)

+> Check **[docs/research/](docs/research/README.md)** first — 12 strategy ideas have already been tested and rejected, including the obvious ones (take-profit exits, regime overlays, inverse-vol sizing, shorts).
+
 1. **Forward monitor the promoted strategy** — the production UI now behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison. Forward paper-trade months are the only evidence the snapshot cannot provide; the July 2026 tuning pass closed every in-sample lead. (Trailing-stop sensitivity and the max-15 capacity check are done — see the tuning table above.)
 2. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only.
 3. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. Now doubly motivated: it is also where the strong `fip_id` signal (see tuning findings) could become tradeable. (Deeper history was considered and declined.)
@@ -152,8 +212,8 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
 - Sentiment analysis with time-decay weighted scoring
 - Fundamental data tracking (P/E, revenue growth, earnings surprise, market cap)
 - 5-dimension scoring engine (technical, S/R quality, sentiment, fundamental, momentum) with configurable weights
- Risk:Reward scanner — long and short setups, ATR-based stops, S/R-based targets, configurable R:R threshold (default 1.5:1)
- Activation gate — qualifies setups on a residual-momentum percentile floor, an R:R floor and a 20% primary-target reach-probability floor (validated long-only edge)
+- Risk:Reward scanner — long and short setups, 1.5x ATR stops, S/R-based nominal targets, configurable scan R:R threshold (default 1.5:1 — distinct from the activation floor below)
+- Activation gate — qualifies setups on a residual-momentum percentile floor (the actual selection), an R:R floor (prod: 2.0) and a 20% primary-level touch-probability floor (validated long-only edge)
 - Recommendation layer — directional confidence, conflict detection, per-target reach-probability
 - Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: 3x ATR trail with a 30-trading-day max hold; time / percent-trailing / target-stop selectable), realized track record + outcome evaluation
 - Market-regime index + FRED early-warning monitor (VIX, credit spreads); weekly backtest + manual event study
@@ -284,7 +344,13 @@ npm run build
 For research loops, run the production backtest locally from a SQLite snapshot
 instead of deploying and clicking the Admin job. The snapshot contains only the
 tables needed by `run_backtest`: tickers, OHLCV bars, SPY benchmark closes, and
-activation/recommendation settings. Secrets and cached reports are not copied.
+the activation / recommendation / paper-exit settings. Secrets and cached reports
+are not copied.
+
+> The `paper_%` settings **must** be copied: the portfolio monitor's Production row
+> replays the *runtime* exit policy via `get_exit_policy()`. Without them a snapshot
+> silently falls back to the code defaults, so a live-tuned exit would not be
+> reflected and the local run would disagree with prod for no visible reason.

 1. Open an SSH tunnel to the production Postgres instance:

@@ -344,8 +410,29 @@ matching decision. Every change still goes through the factor harness first (see
 | `time_exit_sweep` | Net avg R / net R-per-day by hold length | Whether a fixed time exit beats the promoted ATR trail |
 | `portfolio_monitor`, `portfolio_sim`, `strategy_variants` | CAGR, Sharpe, max drawdown, per-year returns | Promote a strategy only if it beats the current baseline on CAGR/Sharpe/DD |
 | `signal_eval` | Mean IC, t-stat, IC>0 %, `reliable` | Iron rule: wire a new factor in only if \|IC\| ≳ 0.03 with a consistent sign and `reliable: true` |
+| `holdout` (opt-in) | Train vs test books, split by entry date | **The only honest OOS read.** Set `BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD` |
 | `recommendation`, `research_recommendation` | The report's own headline read | A starting point, not a substitute for the sections above |

+**Out-of-sample validation.** The `portfolio_monitor` lookbacks (6m / 1y / 3y / 5y / all) are
+**nested windows that all end today** — every one of them overlaps the data an idea was found
+on, so none of them is a holdout. A rule that looks good across all five can still be an
+in-sample artifact (this exact trap ate the clear-air experiment; see the research log). For a
+real train/test split by entry date:
+
+```bash
+BACKTEST_HOLDOUT_SPLIT=2024-07-01 python scripts/run_backtest_snapshot.py \
+    backtest_snapshots/prod.sqlite --workers 7 --allow-spawn
+```
+
+Research-only flags, all off by default (the default report is byte-identical to the shipped baseline):
+
+| Flag | What it does |
+|---|---|
+| `BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD` | Adds a `holdout` section: train (entries before) vs test (entries on/after), as disjoint books |
+| `BACKTEST_RESEARCH_EXITS=1` | Adds the rejected take-profit exit rows to the exit comparison |
+| `BACKTEST_ATR_TARGET_FALLBACK=k` | Synthesizes a k×ATR target where S/R offers none |
+| `BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1` | Restricts that fallback to setups with genuinely no structure ahead |
+
 `recommendation` is the one section surfaced on the deployed page ("What this
 backtest recommends"); everything else in this table is intentionally local-only.

@@ -503,6 +590,13 @@ frontend/
    ├── stores/          # Zustand auth store
    └── styles/          # Global CSS with glassmorphism classes

+docs/
+└── research/            # Experiment log: what was tested, the result, the decision
+    ├── README.md        # Overview — start here before proposing a strategy change
+    └── sr-levels-and-exits.md
+
+reports/                 # Committed backtest reports (JSON) + compare_reports.py
+
 deploy/
 ├── nginx.conf           # Reverse proxy + static file serving
 ├── setup_db.sh          # Idempotent DB setup script
@@ -523,6 +617,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
 - **`app/services/qualification.py` is mirrored in `frontend/src/lib/qualification.ts`.** Any gate change must land in both, or the UI's "qualified" flags silently disagree with the server.
 - **Live scan and backtest share the same pure functions.** The backtest replays production logic through DB-free functions (`compute_technical_from_arrays`, `compute_momentum_from_closes`, `detect_sr_levels`, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
 - **One S/R model app-wide:** `sr_service.detect_sr_levels` + `cluster_sr_zones` (2% tolerance) feed the chart, alerts, and target generation identically.
+- **The S/R target is a gate input, never an exit.** `_atr_trailing_close()` does not take it as a parameter, and it must stay that way — take-profit exits were tested and halve CAGR. Any UI or alert that implies the trade exits at the target is a bug ([research](docs/research/sr-levels-and-exits.md)).
 - **The outcome evaluator evaluates ALL setups**, not just qualified ones — unqualified setups are the control group that makes the Track Record meaningful.
 - **`SystemSetting` access goes through `app/services/settings_store.py`** — don't query the model directly.
 - **Time-series data gets a real table** (see `benchmark_prices`, `regime_snapshots`); `SystemSetting` JSON is only for config and cached reports.
@@ -544,6 +639,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
 | Paper trades + time/trailing/target auto-exit | `app/services/paper_trade_service.py` |
 | Point-in-time setup context snapshots | `app/models/signal_context_snapshot.py` + `app/services/rr_scanner_service.py` |
 | S/R detection & zone clustering | `app/services/sr_service.py` |
+| **Research log — what's been tested and rejected** | **`docs/research/`** |
 | SPY benchmark for residual momentum + paper-trade alpha | `app/services/benchmark_service.py` |
 | Pipelines & job registration | `app/scheduler.py` |