dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: state the actual strategy, add an end-to-end diagram, add a research log

Browse Source 
The README opened with "find the path of least resistance, key S/R zones, and
asymmetric R:R setups" — a description of a strategy we do not run. What we run
is a long-only cross-sectional momentum book with a trailing exit. The S/R
engine, the composite score, sentiment and fundamentals are screening and
display; none has a measured edge.

- Rewrites the intro/philosophy around the real strategy, and says plainly what
  is NOT the edge.
- Adds a mermaid decision graph, universe -> qualified -> ranked -> opened ->
  closed, with the real exit distribution on the terminal nodes: initial stop 45%,
  trailing stop 31%, max hold 24%, S/R target 0%. Validated against the mermaid
  parser, not eyeballed.
- Documents that the R:R and touch-probability are GATE INPUTS, not forecasts of
  the trade — the single easiest way to misread this app.
- Adds win rate, best/worst R and the exit-reason split to the production
  baseline table.
- New docs/research/README.md: every strategy tested, the result, the decision,
  and why we stay with the current one. 12 rejected ideas (take-profit exits,
  clear-air gate relaxation, EV gate, regime overlay, inverse-vol sizing, shorts,
  standalone vol, FIP, ...), the confirmed tuning knobs, the open leads, and the
  method rules we learned the hard way (nested lookbacks are not out-of-sample; a
  rising win rate is a warning, not a win).
- Documents the research flags and the holdout harness, and warns that the
  portfolio_monitor lookbacks are nested windows, NOT a holdout.
- Notes the snapshot must copy paper_% settings or it silently diverges from prod.

All baseline numbers re-verified against reports/backtest-20260711-prod-baseline.json
(506 tickers, 1,089 qualified, CAGR 50.4%, +413.8% vs SPY +95.7%, DD -21.4%,
Sharpe 2.04, 320 trades, 15.3d avg hold, and all five promotion contenders). No
corrections were needed — the numbers were right, the framing was not.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
dennisthiessen
2026-07-12 15:40:34 +02:00
co-authored by Claude Opus 4.8
parent 9789e3d762
commit 906d1db7d1
2 changed files with 214 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+107 -11
View File
@@ -1,8 +1,54 @@
# Signal Dashboard
Investing-signal platform for NASDAQ stocks. Surfaces the best trading opportunities through weighted multi-dimensional scoring — technical indicators, support/resistance quality, sentiment, fundamentals, and momentum — with asymmetric risk:reward scanning.
Investing-signal platform for US equities. It runs one strategy, and it is a boring one:
**Philosophy:** Don't predict price. Find the path of least resistance, key S/R zones, and asymmetric R:R setups.
> **A long-only cross-sectional momentum book.** Buy the top quintile by beta-adjusted 12-1 month momentum, tilt toward higher volatility, hold at most 10 names, cut at 1.5× ATR, then trail at 3× ATR for up to 30 trading days.
**Philosophy:** don't predict price — rank it. The edge is *relative* strength across the universe, and the discipline is in the exit: cut losers fast, let winners run until the trail catches them.
**What is NOT the edge — read this before trusting a number on screen.** The composite score, the 5 dimensions, sentiment, fundamentals, and the support/resistance engine are **display and screening context**. None has a measured edge. In particular:
- **The S/R "target" is not an exit.** It exists only to compute the R:R and touch-odds that admit a setup through the activation gate. The live exit never reads it. Across 320 backtested production trades the exit reasons were **144 initial stop, 98 trailing stop, 78 max hold — and 0 targets.** Honoring the target as a take-profit was tested and *halves CAGR* ([research](docs/research/sr-levels-and-exits.md)).
- **The composite score does not select trades.** Residual momentum does.
Full experiment log — everything tested, kept, and rejected: **[docs/research/](docs/research/README.md)**.
## The strategy, end to end
```mermaid
flowchart TD
U["Universe — ~500 tickers<br/>daily OHLCV"] --> M["Residual 12-1 momentum<br/><i>12-month return, skip last month,<br/>beta-adjusted vs SPY</i>"]
M --> R["Rank cross-sectionally<br/>into percentiles"]
R --> G1{"Top 20%?<br/>percentile ≥ 80"}
G1 -->|no| SKIP["Not traded<br/><i>(still scored — the control group)</i>"]
G1 -->|yes| S["Build the setup<br/>entry = last close<br/><b>stop = entry − 1.5 × ATR</b><br/>level = nearest S/R above"]
S --> G2{"Activation gate"}
G2 --> G2a["R:R ≥ 2.0 <i>(to the S/R level)</i>"]
G2 --> G2b["touch odds ≥ 20%"]
G2 --> G2c["action not NEUTRAL<br/>and matches direction"]
G2a & G2b & G2c --> Q{"qualified?"}
Q -->|no| SKIP
Q -->|yes| RANK["Rank by production score<br/>80% momentum %ile<br/>+ 20% volatility %ile"]
RANK --> BOOK{"Room in the book?<br/>max 10 positions"}
BOOK -->|no| WAIT["Wait for a slot"]
BOOK -->|yes| OPEN["OPEN — size at 1% account risk"]
OPEN --> EXIT{"Exit — whichever comes first"}
EXIT --> E1["Initial stop hit<br/>entry − 1.5 × ATR → −1R<br/><b>45% of trades</b>"]
EXIT --> E2["Trailing stop hit<br/>highest close − 3 × ATR<br/><i>only binds once price is ~1R up</i><br/><b>31% of trades</b>"]
EXIT --> E3["Max hold reached<br/>30 trading days<br/><b>24% of trades</b>"]
EXIT -.->|"NEVER"| E4["S/R target<br/><b>0% of trades</b>"]
style M fill:#1e3a5f,color:#fff
style OPEN fill:#1e4d2b,color:#fff
style E4 fill:#2a2a2a,color:#888
style E1 fill:#4a1f1f,color:#fff
style E2 fill:#1e4d2b,color:#fff
```
**How to read the exit box.** The initial stop is tight (1.5× ATR) and the trail is wide (3× ATR), so the trail sits *below* the initial stop at entry and only takes over once price has advanced roughly 1R. Cut fast when wrong; give room once right. That asymmetry is what produces the right-tailed return profile the strategy depends on — most trades lose a little (win rate ~37.5%), a few win big (best trade +12.9R), and *that is why there is no take-profit*.
## How It Works
@@ -31,24 +77,31 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we
### From score to "top pick"
1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize).
2. **Setups** — the scanner builds long/short setups with ATR stops and S/R targets, then adds a confidence score, conflict flags and a target reach-probability.
3. **Activation gate** — a setup *qualifies* only if it clears the R:R floor, ranks in the top residual-momentum percentile of the universe, **and** its primary target carries at least a 20% reach-probability (the validated edge is long-only; the confidence floor was ablated to zero effect and defaults off).
1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize). **Display and ranking only — it does not select trades.**
2. **Setups** — the scanner builds long/short setups with a 1.5× ATR stop and picks the nearest S/R level as a nominal target, then adds a confidence score, conflict flags and a per-level touch-probability.
3. **Activation gate** — a setup *qualifies* only if it ranks in the top residual-momentum percentile of the universe (**the actual selection**, long-only), clears the R:R floor, **and** its primary level carries at least a 20% touch-probability. The confidence floor was ablated to zero effect and defaults off.
4. **Top pick** — qualified setups are ordered by the production rank: 80% residual momentum percentile + 20% 6-month realized-volatility percentile. The #1 is highlighted on the Dashboard and labelled on the ticker page.
**What the R:R and touch-probability in step 3 actually are.** They are *gate inputs*, computed from an S/R level the trade will never exit at — they exist to filter setups, not to forecast the trade you're about to take. A setup with "R:R 2.4:1, 34% touch odds" is not a claim that you'll make 2.4R with 34% probability; it's a claim that this setup cleared the screen. What actually happens to a trade is in the exit box of the diagram above, and on the "what usually happens" panel in the UI. Conflating the two is the single easiest way to misread this app.
## Strategy Status — What's Validated and What Isn't
**Read this before touching scoring, gating, or setup logic.** The platform measures itself — a weekly-replay backtest plus a factor rank-IC harness (`app/services/backtest_service.py`) — and the verdicts below come from those reports (latest run July 2026, ~5 years of OHLCV), not from opinion.
> **The full experiment log lives in [docs/research/](docs/research/README.md)** — every strategy we've tested, the result, and the decision. Check it before proposing an idea; most of the obvious ones have already been run and rejected.
| Component | Verdict | Evidence |
|---|---|---|
| **Residual 12-1 cross-sectional momentum** (the activation gate, long-only) | **Production gate — in-sample edge** | Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable |
| S/R setup engine (ATR stops, S/R targets, reach-probability) | **Filter/execution context, not the exit** | R:R/room-to-run still earns its keep as a filter, but S/R targets underperform the time exit. The probability model is display-only |
| **3× ATR trailing exit** (+ 1.5× ATR initial stop, 30-day max hold) | **Production exit — best Sharpe of every exit tested** | Beat hold / SMA50 / 20-day-low / technical-40 and both take-profit variants (July 2026) |
| S/R setup engine (ATR stops, S/R levels, touch-probability) | **Gate input only — NOT an exit, and not an edge** | The exit never reads the target (0 of 320 trades). Honoring it as a take-profit drops Sharpe 2.04 → 1.47. The R:R floor does real work *as a filter*; the detector itself is methodologically weak. [Full write-up](docs/research/sr-levels-and-exits.md) |
| Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) |
| LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet |
| Fundamentals | Feeds composite + confidence only | Latest values only, no history — same limitation |
| Short setups | **Excluded while the momentum gate is active** | Backtest showed shorts fight the trend and drag expectancy |
| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups |
| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups. Orphaned settings dropped in migration 020 |
| S/R target as a take-profit (tested July 2026) | **Rejected** | Sharpe 2.04 → 1.47, CAGR halved. Win rate *rose* — it truncates the right tail where the edge lives |
| "Clear-air" gate relaxation (tested July 2026) | **Rejected — failed out-of-sample** | Strictly better in-sample (Sharpe 2.07 / CAGR 62.3% / DD 20.1%), then lost on a real train/test split (Sharpe 2.78 → 2.45). A cautionary tale: nested lookbacks are not OOS |
Caveats on the momentum result: in-sample, roughly one market regime, costs/slippage approximated at 0.1% per side, and residual momentum still needs SPY benchmark history to compute. The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.
@@ -59,7 +112,7 @@ Use this as a regression guardrail for future strategy changes, not as a return
| Item | Current baseline |
|---|---|
| Strategy version | `residual_highvol_80_20_atr_trail3_v1` |
| Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R floor on, primary-target reach-probability >= 20%, NEUTRAL excluded, confidence floor effectively off |
| Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R >= 2.0 (live `activation_min_rr`; the code default is 1.2), primary-level touch-probability >= 20%, NEUTRAL excluded, confidence floor off (0) |
| Production rank | 80% residual momentum percentile + 20% 6-month realized-volatility percentile |
| Exit | Initial ATR stop plus 3x ATR trailing stop, max 30 trading days |
| Portfolio CAGR | +50.4% |
@@ -67,7 +120,12 @@ Use this as a regression guardrail for future strategy changes, not as a return
| Max drawdown | -21.4% |
| Sharpe | 2.04 daily, annualized |
| Trades | 320 |
| Win rate | 37.5% |
| Average hold | 15.3 trading days |
| Best / worst trade | +12.9R / -3.3R |
| **How trades actually ended** | **initial stop 144 (45%) · trailing stop 98 (31%) · max hold 78 (24%) · target 0 (0%)** |
That last row is the strategy in one line: a 37.5% win rate is *fine* because the +12.9R tail pays for every −1R stop. It is also why no take-profit exists — and why the S/R "target" shown in the UI is a screening artifact, not a plan.
Promotion evidence from the same snapshot:
@@ -117,6 +175,8 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
### Highest-value next experiments (in order)
> Check **[docs/research/](docs/research/README.md)** first — 12 strategy ideas have already been tested and rejected, including the obvious ones (take-profit exits, regime overlays, inverse-vol sizing, shorts).
1. **Forward monitor the promoted strategy** — the production UI now behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison. Forward paper-trade months are the only evidence the snapshot cannot provide; the July 2026 tuning pass closed every in-sample lead. (Trailing-stop sensitivity and the max-15 capacity check are done — see the tuning table above.)
2. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only.
3. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. Now doubly motivated: it is also where the strong `fip_id` signal (see tuning findings) could become tradeable. (Deeper history was considered and declined.)
@@ -152,8 +212,8 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
- Sentiment analysis with time-decay weighted scoring
- Fundamental data tracking (P/E, revenue growth, earnings surprise, market cap)
- 5-dimension scoring engine (technical, S/R quality, sentiment, fundamental, momentum) with configurable weights
- Risk:Reward scanner — long and short setups, ATR-based stops, S/R-based targets, configurable R:R threshold (default 1.5:1)
- Activation gate — qualifies setups on a residual-momentum percentile floor, an R:R floor and a 20% primary-target reach-probability floor (validated long-only edge)
- Risk:Reward scanner — long and short setups, 1.5x ATR stops, S/R-based nominal targets, configurable scan R:R threshold (default 1.5:1 — distinct from the activation floor below)
- Activation gate — qualifies setups on a residual-momentum percentile floor (the actual selection), an R:R floor (prod: 2.0) and a 20% primary-level touch-probability floor (validated long-only edge)
- Recommendation layer — directional confidence, conflict detection, per-target reach-probability
- Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: 3x ATR trail with a 30-trading-day max hold; time / percent-trailing / target-stop selectable), realized track record + outcome evaluation
- Market-regime index + FRED early-warning monitor (VIX, credit spreads); weekly backtest + manual event study
@@ -284,7 +344,13 @@ npm run build
For research loops, run the production backtest locally from a SQLite snapshot
instead of deploying and clicking the Admin job. The snapshot contains only the
tables needed by `run_backtest`: tickers, OHLCV bars, SPY benchmark closes, and
activation/recommendation settings. Secrets and cached reports are not copied.
the activation / recommendation / paper-exit settings. Secrets and cached reports
are not copied.
> The `paper_%` settings **must** be copied: the portfolio monitor's Production row
> replays the *runtime* exit policy via `get_exit_policy()`. Without them a snapshot
> silently falls back to the code defaults, so a live-tuned exit would not be
> reflected and the local run would disagree with prod for no visible reason.
1. Open an SSH tunnel to the production Postgres instance:
@@ -344,8 +410,29 @@ matching decision. Every change still goes through the factor harness first (see
| `time_exit_sweep` | Net avg R / net R-per-day by hold length | Whether a fixed time exit beats the promoted ATR trail |
| `portfolio_monitor`, `portfolio_sim`, `strategy_variants` | CAGR, Sharpe, max drawdown, per-year returns | Promote a strategy only if it beats the current baseline on CAGR/Sharpe/DD |
| `signal_eval` | Mean IC, t-stat, IC>0 %, `reliable` | Iron rule: wire a new factor in only if \|IC\| ≳ 0.03 with a consistent sign and `reliable: true` |
| `holdout` (opt-in) | Train vs test books, split by entry date | **The only honest OOS read.** Set `BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD` |
| `recommendation`, `research_recommendation` | The report's own headline read | A starting point, not a substitute for the sections above |
**Out-of-sample validation.** The `portfolio_monitor` lookbacks (6m / 1y / 3y / 5y / all) are
**nested windows that all end today** — every one of them overlaps the data an idea was found
on, so none of them is a holdout. A rule that looks good across all five can still be an
in-sample artifact (this exact trap ate the clear-air experiment; see the research log). For a
real train/test split by entry date:
```bash
BACKTEST_HOLDOUT_SPLIT=2024-07-01 python scripts/run_backtest_snapshot.py \
backtest_snapshots/prod.sqlite --workers 7 --allow-spawn
```
Research-only flags, all off by default (the default report is byte-identical to the shipped baseline):
| Flag | What it does |
|---|---|
| `BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD` | Adds a `holdout` section: train (entries before) vs test (entries on/after), as disjoint books |
| `BACKTEST_RESEARCH_EXITS=1` | Adds the rejected take-profit exit rows to the exit comparison |
| `BACKTEST_ATR_TARGET_FALLBACK=k` | Synthesizes a k×ATR target where S/R offers none |
| `BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1` | Restricts that fallback to setups with genuinely no structure ahead |
`recommendation` is the one section surfaced on the deployed page ("What this
backtest recommends"); everything else in this table is intentionally local-only.
@@ -503,6 +590,13 @@ frontend/
├── stores/ # Zustand auth store
└── styles/ # Global CSS with glassmorphism classes
docs/
└── research/ # Experiment log: what was tested, the result, the decision
├── README.md # Overview — start here before proposing a strategy change
└── sr-levels-and-exits.md
reports/ # Committed backtest reports (JSON) + compare_reports.py
deploy/
├── nginx.conf # Reverse proxy + static file serving
├── setup_db.sh # Idempotent DB setup script
@@ -523,6 +617,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
- **`app/services/qualification.py` is mirrored in `frontend/src/lib/qualification.ts`.** Any gate change must land in both, or the UI's "qualified" flags silently disagree with the server.
- **Live scan and backtest share the same pure functions.** The backtest replays production logic through DB-free functions (`compute_technical_from_arrays`, `compute_momentum_from_closes`, `detect_sr_levels`, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
- **One S/R model app-wide:** `sr_service.detect_sr_levels` + `cluster_sr_zones` (2% tolerance) feed the chart, alerts, and target generation identically.
- **The S/R target is a gate input, never an exit.** `_atr_trailing_close()` does not take it as a parameter, and it must stay that way — take-profit exits were tested and halve CAGR. Any UI or alert that implies the trade exits at the target is a bug ([research](docs/research/sr-levels-and-exits.md)).
- **The outcome evaluator evaluates ALL setups**, not just qualified ones — unqualified setups are the control group that makes the Track Record meaningful.
- **`SystemSetting` access goes through `app/services/settings_store.py`** — don't query the model directly.
- **Time-series data gets a real table** (see `benchmark_prices`, `regime_snapshots`); `SystemSetting` JSON is only for config and cached reports.
@@ -544,6 +639,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
| Paper trades + time/trailing/target auto-exit | `app/services/paper_trade_service.py` |
| Point-in-time setup context snapshots | `app/models/signal_context_snapshot.py` + `app/services/rr_scanner_service.py` |
| S/R detection & zone clustering | `app/services/sr_service.py` |
| **Research log — what's been tested and rejected** | **`docs/research/`** |
| SPY benchmark for residual momentum + paper-trade alpha | `app/services/benchmark_service.py` |
| Pipelines & job registration | `app/scheduler.py` |
docs/research/README.md
+107
View File
@@ -0,0 +1,107 @@
# Research log — what we tested, what happened, what we kept
Every strategy question we've put to the backtest, in one table. The point is to
stop re-litigating settled questions: **if a row says "rejected", the experiment
was run and the data said no.** Detail lives in the linked docs and in
`reports/*.json` (all committed).
**The one-line summary of the whole platform:** it is a **long-only
cross-sectional momentum book** — buy the top quintile by beta-adjusted 12-1
momentum, tilt toward higher volatility, hold ≤ 10 names, cut at 1.5× ATR, then
trail at 3× ATR for up to 30 trading days. Everything else in the app (composite
score, S/R levels, sentiment, fundamentals) is **display or screening**, not edge.
---
## 1. What survived — the production strategy
| Component | Status | Why it's there |
|---|---|---|
| **Residual 12-1 momentum, top 20%, long-only** | **The edge.** Everything else is scaffolding | Only component with a measured cross-sectional IC. Promoted July 2026 |
| 80/20 residual-momentum / 6m-volatility rank | Ranking tilt | Buys ~2pp CAGR over momentum-only; costs ~6pp drawdown |
| 1.5× ATR initial stop | Real exit | Cuts losers fast |
| 3× ATR trailing stop, 30-day max hold | Real exit | Best Sharpe of every exit tested |
| Max 10 concurrent positions, 1% risk per trade | Sizing | Cap never binds in practice |
---
## 2. Rejected — do not resurrect without new data
| # | Experiment | Result | Decision | Evidence |
|---|---|---|---|---|
| 1 | **S/R target as a take-profit** (exit at the target, with or without the trail) | Sharpe **2.04 → 1.47**, CAGR halved (50.4% → 28.9%). Win rate *rose* (37.5% → 40.0%) — the tell: it truncates the right tail | **Rejected.** The target must never become an exit | [sr-levels-and-exits.md](sr-levels-and-exits.md) · `backtest-20260712-sr-target-exit.json` |
| 2 | **Clear-air fallback** — synthesize a 3× ATR target so 52-week-high breakouts stop being vetoed by "no resistance above" | Looked *strictly better* in-sample (Sharpe 2.07, CAGR 62.3%, DD 20.1%) but **failed a real out-of-sample holdout**: Sharpe 2.78 → 2.45, higher drawdown | **Rejected.** Gate stays as-is | [sr-levels-and-exits.md](sr-levels-and-exits.md) · `backtest-20260712-holdout-*.json` |
| 3 | **Blanket S/R fallback** (any missing target, not just clear air) | Sharpe 1.82, per-setup expectancy 0.583 → 0.280 R | **Rejected.** 65% of what it admitted were ATR/R:R filter misses, which are actively bad | [sr-levels-and-exits.md](sr-levels-and-exits.md) |
| 4 | **Expected-value gate** (`min_expected_value` replacing the R:R + probability pair) | Structurally favoured distant lottery targets; selected *worse*-than-random setups | **Removed June 2026.** Settings dropped in migration 020 | migration 009, 020 |
| 5 | **Blue-sky projected targets** (invent a target above when none exists) | Dilutive under the ATR-trail exit | **Reverted July 2026.** Same root cause as #2 — better targets can't help when the exit ignores them | — |
| 6 | **SPY 200d-MA regime overlay** (block entries / go flat) | Halves return (315% → 138%), zero drawdown benefit | **Rejected.** The ATR trail already manages downside; the filter blocks the recovery entries that make the money | `backtest-20260708-regime-overlay.json` |
| 7 | **Short setups** | Fight the trend, drag expectancy | **Excluded** while the momentum gate is active | — |
| 8 | **Standalone volatility ranking** (high-vol 80, no momentum) | CAGR 31.6%, DD −34.8%, Sharpe 1.12 | **Rejected.** Vol is a *tilt*, not a signal | prod-baseline |
| 9 | **Low-volatility ranking** | CAGR 2.7%, Sharpe 0.29 | **Rejected.** No edge | prod-baseline |
| 10 | **Inverse-vol position sizing** | The apparent "win" was **mis-attributed**: the 20% notional cap bound on 95% of entries, so it measured concentration, not vol-sizing. Genuine inverse-vol cuts DD to −18.2% but costs ~58pp return at flat Sharpe | **Rejected** as edge; it's a risk-preference trade | `backtest-20260709-position-sizing*.json` |
| 11 | **FIP path-smoothness** as tie-breaker/filter | Non-monotonic within the qualified set; thinning the entry stream costs more compounding than the tilt returns | **Rejected as a filter** — but see §4, it's the strongest raw signal we've measured | — |
| 12 | **Fixed take-profit sweep** (R-multiples) | No interior optimum ever found — the best TP is "no TP" | **Rejected.** Momentum's edge lives in the right tail | `backtest_service.py:450` |
---
## 3. Tuned and confirmed — don't retest on this snapshot
A systematic single-variable sweep (July 2026) confirmed **every** production
setting. Re-running these against the same ~4-year snapshot is wasted compute and
invites overfitting.
| Knob | Verdict |
|---|---|
| ATR trail multiple {1.5–4.0} | **Keep 3.0** — ≤2.0 whipsaws out the right tail; ≥2.5 is a plateau |
| Momentum lookback (6-1, 3-1, 12-7 Novy-Marx, composites) | **Keep residual 12-1** — the others have IC ≈ 0 or weaker t-stats |
| Selection cutoff {70…90} × book size {10, 15, 20} | **Keep 80 × 10** — monotonically worse in both directions |
| Position sizing (equal-weight, inverse-vol, risk-% sweep) | **Keep 1% fixed-fractional** |
| Primary-target probability floor | **Keep 20%** — pruned lottery targets, 1,428 → 1,089 qualified, lifted Sharpe |
| Exit policy (hold / SMA50 / 20-day low / technical-40 / ATR trail) | **Keep 3× ATR trail** — best Sharpe (2.04) |
---
## 4. Open leads
| Lead | Why it's interesting | Blocker |
|---|---|---|
| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign | Doesn't improve *this* book (the momentum gate already captures it in-sample). Revisit when the universe broadens |
| **Broader universe** (`nasdaq_all`) | Strengthens every week's cross-section and the IC t-stat | Also where `fip_id` could become tradeable |
| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time |
| **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
| **S/R detector quality** | POC/VAH/VAL computed then discarded; HVN = "any above-mean bin"; volume double-counted 1.48×; "touch" counts pass-throughs; no round numbers | Worth fixing for the levels users *see* — but it does **not** reach P&L, so don't justify it on returns |
---
## 5. Method rules learned the hard way
1. **Nested lookback windows are NOT out-of-sample.** The clear-air result (#2) was
clean, large, and consistent across five nested windows — and still died on a
proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT`.
2. **Check what population an ablation actually admits.** The blanket fallback (#3)
looked like it tested the "resistance famine" hypothesis. It didn't — 65% of the
setups it let in were a different population entirely, and they drove the result.
3. **A rising win rate is a warning, not a win.** Both #1 and #12 raised the hit
rate while destroying returns. In a right-tailed strategy, "winning more often"
usually means you clipped the winners.
4. **The iron rule:** a signal earns its way into selection *only* through the
factor harness — |mean IC| ≳ 0.03, consistent sign, `reliable: true` (≥ 12
non-overlapping windows). Never let an unvalidated score gate setups.
---
## 6. Why we stay with the current strategy
Everything we've tried to add has either failed the backtest, failed
out-of-sample, or turned out to be measuring something other than what it claimed.
What's left is a boring, well-documented result: **cross-sectional momentum works;
the machinery around it mostly doesn't.**
The S/R engine, the composite score, the sentiment and fundamentals dimensions are
all still in the product — they make the app legible and are useful context for a
human — but none of them has a measured edge, and the platform is honest about
that in the UI (see the exit plan and base-rate panels on every setup card). The
one component that *does* have an edge is the momentum gate, and every knob on it
has been swept and confirmed.
The next real evidence is **forward**, not backward: the live paper-trade record.