feat: add strategy variant lab and signal context snapshots
Backtest report now includes research-only hold-to-horizon portfolio variants comparing raw vs residual 12-1 momentum, cutoff 80 vs 90, max 10 vs 15 positions, and SPY-200 risk scaling. A dynamic research recommendation panel flags residual momentum, cutoff 90, or regime scaling only when transparent promotion rules pass. Adds signal_context_snapshots with migration 016 and captures one point-in-time context row per newly generated TradeSetup: setup fields, composite/dimensions, latest sentiment, latest fundamentals, and strategy_version=momentum_12_1_rr_time_v1. This is forward-only; no historical sentiment/fundamental backfill is attempted. No live gate, paper-trade exit, or production ranking behavior changes. Verification: 458 backend tests pass, ruff check app/ clean, frontend npm run build clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -42,15 +42,15 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we
|
||||
|
||||
| Component | Verdict | Evidence |
|
||||
|---|---|---|
|
||||
| **12-1 cross-sectional momentum** (the activation gate, long-only) | **The only demonstrated edge — in-sample** | Qualified setups ≈ **+0.25R** avg vs ≈ −0.05R all-setups baseline; the percentile sweep is cleanly monotonic (cutoff 50 → +0.14R, 70 → +0.21R, 80 → +0.25R). Rank-IC ≈ 0.05, t ≈ 1.6 — right sign and size for the classic factor, **not yet statistically significant** |
|
||||
| S/R setup engine (ATR stops, S/R targets, reach-probability) | **No selection edge — execution/timing only** | ≈ breakeven (+0.01R) before the momentum gate. The probability model is honest (calibrated) but does not discriminate winners |
|
||||
| **12-1 cross-sectional momentum** (the activation gate, long-only) | **The only demonstrated edge — in-sample** | Qualified setups beat the all-setups baseline after costs; rank-IC ≈ 0.05. Residual 12-1 momentum is now evaluated as a research signal, but is not production ranking yet |
|
||||
| S/R setup engine (ATR stops, S/R targets, reach-probability) | **Filter/execution context, not the exit** | R:R/room-to-run still earns its keep as a filter, but S/R targets underperform the time exit. The probability model is display-only |
|
||||
| Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) |
|
||||
| LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet |
|
||||
| Fundamentals | Feeds composite + confidence only | Latest values only, no history — same limitation |
|
||||
| Short setups | **Excluded while the momentum gate is active** | Backtest showed shorts fight the trend and drag expectancy |
|
||||
| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups |
|
||||
|
||||
Caveats on the momentum result: in-sample, roughly one market regime, no transaction costs or slippage modeled, and the factor is beta-heavy (6-month volatility posted the top IC — that's beta, not alpha). The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.
|
||||
Caveats on the momentum result: in-sample, roughly one market regime, costs/slippage approximated at 0.1% per side, and the factor is beta-heavy (6-month volatility often posts the top IC — that's beta, not alpha). The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.
|
||||
|
||||
### The iron rule for strategy changes
|
||||
|
||||
@@ -64,16 +64,15 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
|
||||
|
||||
### Highest-value next experiments (in order)
|
||||
|
||||
1. **Volatility-scaled momentum** — add `mom_12_1 / vol_6m` to `_signal_values`; risk-adjusted momentum typically beats raw and dampens momentum crashes.
|
||||
2. **Regime filter on the gate** — momentum crashes cluster in post-bear rebounds; `market_regime_service` already computes the SPY 50/200 trend, so test "qualify only in Risk-On" in the backtest before wiring it live.
|
||||
3. **Cost haircut in the backtest** — subtract a fixed per-trade cost (e.g. 0.1% per side) in the outcome aggregation so expectancy is net; a thin edge must survive costs.
|
||||
1. **Residual momentum portfolio variants** — compare raw vs beta-adjusted 12-1 momentum in the strategy-variant simulator before changing production ranking.
|
||||
2. **Regime/risk scaling** — test whether SPY-200 risk scaling reduces drawdown enough to justify lower exposure.
|
||||
3. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only.
|
||||
4. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. (Deeper history was considered and declined.)
|
||||
5. **Exit tuning with the existing sweeps** — the report already sweeps fixed take-profits and trailing stops against the S/R-target model; momentum's edge lives in the right tail, so wide trailing exits (already the paper-trade default) tend to beat nearby S/R targets. Also worth testing: a pure time-based exit (hold ~1 month, re-rank) instead of the 30-day target/stop race.
|
||||
|
||||
## Key Use Cases
|
||||
|
||||
- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists qualified setups ranked by momentum with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop.
|
||||
- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed on stop/target, and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge.
|
||||
- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed by the active exit policy (default: 30 trading days with the initial stop), and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge.
|
||||
|
||||
## Stack
|
||||
|
||||
@@ -405,6 +404,7 @@ Context for whoever — human or AI — continues this work. The owner pushes st
|
||||
- **The outcome evaluator evaluates ALL setups**, not just qualified ones — unqualified setups are the control group that makes the Track Record meaningful.
|
||||
- **`SystemSetting` access goes through `app/services/settings_store.py`** — don't query the model directly.
|
||||
- **Time-series data gets a real table** (see `benchmark_prices`, `regime_snapshots`); `SystemSetting` JSON is only for config and cached reports.
|
||||
- **Discretionary overlay data is forward-only.** `signal_context_snapshots` captures composite/dimension/sentiment/fundamental context for new setups. Do not approximate historical sentiment/fundamental snapshots from today's data.
|
||||
- Style: surgical changes, minimal new files; extend existing services rather than adding parallel ones.
|
||||
|
||||
### Where the strategy lives
|
||||
@@ -419,7 +419,8 @@ Context for whoever — human or AI — continues this work. The owner pushes st
|
||||
| Gate defaults / admin config | `app/services/admin_service.py` (`ACTIVATION_DEFAULTS`) |
|
||||
| Backtest + factor rank-IC harness ("Signal edge") | `app/services/backtest_service.py` |
|
||||
| Outcome resolution (target/stop/expired/ambiguous) | `app/services/outcome_service.py` |
|
||||
| Paper trades + trailing auto-exit | `app/services/paper_trade_service.py` |
|
||||
| Paper trades + time/trailing/target auto-exit | `app/services/paper_trade_service.py` |
|
||||
| Point-in-time setup context snapshots | `app/models/signal_context_snapshot.py` + `app/services/rr_scanner_service.py` |
|
||||
| S/R detection & zone clustering | `app/services/sr_service.py` |
|
||||
| SPY benchmark for paper-trade alpha | `app/services/benchmark_service.py` |
|
||||
| Pipelines & job registration | `app/scheduler.py` |
|
||||
|
||||
Reference in New Issue
Block a user