# Research log — what we tested, what happened, what we kept

Every strategy question we've put to the backtest, in one table. The point is to
stop re-litigating settled questions: **if a row says "rejected", the experiment
was run and the data said no.** Detail lives in the linked docs and in
`reports/*.json` (all committed).

**The one-line summary of the whole platform:** it is a **long-only
cross-sectional momentum book** — buy the top quintile by beta-adjusted 12-1
momentum, tilt toward higher volatility, hold ≤ 10 names, cut at 1.5× ATR, then
trail at 3× ATR for up to 30 trading days. After an initial stop, require the
daily production gate to fail and subsequently qualify again before re-entry.
Everything else in the app (composite
score, Structural S/R, the Gate Target Ladder, sentiment, fundamentals) is
**display or screening**, not edge.

---

## 1. What survived — the production strategy

| Component | Status | Why it's there |
|---|---|---|
| **Residual 12-1 momentum, top 20%, long-only** | **The edge.** Everything else is scaffolding | Only component with a measured cross-sectional IC. Promoted July 2026 |
| 80/20 residual-momentum / 6m-volatility rank | Ranking tilt | Buys ~2pp CAGR over momentum-only; costs ~6pp drawdown |
| 1.5× ATR initial stop | Real exit | Cuts losers fast |
| 3× ATR trailing stop, 30-day max hold | Real exit | Best Sharpe of every exit tested |
| Post-stop normal gate reset | Re-entry policy | Stop always closes; a later gate failure and subsequent fresh qualification define the next signal episode. The selected study arm reached Sharpe 1.77 / CAGR 48.3% at capacity 10; live scan-before-outcome timing is stricter (Sharpe 1.68 / CAGR 44.8% analogue). [Full study](post-stop-reentry.md) |
| Max 10 concurrent positions, 1% risk per trade | Sizing | Cap never binds in practice |
| Structural S/R | Human-facing product context | Clean, capped zones for charts and alerts; not read by the scanner |
| Gate Target Ladder | Screening machinery | Volume-free transient proposals preserve the production candidate set exactly; never an exit |

---

## 2. Rejected — do not resurrect without new data

| # | Experiment | Result | Decision | Evidence |
|---|---|---|---|---|
| 1 | **Gate target as a take-profit** (exit at the target, with or without the trail) | Sharpe **2.04 → 1.47**, CAGR halved (50.4% → 28.9%). Win rate *rose* (37.5% → 40.0%) — the tell: it truncates the right tail | **Rejected.** The target must never become an exit | [sr-levels-and-exits.md](sr-levels-and-exits.md) · `backtest-20260712-sr-target-exit.json` |
| 2 | **Clear-air fallback** — synthesize a 3× ATR target so 52-week-high breakouts stop being vetoed by "no resistance above" | Looked *strictly better* in-sample (Sharpe 2.07, CAGR 62.3%, DD 20.1%) but **failed a real out-of-sample holdout**: Sharpe 2.78 → 2.45, higher drawdown | **Rejected.** Gate stays as-is | [sr-levels-and-exits.md](sr-levels-and-exits.md) · `backtest-20260712-holdout-*.json` |
| 3 | **Blanket S/R fallback** (any missing target, not just clear air) | Sharpe 1.82, per-setup expectancy 0.583 → 0.280 R | **Rejected.** 65% of what it admitted were ATR/R:R filter misses, which are actively bad | [sr-levels-and-exits.md](sr-levels-and-exits.md) |
| 4 | **Expected-value gate** (`min_expected_value` replacing the R:R + probability pair) | Structurally favoured distant lottery targets; selected *worse*-than-random setups | **Removed June 2026.** Settings dropped in migration 020 | migration 009, 020 |
| 5 | **Blue-sky projected targets** (invent a target above when none exists) | Dilutive under the ATR-trail exit | **Reverted July 2026.** Same root cause as #2 — better targets can't help when the exit ignores them | — |
| 6 | **SPY 200d-MA regime overlay** (block entries / go flat) | Halves return (315% → 138%), zero drawdown benefit | **Rejected.** The ATR trail already manages downside; the filter blocks the recovery entries that make the money | `backtest-20260708-regime-overlay.json` |
| 7 | **Short setups** | Fight the trend, drag expectancy | **Excluded** while the momentum gate is active | — |
| 8 | **Standalone volatility ranking** (high-vol 80, no momentum) | CAGR 31.6%, DD −34.8%, Sharpe 1.12 | **Rejected.** Vol is a *tilt*, not a signal | prod-baseline |
| 9 | **Low-volatility ranking** | CAGR 2.7%, Sharpe 0.29 | **Rejected.** No edge | prod-baseline |
| 10 | **Inverse-vol position sizing** | The apparent "win" was **mis-attributed**: the 20% notional cap bound on 95% of entries, so it measured concentration, not vol-sizing. Genuine inverse-vol cuts DD to −18.2% but costs ~58pp return at flat Sharpe | **Rejected** as edge; it's a risk-preference trade | `backtest-20260709-position-sizing*.json` |
| 11 | **FIP path-smoothness** as tie-breaker/filter | Non-monotonic within the qualified set; thinning the entry stream costs more compounding than the tilt returns | **Rejected as a filter** — but see §4, it's the strongest raw signal we've measured | — |
| 12 | **Fixed take-profit sweep** (R-multiples) | No interior optimum ever found — the best TP is "no TP" | **Rejected.** Momentum's edge lives in the right tail | `backtest_service.py:450` |
| 13 | **Sector-residual 12-1** (`mom_12_1_sector_resid` / sector demean) as replacement for market residual | Short-window IC/A/B looked knife-edge green; deep repaired + **liquid-1500** retest: weeks 83, mild +IC **0.027** / t 1.69, **below iron bar 0.03** (FAIL). Demean already weaker | **Rejected / closed.** Keep production market residual. Do not resurrect without a new pre-registered protocol | [sector-residual-momentum.md](sector-residual-momentum.md) · `sector-resid-deep-20260719-113319.json` · history-depth supersession note |

---

## 3. Tuned and confirmed — don't retest on this snapshot

A systematic single-variable sweep (July 2026) confirmed **every** production
setting. Re-running these against the same ~4-year snapshot is wasted compute and
invites overfitting.

| Knob | Verdict |
|---|---|
| ATR trail multiple {1.5–4.0} | **Keep 3.0** — ≤2.0 whipsaws out the right tail; ≥2.5 is a plateau |
| Momentum lookback (6-1, 3-1, 12-7 Novy-Marx, composites) | **Keep residual 12-1** — the others have IC ≈ 0 or weaker t-stats |
| Selection cutoff {70…90} × book size {10, 15, 20} | **Keep 80 × 10** — monotonically worse in both directions |
| Position sizing (equal-weight, inverse-vol, risk-% sweep) | **Keep 1% fixed-fractional** |
| Primary-target probability floor | **Keep 20%** — pruned lottery targets, 1,428 → 1,089 qualified, lifted Sharpe |
| Primary-target R:R selector | **Keep 1.5** — target choice is intentionally independent of the later 2.0 activation floor |
| Exit policy (hold / SMA50 / 20-day low / technical-40 / ATR trail) | **Keep 3× ATR trail** — best Sharpe (2.04) |
| **Activation R:R floor `min_rr`** (swept 2026-07-12) | **Keep 2.0** — best in-sample *and* out-of-sample. But it is a **spike, not a plateau** — see below |
| Post-stop re-entry (nine daily policy arms) | **Keep normal gate reset at production capacity 10** — Sharpe 1.77 vs 1.67 immediate and 1.47 fixed cooldown 5. The result changes with book capacity; see [post-stop-reentry.md](post-stop-reentry.md) |

### The `min_rr` sweep (2026-07-12)

`min_rr = 2.0` had been hand-set in Admin and **never swept** — the gate ablation only
tested the floor *on vs off*, never its level. Swept against portfolio Sharpe under the
real exit, with a parity self-check (`reproduces_production_gate: true` — the row at 2.0
rebuilds production's exact 1,089-setup qualified set).

Reports: `backtest-20260712-min-rr-sweep.json` (in-sample), `-oos.json` (test window only).

| min_rr | qualified | In-sample Sharpe / CAGR | **OOS** Sharpe / CAGR (entries ≥ 2024-07) |
|---|---|---|---|
| 0.0 (floor off) | 6636 | 1.98 / 58.5% | 2.02 / 66.2% |
| 1.2 (old code default) | 3897 | 1.34 / 33.9% | 1.12 / 28.8% |
| 1.5 | 3127 | 1.20 / 29.6% | 1.12 / 28.8% |
| 1.75 | 1974 | 1.64 / 44.5% | 1.15 / 27.4% |
| **2.0 (live)** | 1089 | **2.04 / 50.4%** | **2.78 / 73.3%** |
| 2.25 | 577 | 1.64 / 31.8% | 1.71 / 31.9% |
| 2.5 | 286 | 1.67 / 29.0% | 0.68 / 8.7% |
| 3.0 | 89 | 1.09 / 9.1% | 0.87 / 5.0% |

**Verdict: keep 2.0.** It is the optimum in **both** windows, and the peak reproducing in
data it was never fitted to is real evidence — the one thing the clear-air experiment
couldn't show.

**But treat it as fragile, and do not nudge it.** Unlike the ATR trail (a plateau above
2.5), this is a **spike with a trough beside it**: ±0.25 costs ~0.4 Sharpe in-sample and
~1.6 Sharpe out-of-sample. A knob that sharp is not a robustly identified parameter, and
the curve is *bimodal* (floor-off is good, 1.2–1.75 is bad, 2.0 is good) — which is not
how a well-behaved threshold behaves. We got lucky: the hand-set value landed on the peak.

**Also worth knowing:** turning the floor **off entirely** is the second-best row in both
windows — nearly the same Sharpe with **substantially higher CAGR** (58.5% / 66.2%) and
more trades. If CAGR ever matters more than Sharpe here, "no R:R floor" is a live option,
and it would also sever the last dependency the *gate* has on the weak S/R detector.

---

## 4. Phase A matrix (2026-07-18) — closed

Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** ·
`reports/research-matrix-phase-a.json`.

| Arm | Decision |
|---|---|
| Max-hold {45,60,90} | **Note and move on** — validation glitter, train collapse (regime interaction) |
| Equity-curve vol targeting | **Reject as edge** on this sample; park vt25 as optional DD insurance only |
| Correlation caps | **Reject**; sector caps stay Phase B with reduced expectations |
| Next-open fill | **Discovery, not reject** — honest deployable ~Sharpe 1.2 / CAGR 30% under overnight scanner. Decision baseline until near-close ships = `next_open` |
| `fip_id` re-derive | **Validated** (IC −0.045, t = −2.92) |

### Execution recovery (same day) — closed as evidence

Full write-up: **[execution-recovery.md](execution-recovery.md)** ·
`reports/execution_recovery_matrix.json`.

| Finding | Decision |
|---|---|
| Monotone fill timing (next_open → stale → close) + DD recovery | **When you fill**, not decaying alpha |
| Live bracket **[1.57, 1.77]** full Sharpe | Near-close expected near top of bracket; no more fill-timing sim |
| Gap-cap | **Dead** — third tail-trim instance |
| Auto-`recover: false` | Not a null — bar hit the lower-bound arm by 0.03 train SE |

**Highest-leverage open work:** **ops** — move the single daily R:R scan to
`America/New_York` near-close (checklist in execution-recovery.md). Not more research
knobs.

---

## 5. Open leads

| Lead | Why it's interesting | Blocker |
|---|---|---|
| **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Schedule + fill_mode shipped; live paper validation ongoing |
| **`fip_id` / liquid breadth** | Fingerprint −0.045 / t −2.91; liquid unconditional **−0.017 / t −1.85** (not green); mom-conditional **−0.088 / t −4.58** | **Parked.** Orphan +0.0575 died (snapshot race). Breadth did not strengthen resid-mom t-stat. Optional reopen = pre-registered two-arm liquid-1500 book first. See [fip-breadth-ic.md](fip-breadth-ic.md) |
| **Broader universe** | Composition changes factor signs (fip tug-of-war); vol-tilt on breadth is only a **directional hypothesis** (auth. −0.048 / t −1.36) | Any prod broaden must re-validate 80/20 tilt; offline research only; research.sqlite requires completion manifest |
| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
| **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |

---

## 6. Method rules learned the hard way

1. **Nested lookback windows are NOT out-of-sample.** The clear-air result (#2) was
   clean, large, and consistent across five nested windows — and still died on a
   proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT` / a named
   validation window — and do not pretend a repeatedly opened window is pristine.
2. **Check what population an ablation actually admits.** The blanket fallback (#3)
   looked like it tested the "resistance famine" hypothesis. It didn't — 65% of the
   setups it let in were a different population entirely, and they drove the result.
3. **A rising win rate is a warning, not a win.** Both #1 and #12 raised the hit
   rate while destroying returns. In a right-tailed strategy, "winning more often"
   usually means you clipped the winners.
4. **The iron rule:** a signal earns its way into selection *only* through the
   factor harness — |mean IC| ≳ 0.03, consistent sign, `reliable: true` (≥ 12
   non-overlapping windows). Never let an unvalidated score gate setups.
5. **Momentum filters are guilty of tail-trimming until proven otherwise.**
   Independent failures: take-profit exits, FIP as an in-book filter, gap-up entry
   caps. Cosmetic quality up, P&L down — the right tail *is* the edge.
6. **Fill timing is part of the strategy.** Close-fill reports are not deployable
   numbers for an overnight scanner. Grade promotion under the fill mode you will
   actually trade.
7. **Incomplete research artifacts are not results.** The Phase B +0.0575 / t +5.12
   liquid-fip row was orphaned within hours: it raced a partially built
   `research.sqlite`. Extender now writes a completion manifest; breadth mode
   refuses without a match. Same class of protection as calendar-truncation
   asserts — do not re-mythologize numbers computed on half a universe.

---

## 7. Why we stay with the current strategy

Everything we've tried to add has either failed the backtest, failed
out-of-sample, or turned out to be measuring something other than what it claimed.
What's left is a boring, well-documented result: **cross-sectional momentum works;
the machinery around it mostly doesn't.**

Structural S/R, the composite score, sentiment and fundamentals remain useful
human context but have no measured edge. The Gate Target Ladder is different:
it is internal screening machinery whose broad historical-price-traffic behavior
was preserved explicitly and volume-free, with exact full-period parity. It is
still neither market structure nor an exit. The one component that *does* have
measured predictive edge is the momentum gate, and every knob on it has been
swept and confirmed. After an initial-stop exit, that same gate now also defines
when a new episode may begin: one later failed observation followed by a fresh
qualification. The [daily re-entry matrix](post-stop-reentry.md) supports this
for the current 10-position book, but not as a universal rule for other
portfolio capacities.

The next real evidence is **forward**, not backward: the live paper-trade record.