dennisthiessen
bb8aa655a1
Drop intermediate history-depth reports, sector-residual runners/map/code hooks (evidence stays in final reports + docs), and slim MacBook helper to ssl/earnings/ prod-book-matrix only. SSL bootstrap and archived research conclusions retained.
236 lines
9.0 KiB
Markdown
236 lines
9.0 KiB
Markdown
# Sector-residual momentum (Tier-1 alpha research)
|
||
|
||
**Status:** **CLOSED / REJECTED** — do not resurrect without a new pre-registered protocol.
|
||
**Branch:** `research/earnings-gap-and-sue` (final grade) · earlier short-window work on `research/sector-residual-momentum`
|
||
**Production impact:** none. Market residual 12-1 remains the production momentum leg.
|
||
**Authoritative deep grade:** `reports/sector-resid-deep-20260719-113319.json` (**FAIL**)
|
||
**Short-window A/B (superseded for promotion):** `reports/sector-residual-20260719-083356.json` — knife-edge only; not decisive after deep masked retest.
|
||
|
||
### Closure (2026-07-19)
|
||
|
||
Pre-registered deep test on repaired snapshot + liquid-1500 mask:
|
||
|
||
| check | result |
|
||
|---|---|
|
||
| weeks extended (≫ 35) | pass (83) |
|
||
| sign +, reliable, eras both + | pass |
|
||
| t ≥ `mom_12_1_resid` same CS | pass (1.69 ≥ 1.30) |
|
||
| \|mean IC\| ≥ 0.03 | **fail (0.0268)** |
|
||
|
||
**Verdict:** Task 1 CLOSED — sector residual dead on deep evidence.
|
||
`mom_12_1_sector_demeaned` remains DEAD for promotion. No further sector-residual variants from this thread.
|
||
|
||
---
|
||
|
||
## Pre-registration (locked before first research run)
|
||
|
||
### Hypothesis
|
||
|
||
Residualizing 12–1 momentum against the sector, not only the market, reduces
|
||
factor volatility at similar return (Blitz / Huij / Martens-style) → higher
|
||
Sharpe on the production book when the residual replaces market-only residual
|
||
as the momentum leg.
|
||
|
||
### Signals (candidates)
|
||
|
||
| signal | construction |
|
||
|---|---|
|
||
| `mom_12_1_sector_resid` | Two-factor residual vs SPY + ticker’s sector ETF. Same window as `mom_12_1_resid`: ≥100 daily obs, 252-bar lookback, 21-bar skip; two-factor OLS betas **without intercept**; cumulate residual returns over the formation window. |
|
||
| `mom_12_1_sector_demeaned` | Plain `mom_12_1` minus the **cross-sectional** mean of `mom_12_1` within the same GICS sector that week (≥2 names in sector). No regression. |
|
||
|
||
### Baselines (same run, same cross-sections — iron rule)
|
||
|
||
Always report side-by-side with:
|
||
|
||
- `mom_12_1`
|
||
- `mom_12_1_resid`
|
||
|
||
Computed on the **identical** weekly non-overlapping cross-sections in this run.
|
||
Never compare against IC numbers from another report.
|
||
|
||
### Iron rule (IC harness)
|
||
|
||
Source of truth: `_signal_evaluation` in `app/services/backtest_service.py`.
|
||
|
||
- Mean weekly Spearman IC on **non-overlapping** weekly windows
|
||
- Bar: \|mean IC\| ≥ ~0.03, **consistent positive sign**, `reliable: true` (≥ 12 windows)
|
||
|
||
### Promotion to portfolio A/B (candidate → book)
|
||
|
||
A candidate promotes to A/B **only if**:
|
||
|
||
1. It clears the iron-rule bar **and**
|
||
2. Its IC **t-stat ≥** that of `mom_12_1_resid` on the same cross-sections.
|
||
|
||
### Portfolio A/B grading (if and only if IC promotion fires)
|
||
|
||
- Swap candidate in as the **momentum leg** of the production 80/20 momentum/vol
|
||
rank **and** as the gate-percentile signal.
|
||
- `fill_mode=close`, `COST_PER_SIDE = 0.001`, full config otherwise unchanged.
|
||
- Validation window = entries ≥ **2024-07-01** (call it **validation**, not
|
||
holdout — contaminated by prior experiments).
|
||
- Pre-registered promotion bar:
|
||
- validation Sharpe ≥ control − 0.5·SE
|
||
- full-period Sharpe and max-DD **not worse** than control
|
||
- Report Lo / Mertens-adjusted SEs.
|
||
|
||
### Optional sector-cap sub-experiment
|
||
|
||
Only if labels are in **and** A/B ran: max **3** positions per sector in the
|
||
10-slot book. Same A/B grading. **Tail-trim presumption of guilt** (rule 4):
|
||
report entry counts and both tails of the R distribution. Rising win rate with
|
||
falling Sharpe/CAGR = red flag → do not promote.
|
||
|
||
**This run:** sector-cap arm **not executed** (optional; A/B unconstrained book
|
||
only). Can be a human-approved follow-up.
|
||
|
||
### Verdict labels
|
||
|
||
| label | meaning |
|
||
|---|---|
|
||
| **PROMOTE** | Clears pre-registered bar; human decides next (wire design separate) |
|
||
| **PARK** | Inconclusive / weak; keep machinery, no book change |
|
||
| **DEAD** | Failed iron rule or worse than residual baseline with clear sign |
|
||
|
||
### Explicit non-goals
|
||
|
||
- No production deploy from this doc
|
||
- Do not resurrect: take-profit exits, EV gate, regime entry-blocking,
|
||
inverse-vol sizing, gap-caps, unconditional FIP filter
|
||
|
||
---
|
||
|
||
## Data provenance
|
||
|
||
### Snapshot race guard
|
||
|
||
| check | result |
|
||
|---|---|
|
||
| Snapshot path | `backtest_snapshots/prod.sqlite` |
|
||
| Manifest | none (expected for prod snapshot); bar-count sanity applied |
|
||
| Tickers / OHLCV | **506** / **629,263** |
|
||
| Bars min / avg / max | 14 / 1246.1 / 1261 |
|
||
| OHLCV range | 2021-06-24 → 2026-07-02 |
|
||
| Partial-build red flags | none (avg bars healthy) |
|
||
|
||
Integrity fingerprint on same run: `fip_id` mean IC **−0.045** / t **−2.91**
|
||
(35 weeks, N≈498) — matches the established prod fingerprint.
|
||
|
||
### Sector labels
|
||
|
||
| source | count |
|
||
|---|---:|
|
||
| Public S&P 500 GICS CSV | 496 newly filled |
|
||
| FMP profile requests | 10 (all missing after CSV) |
|
||
| Mapped / universe | **505 / 506 (99.8%)** |
|
||
| With mappable ETF | 505 |
|
||
| Still missing | **RHM** only |
|
||
|
||
Persist path: `data/research/ticker_sector_map.json`.
|
||
|
||
FMP aliases (`Technology`, `Consumer Defensive`, `Financial Services`) map to
|
||
SPDRs via the alias table in `app/services/sector_map.py`.
|
||
|
||
### Sector ETFs in `benchmark_prices` (auxiliary only — not tradable)
|
||
|
||
| symbol | bars | min date | max date |
|
||
|---|---:|---|---|
|
||
| SPY | 1516 | 2020-07-06 | 2026-07-17 |
|
||
| XLB…XLY (11) | 1512 each | 2020-07-10 | 2026-07-17 |
|
||
|
||
Fetched via Alpaca `Adjustment.SPLIT` into **`benchmark_prices`** (same table as
|
||
SPY) so they never enter the ticker universe or candidate replay.
|
||
|
||
---
|
||
|
||
## Results
|
||
|
||
Generated: `2026-07-19T08:33:56`
|
||
|
||
### IC harness (identical cross-sections, production 506-name universe)
|
||
|
||
| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable | ic+_pct | quintile spread |
|
||
|---|---:|---:|---:|---:|---|---:|---:|
|
||
| **mom_12_1_sector_resid** | **0.0578** | **2.34** | 35 | 497.7 | true | 65.7 | 0.0245 |
|
||
| mom_12_1_resid | 0.0552 | 1.98 | 35 | 497.7 | true | 60.0 | 0.0207 |
|
||
| mom_12_1 | 0.0531 | 1.61 | 35 | 497.7 | true | 65.7 | 0.0206 |
|
||
| mom_12_1_sector_demeaned | 0.0340 | 1.32 | 35 | 496.7 | true | 62.9 | 0.0154 |
|
||
|
||
### IC promotion grades
|
||
|
||
| candidate | iron rule | t ≥ resid | promote_to_ab |
|
||
|---|---|---|---|
|
||
| `mom_12_1_sector_resid` | pass (IC 0.058, +sign, reliable) | **yes** (2.34 ≥ 1.98) | **yes** |
|
||
| `mom_12_1_sector_demeaned` | pass (IC 0.034, +sign, reliable) | **no** (1.32 < 1.98) | **no** |
|
||
|
||
### Portfolio A/B — `mom_12_1_sector_resid` as residual leg
|
||
|
||
Config: production 80/20 residual/high-vol rank + gate percentile, `fill_mode=close`,
|
||
cost 10 bps/side, ATR trail / gate-reset re-entry as live. Validation split
|
||
2024-07-01.
|
||
|
||
| window | arm | Sharpe | Sharpe SE (Mertens) | CAGR % | max DD % | trades | n_days |
|
||
|---|---|---:|---:|---:|---:|---:|---:|
|
||
| train | control (resid) | 1.30 | 0.685 | 29.2 | 21.4 | 176 | 525 |
|
||
| train | treatment (sector resid) | **1.57** | 0.677 | **35.5** | **19.8** | 176 | 530 |
|
||
| validation | control | **2.92** | 0.709 | **76.3** | **11.7** | 150 | 501 |
|
||
| validation | treatment | 2.57 | 0.701 | 66.3 | 14.8 | 163 | 501 |
|
||
| full | control | 2.09 | 0.497 | 51.6 | 21.4 | 322 | 1000 |
|
||
| full | treatment | 2.09 | 0.491 | 51.0 | **19.8** | 337 | 1005 |
|
||
|
||
**Pre-registered A/B checks**
|
||
|
||
| check | result |
|
||
|---|---|
|
||
| val Sharpe ≥ control − 0.5·SE | **pass** (2.57 ≥ 2.92 − 0.5×0.701 = 2.5695) — **knife-edge** |
|
||
| full Sharpe not worse | **pass** (2.09 = 2.09) |
|
||
| full max DD not worse | **pass** (19.8 < 21.4) |
|
||
|
||
Qualified long candidates: control 1086 vs treatment 1210 (sector residual
|
||
gates a slightly larger set).
|
||
|
||
---
|
||
|
||
## Verdict (final — archived)
|
||
|
||
| signal | verdict | note |
|
||
|---|---|---|
|
||
| **`mom_12_1_sector_resid`** | **CLOSED / REJECTED** | Deep masked IC 0.0268 < 0.03 bar (`sector-resid-deep-20260719-113319`). Short-window PROMOTE superseded. |
|
||
| **`mom_12_1_sector_demeaned`** | **DEAD** | Never cleared t vs market residual; stays dead. |
|
||
|
||
Short-window evidence below is **historical only** (pre-deep retest). Do not use it
|
||
to reopen promotion.
|
||
|
||
### Read carefully (archived context)
|
||
|
||
1. Short-window IC (0.058 / t 2.34 vs resid 0.055 / t 1.98 on 35 weeks) and knife-edge
|
||
A/B looked openable — that was the data gap era (shallow prod bars).
|
||
2. Deep repaired + liquid-1500 retest closed the case: weeks 83, mild +IC, **below bar**.
|
||
3. Production keeps **market** residual 12-1. Research harness may still *emit*
|
||
sector residual for diagnostics; it is not a promotion candidate.
|
||
4. **Do not resurrect** without a new pre-registered protocol and new data.
|
||
|
||
---
|
||
|
||
## What a human must decide next
|
||
|
||
**Nothing on Task 1** — archived. Optional: leave research machinery in tree
|
||
(harmless) or delete later as cleanup; not a strategy decision.
|
||
|
||
---
|
||
|
||
## Implementation notes
|
||
|
||
Research runners and sector-residual harness hooks were **removed after close**
|
||
(2026-07-19 cleanup). Evidence remains in the report artifacts below. Do not
|
||
re-add without a new pre-registered protocol.
|
||
|
||
---
|
||
|
||
## Artifacts
|
||
|
||
| file | role |
|
||
|---|---|
|
||
| `reports/sector-resid-deep-20260719-113319.json` | **Authoritative deep FAIL** |
|
||
| `reports/sector-residual-20260719-083356.json` | Short-window IC/A/B (superseded for promotion) |
|