research: sector residual, earnings gap/SUE, history-depth scaffolding
Tier-1 alpha research (local only, no production deploy): Sector residual momentum: two-factor SPY+sector residual and sector demean signals, IC harness + A/B. Sector resid clears pre-registered bars narrowly (PROMOTE for human wire design only). Sector demean fails t vs market resid. Earnings: earnings_events backfill (FMP bulk paid; FMP/AV per-symbol), 2a gap diagnostic report-only, 2b SUE IC (PARK; incomplete 48/506 coverage). History-depth: pre-registered doc + runner for MacBook deep rebuild/harness. Do not ship production residual or filters from this branch.
This commit is contained in:
@@ -0,0 +1,202 @@
|
||||
# Earnings gap diagnostic + SUE / PEAD (Tier-1 alpha research)
|
||||
|
||||
**Status:** **PARK** (incomplete earnings coverage; SUE fails iron rule on available sample).
|
||||
**Branch:** `research/earnings-gap-and-sue`
|
||||
**Production impact:** none. Local research only. **No filters shipped from 2a.**
|
||||
**Artifacts:** `reports/earnings-gap-sue-20260719-093129.json` (+ companion `.md`)
|
||||
|
||||
---
|
||||
|
||||
## Pre-registration (locked before first research run)
|
||||
|
||||
### Data
|
||||
|
||||
- Historical earnings calendar for the production universe over the full snapshot
|
||||
window (and deeper if the feed provides it).
|
||||
- Preferred source: FMP **date-range earnings-calendar** (bulk). If unavailable on
|
||||
free tier, fall back to per-symbol `/stable/earnings` with request accounting.
|
||||
- Store in a real local table `earnings_events` (symbol + announce_date key).
|
||||
- Point-in-time: a surprise is usable only from **announce date + 1 trading day**
|
||||
onward.
|
||||
|
||||
### Experiment 2a — earnings-gap risk (defense, report-only)
|
||||
|
||||
Join simulated production-config trades (`fill_mode=close`) with earnings dates.
|
||||
|
||||
**Pre-registered questions:**
|
||||
|
||||
1. What fraction of losses worse than **−1R** occur with an earnings announcement
|
||||
**between entry and exit** (inclusive of the holding window)?
|
||||
2. What is the mean R of entries taken within **3 trading days BEFORE** an
|
||||
announcement vs all other entries — report **both tails** of the R
|
||||
distribution (rule 4: any earnings-avoid entry filter is presumed guilty of
|
||||
right-tail trimming until the win distribution shows otherwise)?
|
||||
|
||||
**Output:** distributions and counts only.
|
||||
**No filter is shipped.** If numbers argue for a filter → report and stop.
|
||||
|
||||
### Experiment 2b — SUE / PEAD (offense)
|
||||
|
||||
Signal `sue_latest`:
|
||||
|
||||
\[
|
||||
\text{SUE} = \frac{\text{actual} - \text{estimate}}{\sigma(\text{trailing 8 surprises})}
|
||||
\]
|
||||
|
||||
Fallback if estimate history is thin: scale surprise by price.
|
||||
Carry forward from announce+1 for **63 trading days**, else NaN (name drops out
|
||||
of that cross-section).
|
||||
|
||||
**Iron rule (IC harness):** mean weekly Spearman IC on non-overlapping weeks;
|
||||
\|mean IC\| ≥ ~0.03, **positive** sign (drift), `reliable: true` (≥12 windows).
|
||||
|
||||
Always side-by-side with `mom_12_1` and `mom_12_1_resid` on **identical**
|
||||
cross-sections.
|
||||
|
||||
Also report **momentum-conditional** IC (within top momentum quintile).
|
||||
|
||||
**If it passes iron rule:** STOP and report. Book-integration design is a
|
||||
separate human-approved step — do not wire.
|
||||
|
||||
### Verdict labels
|
||||
|
||||
| label | meaning |
|
||||
|---|---|
|
||||
| **PROMOTE** | (2b only) iron rule cleared → human designs tilt/gate |
|
||||
| **PARK** | Interesting but incomplete / weak |
|
||||
| **DEAD** | No edge / diagnostic argues against action |
|
||||
| **REPORT-ONLY** | (2a) always — never auto-filter |
|
||||
|
||||
---
|
||||
|
||||
## Data provenance
|
||||
|
||||
| item | result |
|
||||
|---|---|
|
||||
| Snapshot | `backtest_snapshots/prod.sqlite` (506 names) |
|
||||
| FMP bulk `earnings-calendar` | **402 Premium** — not available on free tier |
|
||||
| FMP per-symbol `/stable/earnings` | used; hit daily rate limit ~225 reqs |
|
||||
| Alpha Vantage `EARNINGS` | used for +24 symbols (announce = `reportedDate`) |
|
||||
| Symbols with events | **48 / 506 (9.5%)** |
|
||||
| Total events | 5,612 (5,018 with actual+estimate) |
|
||||
| Announce range | 1985-08-31 → 2026-07-16 |
|
||||
| FMP requests (first day) | 260 FMP + 25 AV (see `reports/earnings-backfill-status.json`) |
|
||||
|
||||
**Incomplete backfill is first-class.** 2a under-detects earnings overlaps; 2b SUE
|
||||
cross-section averages **~47 names**, not ~500. Resume:
|
||||
|
||||
```bash
|
||||
# Day N (FMP free ~250/day; AV free ~25/day — prefer FMP after reset)
|
||||
python scripts/backfill_earnings_events.py \
|
||||
--snapshot backtest_snapshots/prod.sqlite \
|
||||
--provider fmp --force-symbol --limit 250 --sleep 0.4
|
||||
|
||||
# When done==506:
|
||||
python scripts/run_earnings_research.py \
|
||||
--snapshot backtest_snapshots/prod.sqlite \
|
||||
--workers 6 --allow-spawn
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Results
|
||||
|
||||
Generated: `2026-07-19T09:31:29`
|
||||
|
||||
### 2a — Earnings-gap risk (report-only)
|
||||
|
||||
Production book sim: Sharpe 2.09 (SE 0.497), CAGR 51.6%, max DD 21.4%, **322 trades**,
|
||||
`fill_mode=close`.
|
||||
|
||||
#### Q1 — Losses worse than −1R with earnings in hold
|
||||
|
||||
| metric | value |
|
||||
|---|---:|
|
||||
| n losses < −1R | 28 |
|
||||
| of which earnings in hold | **1** |
|
||||
| fraction | **3.6%** |
|
||||
| all trades with earnings in hold | 14 / 322 (4.4%) |
|
||||
|
||||
**Read:** On incomplete earnings labels this is a **lower bound** on earnings
|
||||
overlap, not a clean “earnings rarely hurt.” Do **not** conclude earnings risk is
|
||||
immaterial until coverage ≥ ~95% of the book’s names.
|
||||
|
||||
#### Q2 — Entry within 3 trading days before announce (both tails)
|
||||
|
||||
| cohort | n | mean R | win rate | p05 | p50 | p95 | max |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| pre-earn (≤3d before) | **4** | 1.94 | 50% | −1.24 | 1.12 | 6.26 | 6.84 |
|
||||
| other | 318 | 0.70 | 37% | −1.11 | −0.83 | 6.08 | **12.87** |
|
||||
| all | 322 | 0.71 | 37% | −1.12 | −0.83 | 6.22 | 12.87 |
|
||||
|
||||
**Tail-trim presumption:** n=4 is not a sample. Point estimate does **not** show
|
||||
right-tail destruction of pre-earn entries (p95 similar; max actually higher in
|
||||
“other”). **No earnings-avoid filter is supported.** Re-run after full backfill.
|
||||
|
||||
---
|
||||
|
||||
### 2b — SUE / PEAD IC
|
||||
|
||||
#### Full-universe harness (mom on ~500; SUE only where labeled)
|
||||
|
||||
| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable |
|
||||
|---|---:|---:|---:|---:|---|
|
||||
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | true |
|
||||
| mom_12_1_resid | 0.0552 | 1.98 | 35 | 497.7 | true |
|
||||
| mom_12_1 | 0.0531 | 1.61 | 35 | 497.7 | true |
|
||||
| **sue_latest** | **0.0172** | **0.6** | 44 | **47.4** | true |
|
||||
| fip_id | −0.045 | −2.91 | 35 | 497.7 | true |
|
||||
|
||||
#### Identical SUE subset (fair side-by-side — use this while coverage is thin)
|
||||
|
||||
| signal | mean_ic | ic_t_stat | weeks | avg_N |
|
||||
|---|---:|---:|---:|---:|
|
||||
| sue_latest | 0.0172 | 0.6 | 44 | 47.4 |
|
||||
| mom_12_1 | −0.0174 | −0.42 | 35 | 47.3 |
|
||||
| mom_12_1_resid | −0.0104 | −0.27 | 35 | 47.3 |
|
||||
|
||||
On the thin labeled subset, momentum itself is noise — so the subset is not yet
|
||||
a meaningful PEAD test.
|
||||
|
||||
#### Momentum-conditional SUE (top mom quintile)
|
||||
|
||||
| metric | value |
|
||||
|---|---:|
|
||||
| mean IC | **−0.0065** |
|
||||
| t | −0.1 |
|
||||
| weeks | 35 |
|
||||
|
||||
Wrong sign vs “ride positive surprises inside the momentum gate.”
|
||||
|
||||
**Iron rule:** fail (\|IC\| 0.017 < 0.03; t 0.6). **No promote.**
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
| piece | verdict |
|
||||
|---|---|
|
||||
| **2a earnings-gap** | **REPORT-ONLY** — no filter. Coverage too thin for risk claims; tails do not argue for an avoid-filter on n=4. |
|
||||
| **2b SUE** | **PARK** (effectively not green). Mild positive IC on ~48 names; fails iron bar; mom-conditional flat/negative. Re-score after full backfill before DEAD. |
|
||||
| **Production** | **no change** |
|
||||
|
||||
---
|
||||
|
||||
## What a human must decide next
|
||||
|
||||
1. Resume multi-day earnings backfill to **506/506**, then re-run
|
||||
`run_earnings_research.py` (heavy — MacBook OK).
|
||||
2. Do **not** ship an earnings-avoid entry filter from 2a.
|
||||
3. Do **not** wire SUE until a full-coverage IC clears the iron rule (and
|
||||
preferably mom-conditional > 0).
|
||||
4. Do not merge into main strategy docs without review.
|
||||
|
||||
---
|
||||
|
||||
## Implementation notes
|
||||
|
||||
| piece | role |
|
||||
|---|---|
|
||||
| `scripts/backfill_earnings_events.py` | bulk attempt → FMP/AV per-symbol; `earnings_events` + meta on snapshot |
|
||||
| `scripts/run_earnings_research.py` | 2a trade join + 2b SUE IC / mom-conditional |
|
||||
| Snapshot table `earnings_events` | real table (not SystemSetting JSON) |
|
||||
@@ -0,0 +1,108 @@
|
||||
# History-depth extension (Tier-1 alpha research)
|
||||
|
||||
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
|
||||
**Branch:** `research/history-depth-extension` (create from latest research stack).
|
||||
**Production impact:** none. **Do not retune any production knob on deep history.**
|
||||
|
||||
---
|
||||
|
||||
## Pre-registration (locked before rebuild)
|
||||
|
||||
### Motivation
|
||||
|
||||
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
|
||||
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
|
||||
the 2018 vol shock and full 2020 crash (where the feed allows).
|
||||
|
||||
### Protocol
|
||||
|
||||
1. **Empirical coverage first** — bars per calendar year per symbol; document
|
||||
where the feed thins out. Do **not** assume a uniform start date.
|
||||
2. **Rebuild the research snapshot completely** from prod source + max history
|
||||
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
|
||||
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
|
||||
`complete=true` and live counts match.
|
||||
4. **Re-run full signal harness** (all existing signals incl. sector residual /
|
||||
SUE if present) on the extended window.
|
||||
5. **Report per signal:** mean IC, t, window count, and **era split**
|
||||
(pre-/post-2021) — diagnostic only, **not a tuning input**.
|
||||
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
|
||||
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
|
||||
**relative** signal comparisons and IC stability, not levels.
|
||||
7. **Do not retune** production knobs. If a knob’s confirmation looks
|
||||
overturned on deep history → report only; human decides.
|
||||
|
||||
### Success / interpretation (not promotion of a new signal)
|
||||
|
||||
| outcome | meaning |
|
||||
|---|---|
|
||||
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
|
||||
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
|
||||
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
|
||||
| Any production knob looks worse deep | report; no auto-retune |
|
||||
|
||||
---
|
||||
|
||||
## MacBook runbook
|
||||
|
||||
```bash
|
||||
# 0. Repo + env
|
||||
git fetch origin
|
||||
git checkout research/earnings-gap-and-sue # or history-depth branch once pushed
|
||||
# ensure .env has ALPACA_* (and FMP if resuming earnings)
|
||||
|
||||
# 1. (Optional) finish earnings backfill first — multi-day free tier
|
||||
python scripts/backfill_earnings_events.py \
|
||||
--snapshot backtest_snapshots/prod.sqlite \
|
||||
--provider fmp --force-symbol --limit 250 --sleep 0.35
|
||||
|
||||
# 2. Coverage probe (before long rebuild)
|
||||
python scripts/run_history_depth_research.py --phase coverage \
|
||||
--snapshot backtest_snapshots/prod.sqlite
|
||||
|
||||
# 3. Full deep rebuild of research.sqlite (LONG — Alpaca per symbol)
|
||||
# Clears prior completion manifest; writes complete=true only at end.
|
||||
python scripts/extend_snapshot_universe.py \
|
||||
--source backtest_snapshots/prod.sqlite \
|
||||
--output backtest_snapshots/research.sqlite \
|
||||
--force-copy \
|
||||
--history-days 5000 \
|
||||
--min-bars 260 \
|
||||
--sleep 0.15
|
||||
|
||||
# 4. Also refresh SPY + sector ETFs to the same depth on BOTH snapshots
|
||||
python scripts/fetch_sector_etfs_to_snapshot.py \
|
||||
--snapshot backtest_snapshots/research.sqlite --history-days 5000
|
||||
python scripts/fetch_sector_etfs_to_snapshot.py \
|
||||
--snapshot backtest_snapshots/prod.sqlite --history-days 5000
|
||||
|
||||
# 5. Harness + era split (after race guard passes)
|
||||
python scripts/run_history_depth_research.py --phase harness \
|
||||
--snapshot backtest_snapshots/research.sqlite \
|
||||
--workers 8 --allow-spawn
|
||||
|
||||
# 6. Copy reports/ + docs/research/history-depth-extension.md results back
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data provenance
|
||||
|
||||
*(filled at run time)*
|
||||
|
||||
---
|
||||
|
||||
## Results
|
||||
|
||||
*(filled at run time)*
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
**Pending MacBook run.**
|
||||
|
||||
## What a human must decide next
|
||||
|
||||
- Do not retune production from deep history without explicit review.
|
||||
- Use relative IC stability to accept/reject Task 1 sector residual wire-in.
|
||||
@@ -0,0 +1,237 @@
|
||||
# Sector-residual momentum (Tier-1 alpha research)
|
||||
|
||||
**Status:** **PROMOTE (to human design decision only)** — IC + A/B bars cleared; **do not ship**.
|
||||
**Branch:** `research/sector-residual-momentum`
|
||||
**Production impact:** none. Local research only. No scheduler / gate / prod-config changes.
|
||||
**Artifacts:** `reports/sector-residual-20260719-083356.json` (+ companion `.md`)
|
||||
|
||||
---
|
||||
|
||||
## Pre-registration (locked before first research run)
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Residualizing 12–1 momentum against the sector, not only the market, reduces
|
||||
factor volatility at similar return (Blitz / Huij / Martens-style) → higher
|
||||
Sharpe on the production book when the residual replaces market-only residual
|
||||
as the momentum leg.
|
||||
|
||||
### Signals (candidates)
|
||||
|
||||
| signal | construction |
|
||||
|---|---|
|
||||
| `mom_12_1_sector_resid` | Two-factor residual vs SPY + ticker’s sector ETF. Same window as `mom_12_1_resid`: ≥100 daily obs, 252-bar lookback, 21-bar skip; two-factor OLS betas **without intercept**; cumulate residual returns over the formation window. |
|
||||
| `mom_12_1_sector_demeaned` | Plain `mom_12_1` minus the **cross-sectional** mean of `mom_12_1` within the same GICS sector that week (≥2 names in sector). No regression. |
|
||||
|
||||
### Baselines (same run, same cross-sections — iron rule)
|
||||
|
||||
Always report side-by-side with:
|
||||
|
||||
- `mom_12_1`
|
||||
- `mom_12_1_resid`
|
||||
|
||||
Computed on the **identical** weekly non-overlapping cross-sections in this run.
|
||||
Never compare against IC numbers from another report.
|
||||
|
||||
### Iron rule (IC harness)
|
||||
|
||||
Source of truth: `_signal_evaluation` in `app/services/backtest_service.py`.
|
||||
|
||||
- Mean weekly Spearman IC on **non-overlapping** weekly windows
|
||||
- Bar: \|mean IC\| ≥ ~0.03, **consistent positive sign**, `reliable: true` (≥ 12 windows)
|
||||
|
||||
### Promotion to portfolio A/B (candidate → book)
|
||||
|
||||
A candidate promotes to A/B **only if**:
|
||||
|
||||
1. It clears the iron-rule bar **and**
|
||||
2. Its IC **t-stat ≥** that of `mom_12_1_resid` on the same cross-sections.
|
||||
|
||||
### Portfolio A/B grading (if and only if IC promotion fires)
|
||||
|
||||
- Swap candidate in as the **momentum leg** of the production 80/20 momentum/vol
|
||||
rank **and** as the gate-percentile signal.
|
||||
- `fill_mode=close`, `COST_PER_SIDE = 0.001`, full config otherwise unchanged.
|
||||
- Validation window = entries ≥ **2024-07-01** (call it **validation**, not
|
||||
holdout — contaminated by prior experiments).
|
||||
- Pre-registered promotion bar:
|
||||
- validation Sharpe ≥ control − 0.5·SE
|
||||
- full-period Sharpe and max-DD **not worse** than control
|
||||
- Report Lo / Mertens-adjusted SEs.
|
||||
|
||||
### Optional sector-cap sub-experiment
|
||||
|
||||
Only if labels are in **and** A/B ran: max **3** positions per sector in the
|
||||
10-slot book. Same A/B grading. **Tail-trim presumption of guilt** (rule 4):
|
||||
report entry counts and both tails of the R distribution. Rising win rate with
|
||||
falling Sharpe/CAGR = red flag → do not promote.
|
||||
|
||||
**This run:** sector-cap arm **not executed** (optional; A/B unconstrained book
|
||||
only). Can be a human-approved follow-up.
|
||||
|
||||
### Verdict labels
|
||||
|
||||
| label | meaning |
|
||||
|---|---|
|
||||
| **PROMOTE** | Clears pre-registered bar; human decides next (wire design separate) |
|
||||
| **PARK** | Inconclusive / weak; keep machinery, no book change |
|
||||
| **DEAD** | Failed iron rule or worse than residual baseline with clear sign |
|
||||
|
||||
### Explicit non-goals
|
||||
|
||||
- No production deploy from this doc
|
||||
- Do not resurrect: take-profit exits, EV gate, regime entry-blocking,
|
||||
inverse-vol sizing, gap-caps, unconditional FIP filter
|
||||
|
||||
---
|
||||
|
||||
## Data provenance
|
||||
|
||||
### Snapshot race guard
|
||||
|
||||
| check | result |
|
||||
|---|---|
|
||||
| Snapshot path | `backtest_snapshots/prod.sqlite` |
|
||||
| Manifest | none (expected for prod snapshot); bar-count sanity applied |
|
||||
| Tickers / OHLCV | **506** / **629,263** |
|
||||
| Bars min / avg / max | 14 / 1246.1 / 1261 |
|
||||
| OHLCV range | 2021-06-24 → 2026-07-02 |
|
||||
| Partial-build red flags | none (avg bars healthy) |
|
||||
|
||||
Integrity fingerprint on same run: `fip_id` mean IC **−0.045** / t **−2.91**
|
||||
(35 weeks, N≈498) — matches the established prod fingerprint.
|
||||
|
||||
### Sector labels
|
||||
|
||||
| source | count |
|
||||
|---|---:|
|
||||
| Public S&P 500 GICS CSV | 496 newly filled |
|
||||
| FMP profile requests | 10 (all missing after CSV) |
|
||||
| Mapped / universe | **505 / 506 (99.8%)** |
|
||||
| With mappable ETF | 505 |
|
||||
| Still missing | **RHM** only |
|
||||
|
||||
Persist path: `data/research/ticker_sector_map.json`.
|
||||
|
||||
FMP aliases (`Technology`, `Consumer Defensive`, `Financial Services`) map to
|
||||
SPDRs via the alias table in `app/services/sector_map.py`.
|
||||
|
||||
### Sector ETFs in `benchmark_prices` (auxiliary only — not tradable)
|
||||
|
||||
| symbol | bars | min date | max date |
|
||||
|---|---:|---|---|
|
||||
| SPY | 1516 | 2020-07-06 | 2026-07-17 |
|
||||
| XLB…XLY (11) | 1512 each | 2020-07-10 | 2026-07-17 |
|
||||
|
||||
Fetched via Alpaca `Adjustment.SPLIT` into **`benchmark_prices`** (same table as
|
||||
SPY) so they never enter the ticker universe or candidate replay.
|
||||
|
||||
---
|
||||
|
||||
## Results
|
||||
|
||||
Generated: `2026-07-19T08:33:56`
|
||||
|
||||
### IC harness (identical cross-sections, production 506-name universe)
|
||||
|
||||
| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable | ic+_pct | quintile spread |
|
||||
|---|---:|---:|---:|---:|---|---:|---:|
|
||||
| **mom_12_1_sector_resid** | **0.0578** | **2.34** | 35 | 497.7 | true | 65.7 | 0.0245 |
|
||||
| mom_12_1_resid | 0.0552 | 1.98 | 35 | 497.7 | true | 60.0 | 0.0207 |
|
||||
| mom_12_1 | 0.0531 | 1.61 | 35 | 497.7 | true | 65.7 | 0.0206 |
|
||||
| mom_12_1_sector_demeaned | 0.0340 | 1.32 | 35 | 496.7 | true | 62.9 | 0.0154 |
|
||||
|
||||
### IC promotion grades
|
||||
|
||||
| candidate | iron rule | t ≥ resid | promote_to_ab |
|
||||
|---|---|---|---|
|
||||
| `mom_12_1_sector_resid` | pass (IC 0.058, +sign, reliable) | **yes** (2.34 ≥ 1.98) | **yes** |
|
||||
| `mom_12_1_sector_demeaned` | pass (IC 0.034, +sign, reliable) | **no** (1.32 < 1.98) | **no** |
|
||||
|
||||
### Portfolio A/B — `mom_12_1_sector_resid` as residual leg
|
||||
|
||||
Config: production 80/20 residual/high-vol rank + gate percentile, `fill_mode=close`,
|
||||
cost 10 bps/side, ATR trail / gate-reset re-entry as live. Validation split
|
||||
2024-07-01.
|
||||
|
||||
| window | arm | Sharpe | Sharpe SE (Mertens) | CAGR % | max DD % | trades | n_days |
|
||||
|---|---|---:|---:|---:|---:|---:|---:|
|
||||
| train | control (resid) | 1.30 | 0.685 | 29.2 | 21.4 | 176 | 525 |
|
||||
| train | treatment (sector resid) | **1.57** | 0.677 | **35.5** | **19.8** | 176 | 530 |
|
||||
| validation | control | **2.92** | 0.709 | **76.3** | **11.7** | 150 | 501 |
|
||||
| validation | treatment | 2.57 | 0.701 | 66.3 | 14.8 | 163 | 501 |
|
||||
| full | control | 2.09 | 0.497 | 51.6 | 21.4 | 322 | 1000 |
|
||||
| full | treatment | 2.09 | 0.491 | 51.0 | **19.8** | 337 | 1005 |
|
||||
|
||||
**Pre-registered A/B checks**
|
||||
|
||||
| check | result |
|
||||
|---|---|
|
||||
| val Sharpe ≥ control − 0.5·SE | **pass** (2.57 ≥ 2.92 − 0.5×0.701 = 2.5695) — **knife-edge** |
|
||||
| full Sharpe not worse | **pass** (2.09 = 2.09) |
|
||||
| full max DD not worse | **pass** (19.8 < 21.4) |
|
||||
|
||||
Qualified long candidates: control 1086 vs treatment 1210 (sector residual
|
||||
gates a slightly larger set).
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
| signal | verdict | note |
|
||||
|---|---|---|
|
||||
| **`mom_12_1_sector_resid`** | **PROMOTE → human wire-in decision** | IC modestly beats market residual; A/B clears pre-reg bar narrowly. **Do not ship from this branch.** |
|
||||
| **`mom_12_1_sector_demeaned`** | **DEAD** (for promotion) | Iron-rule IC magnitude ok, but t-stat loses to `mom_12_1_resid`. Cheap variant not competitive. |
|
||||
|
||||
### Read carefully (for the human)
|
||||
|
||||
1. **IC edge is real but small.** Sector residual IC 0.0578 / t 2.34 vs market
|
||||
residual 0.0552 / t 1.98 on the **same** 35 windows — better consistency
|
||||
(ic+ 65.7% vs 60%) and slightly higher mean, not a different factor class.
|
||||
2. **A/B is not a clear Sharpe win.** Full-period Sharpe is flat (2.09).
|
||||
Validation Sharpe is **lower** than control (2.57 vs 2.92) and only clears
|
||||
the pre-registered “within 0.5 SE” cushion by ~0.001. Train improves;
|
||||
validation worsens — classic regime-split noise on ~2 years.
|
||||
3. **Risk side is friendly.** Full max DD improves (19.8% vs 21.4%); train DD
|
||||
also better. Matches the “lower factor vol” half of the hypothesis more than
|
||||
the “higher Sharpe” half on this window.
|
||||
4. **Survivorship / short history.** Same caveats as all current research:
|
||||
today’s constituents, ~35 independent weekly windows, one post-2021 regime
|
||||
dominant. Task 3 (history depth) should re-check IC stability before any
|
||||
wire-in.
|
||||
5. **Not shipped.** Machinery lives on the research branch; production residual
|
||||
path is untouched.
|
||||
|
||||
---
|
||||
|
||||
## What a human must decide next
|
||||
|
||||
1. **Accept or reject** replacing `mom_12_1_resid` with `mom_12_1_sector_resid`
|
||||
as the production residual (gate + 80/20 mom leg), **or** keep market residual
|
||||
and treat sector residual as research-only.
|
||||
2. If leaning accept: require **Task 3 history-depth** confirmation (IC era split
|
||||
pre/post-2021) before any production PR.
|
||||
3. Optional: run **sector-cap ≤3** A/B with full tail diagnostics (not run here).
|
||||
4. **Do not** merge this verdict into main strategy docs without review.
|
||||
5. Wire-in design (live sector map refresh, ETF series ops, fallback when sector
|
||||
missing) is a **separate** approved engineering step.
|
||||
|
||||
---
|
||||
|
||||
## Implementation notes (research machinery)
|
||||
|
||||
| piece | role |
|
||||
|---|---|
|
||||
| `app/services/sector_map.py` | GICS→ETF map, symbol normalise, JSON load/save |
|
||||
| `app/services/backtest_service.py` | multi-factor residual; `mom_12_1_sector_resid` in `_signal_values`; demean inject |
|
||||
| `scripts/build_ticker_sector_map.py` | SP500 CSV + FMP gap fill |
|
||||
| `scripts/fetch_sector_etfs_to_snapshot.py` | Alpaca → snapshot `benchmark_prices` |
|
||||
| `scripts/run_sector_residual_research.py` | race guard, IC, optional A/B, reports |
|
||||
| `data/research/ticker_sector_map.json` | persisted labels (research only) |
|
||||
|
||||
---
|
||||
|
||||
## Artifacts
|
||||
|
||||
- JSON: `reports/sector-residual-20260719-083356.json`
|
||||
- MD copy: `reports/sector-residual-20260719-083356.md`
|
||||
Reference in New Issue
Block a user