dennisthiessen
9717d8176b
Deep masked retest failed the iron IC bar (0.027 < 0.03). Log as rejected #13 in research README; close sector-residual and history-depth docs. Production market residual unchanged.
204 lines
9.0 KiB
Markdown
204 lines
9.0 KiB
Markdown
# History-depth extension (Tier-1 alpha research)
|
||
|
||
**Status:** **CLOSED.** Sector-residual deep test **FAIL** — Task 1 archived as rejected (see supersession).
|
||
**Branch:** `research/earnings-gap-and-sue`
|
||
**Superseded artifact (do not cite):** `reports/history-depth-20260719-103315.json` — **UNMASKED, TWO-TIER SNAPSHOT**
|
||
**Authoritative sector grade:** `reports/sector-resid-deep-20260719-113319.json`
|
||
**Production impact:** none. **Do not retune any production knob on deep history.**
|
||
|
||
---
|
||
|
||
## Pre-registration (locked before rebuild)
|
||
|
||
### Motivation
|
||
|
||
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
|
||
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
|
||
the 2018 vol shock and full 2020 crash (where the feed allows).
|
||
|
||
### Protocol
|
||
|
||
1. **Empirical coverage first** — bars per calendar year per symbol; document
|
||
where the feed thins out. Do **not** assume a uniform start date.
|
||
2. **Rebuild the research snapshot completely** from prod source + max history
|
||
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
|
||
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
|
||
`complete=true` and live counts match.
|
||
4. **Re-run full signal harness** (all existing signals incl. sector residual /
|
||
SUE if present) on the extended window.
|
||
5. **Report per signal:** mean IC, t, window count, and **era split**
|
||
(pre-/post-2021) — diagnostic only, **not a tuning input**.
|
||
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
|
||
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
|
||
**relative** signal comparisons and IC stability, not levels.
|
||
7. **Do not retune** production knobs. If a knob’s confirmation looks
|
||
overturned on deep history → report only; human decides.
|
||
|
||
### Success / interpretation (not promotion of a new signal)
|
||
|
||
| outcome | meaning |
|
||
|---|---|
|
||
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
|
||
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
|
||
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
|
||
| Any production knob looks worse deep | report; no auto-retune |
|
||
|
||
---
|
||
|
||
## Data provenance
|
||
|
||
| check | result |
|
||
|---|---|
|
||
| Snapshot | MacBook `research.sqlite` |
|
||
| Manifest `complete` | **true** (finished 2026-07-19T08:19Z) |
|
||
| Live counts match | yes — 4655 tickers / 6,609,926 OHLCV / 4149 rank_only |
|
||
| `history_days` | 5000 |
|
||
| fetch_ok / fail | 4152 / 0 |
|
||
| Race guard | **pass** |
|
||
|
||
> **SURVIVORSHIP BIAS:** today’s constituents backfilled historically. Absolute
|
||
> Sharpe/CAGR levels on deep history are optimistic. Use **relative** signal IC
|
||
> comparisons and era stability only — not levels.
|
||
|
||
**Coverage JSON was empty in the auto-written doc** (harness-only phase after
|
||
rebuild). Manifest is the race-guard source of truth for this run.
|
||
|
||
Earlier MacBook files `history-depth-20260719-093853` … `095156` are intermediate
|
||
/ incomplete passes — **do not cite**. Only **103315** is authoritative.
|
||
|
||
---
|
||
|
||
## Results (authoritative: 103315)
|
||
|
||
### Full-window signal IC (broad research universe, deep bars)
|
||
|
||
| signal | mean_ic | t | weeks | avg_N | notes |
|
||
|---|---:|---:|---:|---:|---|
|
||
| high_52w | **0.111** | **6.41** | 84 | 2280 | strong on deep breadth |
|
||
| mom_12_1 | **0.066** | **5.11** | 83 | 2278 | raw momentum strong |
|
||
| trend_200 | 0.055 | 4.30 | 85 | 2308 | |
|
||
| mom_6_1 | 0.049 | 4.91 | 88 | 2376 | |
|
||
| mom_3_1 | 0.036 | 3.58 | 90 | 2426 | |
|
||
| fip_id | **+0.027** | **3.25** | 83 | 2278 | **sign flip vs prod fingerprint** |
|
||
| mom_12_1_resid | 0.026 | 2.21 | 83 | 2278 | market residual still + but weaker than raw |
|
||
| reversal_1m | ~0 | 0.31 | 91 | 2443 | dead |
|
||
| vol_6m | **−0.123** | **−6.34** | 88 | 2376 | low-vol anomaly strong |
|
||
| mom_12_1_sector_resid | 0.058 | 2.34 | **35** | **498** | **not deep-sample — see caveats** |
|
||
| mom_12_1_sector_demeaned | 0.034 | 1.32 | **35** | **497** | same short fingerprint |
|
||
|
||
### Era split (diagnostic only — not a tuning input)
|
||
|
||
| signal | pre-2021 IC / t / w / N | post-2021 IC / t / w / N |
|
||
|---|---|---|
|
||
| mom_12_1 | +0.041 / 2.94 / 36 / 1425 | +0.079 / 3.64 / 48 / 2913 |
|
||
| mom_12_1_resid | +0.023 / 1.68 / 36 / 1425 | +0.027 / 1.37 / 48 / 2913 |
|
||
| fip_id | +0.012 / 1.35 / 36 / 1425 | +0.037 / 2.91 / 48 / 2913 |
|
||
| vol_6m | −0.056 / −2.16 / 40 / 1461 | −0.162 / −4.94 / 48 / 3123 |
|
||
| high_52w | +0.049 / 2.0 / 36 / 1422 | +0.138 / 4.34 / 48 / 2915 |
|
||
| **sector_resid** | **absent** | 0.058 / 2.34 / 35 / 498 (short only) |
|
||
| **sector_demeaned** | **absent** | 0.034 / 1.32 / 35 / 497 (short only) |
|
||
|
||
---
|
||
|
||
## Critical caveats (must read)
|
||
|
||
### 1. Sector residual did **not** get a deep-history stress test
|
||
|
||
`mom_12_1_sector_resid` / `_demeaned` still show **exactly** the Task‑1 short-window
|
||
fingerprint: **35 weeks, N≈498, IC 0.0578, t 2.34**.
|
||
|
||
On the same run, raw `mom_12_1` has **83 weeks, N≈2278**. So depth worked for
|
||
price-only signals, but sector residual is still limited to the **~505 labeled
|
||
prod names × short factor calendar** (sector map only covers prod; and/or sector
|
||
ETF / two-factor path did not extend usable residual weeks).
|
||
|
||
**Pre-registered rule:** “Sector residual collapses pre-2021 → PARK Task 1
|
||
wire-in.” Pre-2021 sector residual is **absent** from the era table. That is a
|
||
**PARK**, not a confirmation of the short-window PROMOTE.
|
||
|
||
Do **not** claim “sector residual beats market residual on deep history” from
|
||
this table — the two rows are **not the same cross-section or window count**.
|
||
|
||
### 2. `fip_id` sign flips vs production fingerprint
|
||
|
||
| sample | fip mean IC | t |
|
||
|---|---:|---:|
|
||
| Prod 505, ~5y (fingerprint) | **−0.045** | −2.91 |
|
||
| Research breadth, deep (this run) | **+0.027** | +3.25 |
|
||
|
||
This does **not** authorize resurrecting unconditional FIP as a book filter. It
|
||
confirms earlier Phase‑B caution: FIP edge is **universe- and sample-dependent**.
|
||
Production display card can stay context-only. Nested lookbacks still not OOS.
|
||
|
||
### 3. Market residual vs raw momentum on deep breadth
|
||
|
||
On deep broad IC, **raw 12‑1 (0.066 / t 5.1) ≫ market residual (0.026 / t 2.2)**.
|
||
That does **not** by itself overturn production residual ranking (book A/B was
|
||
on 505 + GTL gate, not pure factor IC), but it is a yellow flag for “residual is
|
||
always the better rank key” stories on broad history. **No auto-retune.**
|
||
|
||
### 4. Low-vol anomaly is the cleanest deep-history result
|
||
|
||
`vol_6m` IC −0.12 / t −6.3 full; stronger post-2021. Consistent sign across eras.
|
||
Production already blends **high**-vol (not low-vol) into the 80/20 rank — this
|
||
report does not change that without a separate A/B. Flag for human awareness only.
|
||
|
||
---
|
||
|
||
## Verdicts (vs pre-registration)
|
||
|
||
| question | verdict |
|
||
|---|---|
|
||
| Task 1 sector residual wire-in | **PARK** — no pre-2021 sector residual; deep-sample IC not established; short-window PROMOTE stays “human design only,” **not strengthened** by this run |
|
||
| Sector demean | still **DEAD** for promotion (t 1.32, short only) |
|
||
| SUE | **not re-scored here** (no `sue_latest` in harness table) — leave Task 2 **PARK** until full earnings backfill |
|
||
| fip unconditional book filter | remains **rejected / parked** despite sign flip on broad deep sample |
|
||
| Production residual / 80/20 / trail knobs | **no retune** from this report |
|
||
| Overall Task 3 | **COMPLETE as diagnostic** — payload is relative IC + caveats above |
|
||
|
||
---
|
||
|
||
## What a human must decide next
|
||
|
||
1. **Sector residual — decided:** CLOSED / REJECTED (archive complete). No wire-in.
|
||
2. **Do not** retune residual vs raw, FIP, or vol blend from deep IC tables
|
||
without a pre-registered book A/B on the intended universe.
|
||
3. Optional (separate threads only): finish earnings backfill and re-run SUE;
|
||
snapshot per-symbol depth guard as tooling.
|
||
|
||
---
|
||
|
||
## Artifacts
|
||
|
||
| file | role |
|
||
|---|---|
|
||
| `reports/history-depth-20260719-103315.json` | **authoritative** |
|
||
| `reports/history-depth-20260719-103315.md` | companion dump |
|
||
| `reports/history-depth-20260719-093853` … `095156` | **ignore** (partial) |
|
||
|
||
---
|
||
|
||
## Supersession notice (2026-07-19 sector-resid deep test)
|
||
|
||
The table and interpretation from **`history-depth-20260719-103315`** are **UNMASKED, TWO-TIER SNAPSHOT — superseded, directional only, do not cite**. Prod-universe names (and sector residual coverage) were left shallow while breadth names were deepened; sector residual weeks=35 was a data gap.
|
||
|
||
### Sector-residual deep test outcome: **FAIL** (archived)
|
||
|
||
**Task 1 CLOSED / REJECTED** — sector residual dead on deep evidence. Archived in
|
||
the research log rejected table (#13). Do not resurrect without a new
|
||
pre-registered protocol.
|
||
|
||
| check | result |
|
||
|---|---|
|
||
| weeks | **83** (data fix worked) |
|
||
| mean IC | **0.0268** (below 0.03 bar) → FAIL |
|
||
| t vs resid same CS | 1.69 ≥ 1.30 pass |
|
||
| era signs | both + pass |
|
||
|
||
- Artifact: `reports/sector-resid-deep-20260719-113319.json`
|
||
- Summary write-up: [sector-residual-momentum.md](sector-residual-momentum.md)
|
||
|
||
**Future snapshot rebuilds must verify per-symbol depth** (earliest-bar
|
||
uniformity across the intended universe) — guard is a to-do, not part of this
|
||
order.
|