178 lines
5.1 KiB
Markdown
178 lines
5.1 KiB
Markdown
# History-depth extension (Tier-1 alpha research)
|
||
|
||
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
|
||
**Branch:** `research/history-depth-extension` (create from latest research stack).
|
||
**Production impact:** none. **Do not retune any production knob on deep history.**
|
||
|
||
---
|
||
|
||
## Pre-registration (locked before rebuild)
|
||
|
||
### Motivation
|
||
|
||
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
|
||
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
|
||
the 2018 vol shock and full 2020 crash (where the feed allows).
|
||
|
||
### Protocol
|
||
|
||
1. **Empirical coverage first** — bars per calendar year per symbol; document
|
||
where the feed thins out. Do **not** assume a uniform start date.
|
||
2. **Rebuild the research snapshot completely** from prod source + max history
|
||
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
|
||
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
|
||
`complete=true` and live counts match.
|
||
4. **Re-run full signal harness** (all existing signals incl. sector residual /
|
||
SUE if present) on the extended window.
|
||
5. **Report per signal:** mean IC, t, window count, and **era split**
|
||
(pre-/post-2021) — diagnostic only, **not a tuning input**.
|
||
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
|
||
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
|
||
**relative** signal comparisons and IC stability, not levels.
|
||
7. **Do not retune** production knobs. If a knob’s confirmation looks
|
||
overturned on deep history → report only; human decides.
|
||
|
||
### Success / interpretation (not promotion of a new signal)
|
||
|
||
| outcome | meaning |
|
||
|---|---|
|
||
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
|
||
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
|
||
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
|
||
| Any production knob looks worse deep | report; no auto-retune |
|
||
|
||
---
|
||
|
||
## MacBook runbook
|
||
|
||
Prefer the bundled script (one entry point):
|
||
|
||
```bash
|
||
git fetch origin && git checkout research/earnings-gap-and-sue
|
||
# .env: ALPACA_* required; FMP_* if resuming earnings
|
||
# copy backtest_snapshots/prod.sqlite if not already local
|
||
|
||
chmod +x scripts/run_tier1_macbook.sh
|
||
|
||
# Default: coverage → deep rebuild → harness (+ era split)
|
||
./scripts/run_tier1_macbook.sh
|
||
|
||
# Optional variants
|
||
./scripts/run_tier1_macbook.sh --all # + earnings resume first
|
||
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
|
||
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
|
||
./scripts/run_tier1_macbook.sh --coverage-only
|
||
|
||
# Tunables
|
||
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
|
||
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
|
||
```
|
||
|
||
Then commit `reports/` + updated research docs, or copy them back to Windows.
|
||
|
||
---
|
||
|
||
## Data provenance
|
||
|
||
*(filled at run time)*
|
||
|
||
---
|
||
|
||
## Results
|
||
|
||
Generated: `2026-07-19T09:38:53.936226`
|
||
|
||
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
|
||
|
||
### Coverage
|
||
|
||
```json
|
||
{
|
||
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
|
||
"ticker_count": 506,
|
||
"ohlcv_row_count": 629263,
|
||
"date_range": {
|
||
"min": "2021-06-24",
|
||
"max": "2026-07-02"
|
||
},
|
||
"bars_per_year": [
|
||
{
|
||
"year": "2021",
|
||
"bars": 65678,
|
||
"tickers_with_bars": 494
|
||
},
|
||
{
|
||
"year": "2022",
|
||
"bars": 124423,
|
||
"tickers_with_bars": 497
|
||
},
|
||
{
|
||
"year": "2023",
|
||
"bars": 124479,
|
||
"tickers_with_bars": 499
|
||
},
|
||
{
|
||
"year": "2024",
|
||
"bars": 126129,
|
||
"tickers_with_bars": 501
|
||
},
|
||
{
|
||
"year": "2025",
|
||
"bars": 125615,
|
||
"tickers_with_bars": 504
|
||
},
|
||
{
|
||
"year": "2026",
|
||
"bars": 62939,
|
||
"tickers_with_bars": 505
|
||
}
|
||
],
|
||
"bars_per_symbol": {
|
||
"min": 14,
|
||
"p10": 1261,
|
||
"p50": 1261,
|
||
"p90": 1261,
|
||
"max": 1261
|
||
},
|
||
"symbols_by_start_year": {
|
||
"2021": 494,
|
||
"2022": 3,
|
||
"2023": 2,
|
||
"2024": 2,
|
||
"2025": 3,
|
||
"2026": 1
|
||
},
|
||
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
|
||
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
|
||
}
|
||
```
|
||
|
||
### Race guard
|
||
|
||
```json
|
||
{}
|
||
```
|
||
|
||
### Signal IC (full extended window)
|
||
|
||
_Harness not run this pass._
|
||
|
||
### Era split (diagnostic only)
|
||
|
||
_No era split._
|
||
|
||
## Verdict
|
||
|
||
**COVERAGE_ONLY**
|
||
|
||
Coverage probe only; run --phase harness after deep rebuild.
|
||
|
||
## What a human must decide next
|
||
|
||
- Compare sector residual vs market residual across eras.
|
||
- If pre-2021 IC collapses, park Task 1 wire-in.
|
||
- Do not retune production knobs on deep history levels.
|
||
|
||
Artifacts: `reports/history-depth-20260719-093853.json`
|
||
|