signal-platform/docs/research/history-depth-extension.md

History-depth extension (Tier-1 alpha research)

Status: PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
Branch: research/history-depth-extension (create from latest research stack).
Production impact: none. Do not retune any production knob on deep history.

---

Pre-registration (locked before rebuild)

Motivation

All current conclusions rest on ~35 non-overlapping weekly windows in essentially one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds the 2018 vol shock and full 2020 crash (where the feed allows).

Protocol

1. Empirical coverage first — bars per calendar year per symbol; document where the feed thins out. Do not assume a uniform start date.
2. Rebuild the research snapshot completely from prod source + max history per symbol (Adjustment.SPLIT, ~200 req/min pacing via existing extender).
3. Race guard (rule 6) — refuse analysis until completion manifest is complete=true and live counts match.
4. Re-run full signal harness (all existing signals incl. sector residual / SUE if present) on the extended window.
5. Report per signal: mean IC, t, window count, and era split (pre-/post-2021) — diagnostic only, not a tuning input.
6. Log prominently: survivorship bias grows with depth (today’s constituents backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is relative signal comparisons and IC stability, not levels.
7. Do not retune production knobs. If a knob’s confirmation looks overturned on deep history → report only; human decides.

Success / interpretation (not promotion of a new signal)

outcome	meaning
Sector residual still ≥ market residual on deep IC + stable sign	strengthens Task 1 PROMOTE case
Sector residual collapses pre-2021	PARK Task 1 wire-in
SUE remains weak after full earnings + depth	DEAD SUE for this stack
Any production knob looks worse deep	report; no auto-retune

---

MacBook runbook

# 0. Repo + env
git fetch origin
git checkout research/earnings-gap-and-sue   # or history-depth branch once pushed
# ensure .env has ALPACA_* (and FMP if resuming earnings)

# 1. (Optional) finish earnings backfill first — multi-day free tier
python scripts/backfill_earnings_events.py \
  --snapshot backtest_snapshots/prod.sqlite \
  --provider fmp --force-symbol --limit 250 --sleep 0.35

# 2. Coverage probe (before long rebuild)
python scripts/run_history_depth_research.py --phase coverage \
  --snapshot backtest_snapshots/prod.sqlite

# 3. Full deep rebuild of research.sqlite (LONG — Alpaca per symbol)
#    Clears prior completion manifest; writes complete=true only at end.
python scripts/extend_snapshot_universe.py \
  --source backtest_snapshots/prod.sqlite \
  --output backtest_snapshots/research.sqlite \
  --force-copy \
  --history-days 5000 \
  --min-bars 260 \
  --sleep 0.15

# 4. Also refresh SPY + sector ETFs to the same depth on BOTH snapshots
python scripts/fetch_sector_etfs_to_snapshot.py \
  --snapshot backtest_snapshots/research.sqlite --history-days 5000
python scripts/fetch_sector_etfs_to_snapshot.py \
  --snapshot backtest_snapshots/prod.sqlite --history-days 5000

# 5. Harness + era split (after race guard passes)
python scripts/run_history_depth_research.py --phase harness \
  --snapshot backtest_snapshots/research.sqlite \
  --workers 8 --allow-spawn

# 6. Copy reports/ + docs/research/history-depth-extension.md results back

---

Data provenance

(filled at run time)

---

Results

(filled at run time)

---

Verdict

Pending MacBook run.

What a human must decide next

• Do not retune production from deep history without explicit review.
• Use relative IC stability to accept/reject Task 1 sector residual wire-in.