tests done

2026-07-19 10:40:19 +02:00
parent 06cf054f60
commit f6e0ca734f
12 changed files with 1801 additions and 5 deletions
@@ -0,0 +1,208 @@
+# History-depth extension (Tier-1 alpha research)
+
+**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).  
+**Branch:** `research/history-depth-extension` (create from latest research stack).  
+**Production impact:** none. **Do not retune any production knob on deep history.**
+
+---
+
+## Pre-registration (locked before rebuild)
+
+### Motivation
+
+All current conclusions rest on ~35 non-overlapping weekly windows in essentially
+one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
+the 2018 vol shock and full 2020 crash (where the feed allows).
+
+### Protocol
+
+1. **Empirical coverage first** — bars per calendar year per symbol; document
+   where the feed thins out. Do **not** assume a uniform start date.
+2. **Rebuild the research snapshot completely** from prod source + max history
+   per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
+3. **Race guard (rule 6)** — refuse analysis until completion manifest is
+   `complete=true` and live counts match.
+4. **Re-run full signal harness** (all existing signals incl. sector residual /
+   SUE if present) on the extended window.
+5. **Report per signal:** mean IC, t, window count, and **era split**
+   (pre-/post-2021) — diagnostic only, **not a tuning input**.
+6. **Log prominently:** survivorship bias grows with depth (today’s constituents
+   backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
+   **relative** signal comparisons and IC stability, not levels.
+7. **Do not retune** production knobs. If a knob’s confirmation looks
+   overturned on deep history → report only; human decides.
+
+### Success / interpretation (not promotion of a new signal)
+
+| outcome | meaning |
+|---|---|
+| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
+| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
+| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
+| Any production knob looks worse deep | report; no auto-retune |
+
+---
+
+## MacBook runbook
+
+Prefer the bundled script (one entry point):
+
+```bash
+git fetch origin && git checkout research/earnings-gap-and-sue
+# .env: ALPACA_* required; FMP_* if resuming earnings
+# copy backtest_snapshots/prod.sqlite if not already local
+
+chmod +x scripts/run_tier1_macbook.sh
+
+# Default: coverage → deep rebuild → harness (+ era split)
+./scripts/run_tier1_macbook.sh
+
+# Optional variants
+./scripts/run_tier1_macbook.sh --all            # + earnings resume first
+./scripts/run_tier1_macbook.sh --earnings-only  # multi-day FMP + 2a/2b only
+./scripts/run_tier1_macbook.sh --harness-only   # skip rebuild
+./scripts/run_tier1_macbook.sh --coverage-only
+
+# Tunables
+WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
+./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
+```
+
+Then commit `reports/` + updated research docs, or copy them back to Windows.
+
+---
+
+## Data provenance
+
+*(filled at run time)*
+
+---
+
+## Results
+
+Generated: `2026-07-19T10:33:15.322673`
+
+> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
+
+### Coverage
+
+```json
+{}
+```
+
+### Race guard
+
+```json
+{
+  "manifest": {
+    "schema_version": 1,
+    "snapshot": "research.sqlite",
+    "snapshot_resolved": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/research.sqlite",
+    "complete": true,
+    "finished_at": "2026-07-19T08:19:02.992206+00:00",
+    "ticker_count": 4655,
+    "ohlcv_row_count": 6609926,
+    "rank_only_count": 4149,
+    "sources": {
+      "nasdaq_all": "nasdaq_trader",
+      "sp500": "wikipedia_sp500"
+    },
+    "history_days": 5000,
+    "min_bars": 260,
+    "fetch_ok": 4152,
+    "fetch_fail": 0,
+    "limit": null,
+    "extra": {
+      "prod_symbols_at_start": 506,
+      "pool_size": 4648,
+      "to_fetch": 4152
+    },
+    "live_counts": {
+      "ticker_count": 4655,
+      "ohlcv_row_count": 6609926,
+      "rank_only_count": 4149
+    }
+  },
+  "ok": true
+}
+```
+
+### Signal IC (full extended window)
+
+| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable |
+|---|---:|---:|---:|---:|---|
+| high_52w | 0.1111 | 6.41 | 84 | 2280.3 | True |
+| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 | True |
+| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | True |
+| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 | True |
+| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 | True |
+| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 | True |
+| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 | True |
+| fip_id | 0.0267 | 3.25 | 83 | 2277.5 | True |
+| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 | True |
+| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 | True |
+| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 | True |
+
+### Era split (diagnostic only)
+
+#### full
+
+| signal | mean_ic | t | weeks | N |
+|---|---:|---:|---:|---:|
+| fip_id | 0.0267 | 3.25 | 83 | 2277.5 |
+| high_52w | 0.1111 | 6.41 | 84 | 2280.3 |
+| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 |
+| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 |
+| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
+| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
+| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 |
+| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 |
+| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 |
+| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 |
+| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 |
+
+#### pre_2021
+
+| signal | mean_ic | t | weeks | N |
+|---|---:|---:|---:|---:|
+| fip_id | 0.0116 | 1.35 | 36 | 1424.8 |
+| high_52w | 0.0494 | 2.0 | 36 | 1422.2 |
+| mom_12_1 | 0.0413 | 2.94 | 36 | 1424.8 |
+| mom_12_1_resid | 0.0226 | 1.68 | 36 | 1424.8 |
+| mom_3_1 | 0.0217 | 1.84 | 42 | 1480.0 |
+| mom_6_1 | 0.0205 | 1.63 | 40 | 1461.2 |
+| reversal_1m | 0.0019 | 0.15 | 44 | 1501.4 |
+| trend_200 | 0.0322 | 2.23 | 38 | 1440.9 |
+| vol_6m | -0.056 | -2.16 | 40 | 1461.2 |
+
+#### post_2021
+
+| signal | mean_ic | t | weeks | N |
+|---|---:|---:|---:|---:|
+| fip_id | 0.0366 | 2.91 | 48 | 2913.0 |
+| high_52w | 0.1375 | 4.34 | 48 | 2914.5 |
+| mom_12_1 | 0.0791 | 3.64 | 48 | 2913.0 |
+| mom_12_1_resid | 0.0265 | 1.37 | 48 | 2913.0 |
+| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
+| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
+| mom_3_1 | 0.0291 | 1.54 | 48 | 3234.2 |
+| mom_6_1 | 0.0779 | 4.36 | 48 | 3122.6 |
+| reversal_1m | -0.0126 | -0.72 | 48 | 3315.8 |
+| trend_200 | 0.0585 | 2.68 | 48 | 3001.5 |
+| vol_6m | -0.1623 | -4.94 | 48 | 3122.6 |
+
+
+## Verdict
+
+**PENDING_HUMAN**
+
+Harness complete — human interprets relative IC / era stability. No production retune from this artifact.
+
+## What a human must decide next
+
+- Compare sector residual vs market residual across eras.
+- If pre-2021 IC collapses, park Task 1 wire-in.
+- Do not retune production knobs on deep history levels.
+
+Artifacts: `reports/history-depth-20260719-103315.json`
+