tests done

2026-07-19 10:40:19 +02:00
parent 06cf054f60
commit f6e0ca734f
12 changed files with 1801 additions and 5 deletions
@@ -0,0 +1,177 @@
+# History-depth extension (Tier-1 alpha research)
+
+**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).  
+**Branch:** `research/history-depth-extension` (create from latest research stack).  
+**Production impact:** none. **Do not retune any production knob on deep history.**
+
+---
+
+## Pre-registration (locked before rebuild)
+
+### Motivation
+
+All current conclusions rest on ~35 non-overlapping weekly windows in essentially
+one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
+the 2018 vol shock and full 2020 crash (where the feed allows).
+
+### Protocol
+
+1. **Empirical coverage first** — bars per calendar year per symbol; document
+   where the feed thins out. Do **not** assume a uniform start date.
+2. **Rebuild the research snapshot completely** from prod source + max history
+   per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
+3. **Race guard (rule 6)** — refuse analysis until completion manifest is
+   `complete=true` and live counts match.
+4. **Re-run full signal harness** (all existing signals incl. sector residual /
+   SUE if present) on the extended window.
+5. **Report per signal:** mean IC, t, window count, and **era split**
+   (pre-/post-2021) — diagnostic only, **not a tuning input**.
+6. **Log prominently:** survivorship bias grows with depth (today’s constituents
+   backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
+   **relative** signal comparisons and IC stability, not levels.
+7. **Do not retune** production knobs. If a knob’s confirmation looks
+   overturned on deep history → report only; human decides.
+
+### Success / interpretation (not promotion of a new signal)
+
+| outcome | meaning |
+|---|---|
+| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
+| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
+| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
+| Any production knob looks worse deep | report; no auto-retune |
+
+---
+
+## MacBook runbook
+
+Prefer the bundled script (one entry point):
+
+```bash
+git fetch origin && git checkout research/earnings-gap-and-sue
+# .env: ALPACA_* required; FMP_* if resuming earnings
+# copy backtest_snapshots/prod.sqlite if not already local
+
+chmod +x scripts/run_tier1_macbook.sh
+
+# Default: coverage → deep rebuild → harness (+ era split)
+./scripts/run_tier1_macbook.sh
+
+# Optional variants
+./scripts/run_tier1_macbook.sh --all            # + earnings resume first
+./scripts/run_tier1_macbook.sh --earnings-only  # multi-day FMP + 2a/2b only
+./scripts/run_tier1_macbook.sh --harness-only   # skip rebuild
+./scripts/run_tier1_macbook.sh --coverage-only
+
+# Tunables
+WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
+./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
+```
+
+Then commit `reports/` + updated research docs, or copy them back to Windows.
+
+---
+
+## Data provenance
+
+*(filled at run time)*
+
+---
+
+## Results
+
+Generated: `2026-07-19T09:38:53.936226`
+
+> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
+
+### Coverage
+
+```json
+{
+  "snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
+  "ticker_count": 506,
+  "ohlcv_row_count": 629263,
+  "date_range": {
+    "min": "2021-06-24",
+    "max": "2026-07-02"
+  },
+  "bars_per_year": [
+    {
+      "year": "2021",
+      "bars": 65678,
+      "tickers_with_bars": 494
+    },
+    {
+      "year": "2022",
+      "bars": 124423,
+      "tickers_with_bars": 497
+    },
+    {
+      "year": "2023",
+      "bars": 124479,
+      "tickers_with_bars": 499
+    },
+    {
+      "year": "2024",
+      "bars": 126129,
+      "tickers_with_bars": 501
+    },
+    {
+      "year": "2025",
+      "bars": 125615,
+      "tickers_with_bars": 504
+    },
+    {
+      "year": "2026",
+      "bars": 62939,
+      "tickers_with_bars": 505
+    }
+  ],
+  "bars_per_symbol": {
+    "min": 14,
+    "p10": 1261,
+    "p50": 1261,
+    "p90": 1261,
+    "max": 1261
+  },
+  "symbols_by_start_year": {
+    "2021": 494,
+    "2022": 3,
+    "2023": 2,
+    "2024": 2,
+    "2025": 3,
+    "2026": 1
+  },
+  "note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
+  "survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
+}
+```
+
+### Race guard
+
+```json
+{}
+```
+
+### Signal IC (full extended window)
+
+_Harness not run this pass._
+
+### Era split (diagnostic only)
+
+_No era split._
+
+## Verdict
+
+**COVERAGE_ONLY**
+
+Coverage probe only; run --phase harness after deep rebuild.
+
+## What a human must decide next
+
+- Compare sector residual vs market residual across eras.
+- If pre-2021 IC collapses, park Task 1 wire-in.
+- Do not retune production knobs on deep history levels.
+
+Artifacts: `reports/history-depth-20260719-093853.json`
+