dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests done

Browse Source
This commit is contained in:
Dennis Thiessen
2026-07-19 10:40:19 +02:00
parent 06cf054f60
commit f6e0ca734f
12 changed files with 1801 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/research/history-depth-extension.md
+119 -5
View File
@@ -80,15 +80,129 @@ Then commit `reports/` + updated research docs, or copy them back to Windows.
## Results ## Results
*(filled at run time)* Generated: `2026-07-19T10:33:15.322673`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{}
```
### Race guard
```json
{
"manifest": {
"schema_version": 1,
"snapshot": "research.sqlite",
"snapshot_resolved": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/research.sqlite",
"complete": true,
"finished_at": "2026-07-19T08:19:02.992206+00:00",
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149,
"sources": {
"nasdaq_all": "nasdaq_trader",
"sp500": "wikipedia_sp500"
},
"history_days": 5000,
"min_bars": 260,
"fetch_ok": 4152,
"fetch_fail": 0,
"limit": null,
"extra": {
"prod_symbols_at_start": 506,
"pool_size": 4648,
"to_fetch": 4152
},
"live_counts": {
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149
}
},
"ok": true
}
```
### Signal IC (full extended window)
| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable |
|---|---:|---:|---:|---:|---|
| high_52w | 0.1111 | 6.41 | 84 | 2280.3 | True |
| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 | True |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | True |
| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 | True |
| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 | True |
| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 | True |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 | True |
| fip_id | 0.0267 | 3.25 | 83 | 2277.5 | True |
| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 | True |
| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 | True |
| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 | True |
### Era split (diagnostic only)
#### full
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0267 | 3.25 | 83 | 2277.5 |
| high_52w | 0.1111 | 6.41 | 84 | 2280.3 |
| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 |
| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 |
| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 |
| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 |
| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 |
| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 |
#### pre_2021
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0116 | 1.35 | 36 | 1424.8 |
| high_52w | 0.0494 | 2.0 | 36 | 1422.2 |
| mom_12_1 | 0.0413 | 2.94 | 36 | 1424.8 |
| mom_12_1_resid | 0.0226 | 1.68 | 36 | 1424.8 |
| mom_3_1 | 0.0217 | 1.84 | 42 | 1480.0 |
| mom_6_1 | 0.0205 | 1.63 | 40 | 1461.2 |
| reversal_1m | 0.0019 | 0.15 | 44 | 1501.4 |
| trend_200 | 0.0322 | 2.23 | 38 | 1440.9 |
| vol_6m | -0.056 | -2.16 | 40 | 1461.2 |
#### post_2021
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0366 | 2.91 | 48 | 2913.0 |
| high_52w | 0.1375 | 4.34 | 48 | 2914.5 |
| mom_12_1 | 0.0791 | 3.64 | 48 | 2913.0 |
| mom_12_1_resid | 0.0265 | 1.37 | 48 | 2913.0 |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
| mom_3_1 | 0.0291 | 1.54 | 48 | 3234.2 |
| mom_6_1 | 0.0779 | 4.36 | 48 | 3122.6 |
| reversal_1m | -0.0126 | -0.72 | 48 | 3315.8 |
| trend_200 | 0.0585 | 2.68 | 48 | 3001.5 |
| vol_6m | -0.1623 | -4.94 | 48 | 3122.6 |
---
## Verdict ## Verdict
**Pending MacBook run.** **PENDING_HUMAN**
Harness complete — human interprets relative IC / era stability. No production retune from this artifact.
## What a human must decide next ## What a human must decide next
- Do not retune production from deep history without explicit review. - Compare sector residual vs market residual across eras.
- Use relative IC stability to accept/reject Task 1 sector residual wire-in. - If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-103315.json`
reports/history-depth-20260719-093853.json
+68
View File
@@ -0,0 +1,68 @@
{
"generated_at": "2026-07-19T09:38:53.936226",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"coverage": {
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
},
"race_guard": null,
"harness": null,
"verdict": "COVERAGE_ONLY",
"verdict_detail": "Coverage probe only; run --phase harness after deep rebuild.",
"human_next": "- Compare sector residual vs market residual across eras.\n- If pre-2021 IC collapses, park Task 1 wire-in.\n- Do not retune production knobs on deep history levels.",
"report_path": "reports/history-depth-20260719-093853.json"
}
reports/history-depth-20260719-093853.md
+177
View File
@@ -0,0 +1,177 @@
# History-depth extension (Tier-1 alpha research)
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
**Branch:** `research/history-depth-extension` (create from latest research stack).
**Production impact:** none. **Do not retune any production knob on deep history.**
---
## Pre-registration (locked before rebuild)
### Motivation
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
the 2018 vol shock and full 2020 crash (where the feed allows).
### Protocol
1. **Empirical coverage first** — bars per calendar year per symbol; document
where the feed thins out. Do **not** assume a uniform start date.
2. **Rebuild the research snapshot completely** from prod source + max history
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
`complete=true` and live counts match.
4. **Re-run full signal harness** (all existing signals incl. sector residual /
SUE if present) on the extended window.
5. **Report per signal:** mean IC, t, window count, and **era split**
(pre-/post-2021) — diagnostic only, **not a tuning input**.
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
**relative** signal comparisons and IC stability, not levels.
7. **Do not retune** production knobs. If a knob’s confirmation looks
overturned on deep history → report only; human decides.
### Success / interpretation (not promotion of a new signal)
| outcome | meaning |
|---|---|
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
| Any production knob looks worse deep | report; no auto-retune |
---
## MacBook runbook
Prefer the bundled script (one entry point):
```bash
git fetch origin && git checkout research/earnings-gap-and-sue
# .env: ALPACA_* required; FMP_* if resuming earnings
# copy backtest_snapshots/prod.sqlite if not already local
chmod +x scripts/run_tier1_macbook.sh
# Default: coverage → deep rebuild → harness (+ era split)
./scripts/run_tier1_macbook.sh
# Optional variants
./scripts/run_tier1_macbook.sh --all # + earnings resume first
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
./scripts/run_tier1_macbook.sh --coverage-only
# Tunables
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
```
Then commit `reports/` + updated research docs, or copy them back to Windows.
---
## Data provenance
*(filled at run time)*
---
## Results
Generated: `2026-07-19T09:38:53.936226`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
}
```
### Race guard
```json
{}
```
### Signal IC (full extended window)
_Harness not run this pass._
### Era split (diagnostic only)
_No era split._
## Verdict
**COVERAGE_ONLY**
Coverage probe only; run --phase harness after deep rebuild.
## What a human must decide next
- Compare sector residual vs market residual across eras.
- If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-093853.json`
reports/history-depth-20260719-094134.json
+68
View File
@@ -0,0 +1,68 @@
{
"generated_at": "2026-07-19T09:41:34.109378",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"coverage": {
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
},
"race_guard": null,
"harness": null,
"verdict": "COVERAGE_ONLY",
"verdict_detail": "Coverage probe only; run --phase harness after deep rebuild.",
"human_next": "- Compare sector residual vs market residual across eras.\n- If pre-2021 IC collapses, park Task 1 wire-in.\n- Do not retune production knobs on deep history levels.",
"report_path": "reports/history-depth-20260719-094134.json"
}
reports/history-depth-20260719-094134.md
+177
View File
@@ -0,0 +1,177 @@
# History-depth extension (Tier-1 alpha research)
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
**Branch:** `research/history-depth-extension` (create from latest research stack).
**Production impact:** none. **Do not retune any production knob on deep history.**
---
## Pre-registration (locked before rebuild)
### Motivation
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
the 2018 vol shock and full 2020 crash (where the feed allows).
### Protocol
1. **Empirical coverage first** — bars per calendar year per symbol; document
where the feed thins out. Do **not** assume a uniform start date.
2. **Rebuild the research snapshot completely** from prod source + max history
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
`complete=true` and live counts match.
4. **Re-run full signal harness** (all existing signals incl. sector residual /
SUE if present) on the extended window.
5. **Report per signal:** mean IC, t, window count, and **era split**
(pre-/post-2021) — diagnostic only, **not a tuning input**.
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
**relative** signal comparisons and IC stability, not levels.
7. **Do not retune** production knobs. If a knob’s confirmation looks
overturned on deep history → report only; human decides.
### Success / interpretation (not promotion of a new signal)
| outcome | meaning |
|---|---|
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
| Any production knob looks worse deep | report; no auto-retune |
---
## MacBook runbook
Prefer the bundled script (one entry point):
```bash
git fetch origin && git checkout research/earnings-gap-and-sue
# .env: ALPACA_* required; FMP_* if resuming earnings
# copy backtest_snapshots/prod.sqlite if not already local
chmod +x scripts/run_tier1_macbook.sh
# Default: coverage → deep rebuild → harness (+ era split)
./scripts/run_tier1_macbook.sh
# Optional variants
./scripts/run_tier1_macbook.sh --all # + earnings resume first
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
./scripts/run_tier1_macbook.sh --coverage-only
# Tunables
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
```
Then commit `reports/` + updated research docs, or copy them back to Windows.
---
## Data provenance
*(filled at run time)*
---
## Results
Generated: `2026-07-19T09:41:34.109378`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
}
```
### Race guard
```json
{}
```
### Signal IC (full extended window)
_Harness not run this pass._
### Era split (diagnostic only)
_No era split._
## Verdict
**COVERAGE_ONLY**
Coverage probe only; run --phase harness after deep rebuild.
## What a human must decide next
- Compare sector residual vs market residual across eras.
- If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-094134.json`
reports/history-depth-20260719-094344.json
+68
View File
@@ -0,0 +1,68 @@
{
"generated_at": "2026-07-19T09:43:44.520126",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"coverage": {
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
},
"race_guard": null,
"harness": null,
"verdict": "COVERAGE_ONLY",
"verdict_detail": "Coverage probe only; run --phase harness after deep rebuild.",
"human_next": "- Compare sector residual vs market residual across eras.\n- If pre-2021 IC collapses, park Task 1 wire-in.\n- Do not retune production knobs on deep history levels.",
"report_path": "reports/history-depth-20260719-094344.json"
}
reports/history-depth-20260719-094344.md
+177
View File
@@ -0,0 +1,177 @@
# History-depth extension (Tier-1 alpha research)
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
**Branch:** `research/history-depth-extension` (create from latest research stack).
**Production impact:** none. **Do not retune any production knob on deep history.**
---
## Pre-registration (locked before rebuild)
### Motivation
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
the 2018 vol shock and full 2020 crash (where the feed allows).
### Protocol
1. **Empirical coverage first** — bars per calendar year per symbol; document
where the feed thins out. Do **not** assume a uniform start date.
2. **Rebuild the research snapshot completely** from prod source + max history
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
`complete=true` and live counts match.
4. **Re-run full signal harness** (all existing signals incl. sector residual /
SUE if present) on the extended window.
5. **Report per signal:** mean IC, t, window count, and **era split**
(pre-/post-2021) — diagnostic only, **not a tuning input**.
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
**relative** signal comparisons and IC stability, not levels.
7. **Do not retune** production knobs. If a knob’s confirmation looks
overturned on deep history → report only; human decides.
### Success / interpretation (not promotion of a new signal)
| outcome | meaning |
|---|---|
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
| Any production knob looks worse deep | report; no auto-retune |
---
## MacBook runbook
Prefer the bundled script (one entry point):
```bash
git fetch origin && git checkout research/earnings-gap-and-sue
# .env: ALPACA_* required; FMP_* if resuming earnings
# copy backtest_snapshots/prod.sqlite if not already local
chmod +x scripts/run_tier1_macbook.sh
# Default: coverage → deep rebuild → harness (+ era split)
./scripts/run_tier1_macbook.sh
# Optional variants
./scripts/run_tier1_macbook.sh --all # + earnings resume first
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
./scripts/run_tier1_macbook.sh --coverage-only
# Tunables
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
```
Then commit `reports/` + updated research docs, or copy them back to Windows.
---
## Data provenance
*(filled at run time)*
---
## Results
Generated: `2026-07-19T09:43:44.520126`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
}
```
### Race guard
```json
{}
```
### Signal IC (full extended window)
_Harness not run this pass._
### Era split (diagnostic only)
_No era split._
## Verdict
**COVERAGE_ONLY**
Coverage probe only; run --phase harness after deep rebuild.
## What a human must decide next
- Compare sector residual vs market residual across eras.
- If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-094344.json`
reports/history-depth-20260719-095156.json
+68
View File
@@ -0,0 +1,68 @@
{
"generated_at": "2026-07-19T09:51:56.525634",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"coverage": {
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
},
"race_guard": null,
"harness": null,
"verdict": "COVERAGE_ONLY",
"verdict_detail": "Coverage probe only; run --phase harness after deep rebuild.",
"human_next": "- Compare sector residual vs market residual across eras.\n- If pre-2021 IC collapses, park Task 1 wire-in.\n- Do not retune production knobs on deep history levels.",
"report_path": "reports/history-depth-20260719-095156.json"
}
reports/history-depth-20260719-095156.md
+177
View File
@@ -0,0 +1,177 @@
# History-depth extension (Tier-1 alpha research)
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
**Branch:** `research/history-depth-extension` (create from latest research stack).
**Production impact:** none. **Do not retune any production knob on deep history.**
---
## Pre-registration (locked before rebuild)
### Motivation
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
the 2018 vol shock and full 2020 crash (where the feed allows).
### Protocol
1. **Empirical coverage first** — bars per calendar year per symbol; document
where the feed thins out. Do **not** assume a uniform start date.
2. **Rebuild the research snapshot completely** from prod source + max history
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
`complete=true` and live counts match.
4. **Re-run full signal harness** (all existing signals incl. sector residual /
SUE if present) on the extended window.
5. **Report per signal:** mean IC, t, window count, and **era split**
(pre-/post-2021) — diagnostic only, **not a tuning input**.
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
**relative** signal comparisons and IC stability, not levels.
7. **Do not retune** production knobs. If a knob’s confirmation looks
overturned on deep history → report only; human decides.
### Success / interpretation (not promotion of a new signal)
| outcome | meaning |
|---|---|
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
| Any production knob looks worse deep | report; no auto-retune |
---
## MacBook runbook
Prefer the bundled script (one entry point):
```bash
git fetch origin && git checkout research/earnings-gap-and-sue
# .env: ALPACA_* required; FMP_* if resuming earnings
# copy backtest_snapshots/prod.sqlite if not already local
chmod +x scripts/run_tier1_macbook.sh
# Default: coverage → deep rebuild → harness (+ era split)
./scripts/run_tier1_macbook.sh
# Optional variants
./scripts/run_tier1_macbook.sh --all # + earnings resume first
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
./scripts/run_tier1_macbook.sh --coverage-only
# Tunables
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
```
Then commit `reports/` + updated research docs, or copy them back to Windows.
---
## Data provenance
*(filled at run time)*
---
## Results
Generated: `2026-07-19T09:51:56.525634`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{
"snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
"ticker_count": 506,
"ohlcv_row_count": 629263,
"date_range": {
"min": "2021-06-24",
"max": "2026-07-02"
},
"bars_per_year": [
{
"year": "2021",
"bars": 65678,
"tickers_with_bars": 494
},
{
"year": "2022",
"bars": 124423,
"tickers_with_bars": 497
},
{
"year": "2023",
"bars": 124479,
"tickers_with_bars": 499
},
{
"year": "2024",
"bars": 126129,
"tickers_with_bars": 501
},
{
"year": "2025",
"bars": 125615,
"tickers_with_bars": 504
},
{
"year": "2026",
"bars": 62939,
"tickers_with_bars": 505
}
],
"bars_per_symbol": {
"min": 14,
"p10": 1261,
"p50": 1261,
"p90": 1261,
"max": 1261
},
"symbols_by_start_year": {
"2021": 494,
"2022": 3,
"2023": 2,
"2024": 2,
"2025": 3,
"2026": 1
},
"note": "Where ticker counts drop in early years, the feed (or listing history) thins \u2014 do not treat those years as a full 505-name cross-section.",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels."
}
```
### Race guard
```json
{}
```
### Signal IC (full extended window)
_Harness not run this pass._
### Era split (diagnostic only)
_No era split._
## Verdict
**COVERAGE_ONLY**
Coverage probe only; run --phase harness after deep rebuild.
## What a human must decide next
- Compare sector residual vs market residual across eras.
- If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-095156.json`
reports/history-depth-20260719-103315.json
+494
View File
@@ -0,0 +1,494 @@
{
"generated_at": "2026-07-19T10:33:15.322673",
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"coverage": null,
"race_guard": {
"manifest": {
"schema_version": 1,
"snapshot": "research.sqlite",
"snapshot_resolved": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/research.sqlite",
"complete": true,
"finished_at": "2026-07-19T08:19:02.992206+00:00",
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149,
"sources": {
"nasdaq_all": "nasdaq_trader",
"sp500": "wikipedia_sp500"
},
"history_days": 5000,
"min_bars": 260,
"fetch_ok": 4152,
"fetch_fail": 0,
"limit": null,
"extra": {
"prod_symbols_at_start": 506,
"pool_size": 4648,
"to_fetch": 4152
},
"live_counts": {
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149
}
},
"ok": true
},
"harness": {
"survivorship_banner": "SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only \u2014 not levels.",
"signal_eval": [
{
"signal": "high_52w",
"weeks": 84,
"avg_cross_section": 2280.3,
"mean_ic": 0.1111,
"ic_t_stat": 6.41,
"ic_positive_pct": 76.2,
"mean_quintile_spread": -13.5104,
"reliable": true
},
{
"signal": "mom_12_1",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0663,
"ic_t_stat": 5.11,
"ic_positive_pct": 74.7,
"mean_quintile_spread": -10.1818,
"reliable": true
},
{
"signal": "mom_12_1_sector_resid",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": 0.0578,
"ic_t_stat": 2.34,
"ic_positive_pct": 65.7,
"mean_quintile_spread": 0.0245,
"reliable": true
},
{
"signal": "trend_200",
"weeks": 85,
"avg_cross_section": 2308.1,
"mean_ic": 0.0546,
"ic_t_stat": 4.3,
"ic_positive_pct": 70.6,
"mean_quintile_spread": -12.4094,
"reliable": true
},
{
"signal": "mom_6_1",
"weeks": 88,
"avg_cross_section": 2375.7,
"mean_ic": 0.0493,
"ic_t_stat": 4.91,
"ic_positive_pct": 70.5,
"mean_quintile_spread": 3.4224,
"reliable": true
},
{
"signal": "mom_3_1",
"weeks": 90,
"avg_cross_section": 2425.6,
"mean_ic": 0.0363,
"ic_t_stat": 3.58,
"ic_positive_pct": 72.2,
"mean_quintile_spread": 0.9289,
"reliable": true
},
{
"signal": "mom_12_1_sector_demeaned",
"weeks": 35,
"avg_cross_section": 496.7,
"mean_ic": 0.034,
"ic_t_stat": 1.32,
"ic_positive_pct": 62.9,
"mean_quintile_spread": 0.0154,
"reliable": true
},
{
"signal": "fip_id",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0267,
"ic_t_stat": 3.25,
"ic_positive_pct": 67.5,
"mean_quintile_spread": -0.0017,
"reliable": true
},
{
"signal": "mom_12_1_resid",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0256,
"ic_t_stat": 2.21,
"ic_positive_pct": 65.1,
"mean_quintile_spread": 10.1373,
"reliable": true
},
{
"signal": "reversal_1m",
"weeks": 91,
"avg_cross_section": 2443.2,
"mean_ic": 0.003,
"ic_t_stat": 0.31,
"ic_positive_pct": 48.4,
"mean_quintile_spread": -7.0153,
"reliable": true
},
{
"signal": "vol_6m",
"weeks": 88,
"avg_cross_section": 2375.7,
"mean_ic": -0.1226,
"ic_t_stat": -6.34,
"ic_positive_pct": 21.6,
"mean_quintile_spread": 1.4646,
"reliable": true
}
],
"era_split": {
"era_split_date": "2021-01-01",
"note": "Diagnostic only \u2014 not a tuning input. Nested lookbacks are not OOS.",
"full": {
"high_52w": {
"signal": "high_52w",
"weeks": 84,
"avg_cross_section": 2280.3,
"mean_ic": 0.1111,
"ic_t_stat": 6.41,
"ic_positive_pct": 76.2,
"mean_quintile_spread": -13.5104,
"reliable": true
},
"mom_12_1": {
"signal": "mom_12_1",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0663,
"ic_t_stat": 5.11,
"ic_positive_pct": 74.7,
"mean_quintile_spread": -10.1818,
"reliable": true
},
"mom_12_1_sector_resid": {
"signal": "mom_12_1_sector_resid",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": 0.0578,
"ic_t_stat": 2.34,
"ic_positive_pct": 65.7,
"mean_quintile_spread": 0.0245,
"reliable": true
},
"trend_200": {
"signal": "trend_200",
"weeks": 85,
"avg_cross_section": 2308.1,
"mean_ic": 0.0546,
"ic_t_stat": 4.3,
"ic_positive_pct": 70.6,
"mean_quintile_spread": -12.4094,
"reliable": true
},
"mom_6_1": {
"signal": "mom_6_1",
"weeks": 88,
"avg_cross_section": 2375.7,
"mean_ic": 0.0493,
"ic_t_stat": 4.91,
"ic_positive_pct": 70.5,
"mean_quintile_spread": 3.4224,
"reliable": true
},
"mom_3_1": {
"signal": "mom_3_1",
"weeks": 90,
"avg_cross_section": 2425.6,
"mean_ic": 0.0363,
"ic_t_stat": 3.58,
"ic_positive_pct": 72.2,
"mean_quintile_spread": 0.9289,
"reliable": true
},
"mom_12_1_sector_demeaned": {
"signal": "mom_12_1_sector_demeaned",
"weeks": 35,
"avg_cross_section": 496.7,
"mean_ic": 0.034,
"ic_t_stat": 1.32,
"ic_positive_pct": 62.9,
"mean_quintile_spread": 0.0154,
"reliable": true
},
"fip_id": {
"signal": "fip_id",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0267,
"ic_t_stat": 3.25,
"ic_positive_pct": 67.5,
"mean_quintile_spread": -0.0017,
"reliable": true
},
"mom_12_1_resid": {
"signal": "mom_12_1_resid",
"weeks": 83,
"avg_cross_section": 2277.5,
"mean_ic": 0.0256,
"ic_t_stat": 2.21,
"ic_positive_pct": 65.1,
"mean_quintile_spread": 10.1373,
"reliable": true
},
"reversal_1m": {
"signal": "reversal_1m",
"weeks": 91,
"avg_cross_section": 2443.2,
"mean_ic": 0.003,
"ic_t_stat": 0.31,
"ic_positive_pct": 48.4,
"mean_quintile_spread": -7.0153,
"reliable": true
},
"vol_6m": {
"signal": "vol_6m",
"weeks": 88,
"avg_cross_section": 2375.7,
"mean_ic": -0.1226,
"ic_t_stat": -6.34,
"ic_positive_pct": 21.6,
"mean_quintile_spread": 1.4646,
"reliable": true
}
},
"pre_2021": {
"high_52w": {
"signal": "high_52w",
"weeks": 36,
"avg_cross_section": 1422.2,
"mean_ic": 0.0494,
"ic_t_stat": 2.0,
"ic_positive_pct": 63.9,
"mean_quintile_spread": -31.5146,
"reliable": true
},
"mom_12_1": {
"signal": "mom_12_1",
"weeks": 36,
"avg_cross_section": 1424.8,
"mean_ic": 0.0413,
"ic_t_stat": 2.94,
"ic_positive_pct": 66.7,
"mean_quintile_spread": -23.3935,
"reliable": true
},
"trend_200": {
"signal": "trend_200",
"weeks": 38,
"avg_cross_section": 1440.9,
"mean_ic": 0.0322,
"ic_t_stat": 2.23,
"ic_positive_pct": 68.4,
"mean_quintile_spread": -27.5982,
"reliable": true
},
"mom_12_1_resid": {
"signal": "mom_12_1_resid",
"weeks": 36,
"avg_cross_section": 1424.8,
"mean_ic": 0.0226,
"ic_t_stat": 1.68,
"ic_positive_pct": 69.4,
"mean_quintile_spread": 23.4097,
"reliable": true
},
"mom_3_1": {
"signal": "mom_3_1",
"weeks": 42,
"avg_cross_section": 1480.0,
"mean_ic": 0.0217,
"ic_t_stat": 1.84,
"ic_positive_pct": 73.8,
"mean_quintile_spread": 2.0077,
"reliable": true
},
"mom_6_1": {
"signal": "mom_6_1",
"weeks": 40,
"avg_cross_section": 1461.2,
"mean_ic": 0.0205,
"ic_t_stat": 1.63,
"ic_positive_pct": 65.0,
"mean_quintile_spread": 7.4512,
"reliable": true
},
"fip_id": {
"signal": "fip_id",
"weeks": 36,
"avg_cross_section": 1424.8,
"mean_ic": 0.0116,
"ic_t_stat": 1.35,
"ic_positive_pct": 58.3,
"mean_quintile_spread": -0.0118,
"reliable": true
},
"reversal_1m": {
"signal": "reversal_1m",
"weeks": 44,
"avg_cross_section": 1501.4,
"mean_ic": 0.0019,
"ic_t_stat": 0.15,
"ic_positive_pct": 50.0,
"mean_quintile_spread": -14.4299,
"reliable": true
},
"vol_6m": {
"signal": "vol_6m",
"weeks": 40,
"avg_cross_section": 1461.2,
"mean_ic": -0.056,
"ic_t_stat": -2.16,
"ic_positive_pct": 35.0,
"mean_quintile_spread": 3.1335,
"reliable": true
}
},
"post_2021": {
"high_52w": {
"signal": "high_52w",
"weeks": 48,
"avg_cross_section": 2914.5,
"mean_ic": 0.1375,
"ic_t_stat": 4.34,
"ic_positive_pct": 79.2,
"mean_quintile_spread": -0.0856,
"reliable": true
},
"mom_12_1": {
"signal": "mom_12_1",
"weeks": 48,
"avg_cross_section": 2913.0,
"mean_ic": 0.0791,
"ic_t_stat": 3.64,
"ic_positive_pct": 77.1,
"mean_quintile_spread": -0.0858,
"reliable": true
},
"mom_6_1": {
"signal": "mom_6_1",
"weeks": 48,
"avg_cross_section": 3122.6,
"mean_ic": 0.0779,
"ic_t_stat": 4.36,
"ic_positive_pct": 75.0,
"mean_quintile_spread": -0.0781,
"reliable": true
},
"trend_200": {
"signal": "trend_200",
"weeks": 48,
"avg_cross_section": 3001.5,
"mean_ic": 0.0585,
"ic_t_stat": 2.68,
"ic_positive_pct": 72.9,
"mean_quintile_spread": -0.1129,
"reliable": true
},
"mom_12_1_sector_resid": {
"signal": "mom_12_1_sector_resid",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": 0.0578,
"ic_t_stat": 2.34,
"ic_positive_pct": 65.7,
"mean_quintile_spread": 0.0245,
"reliable": true
},
"fip_id": {
"signal": "fip_id",
"weeks": 48,
"avg_cross_section": 2913.0,
"mean_ic": 0.0366,
"ic_t_stat": 2.91,
"ic_positive_pct": 70.8,
"mean_quintile_spread": -0.0151,
"reliable": true
},
"mom_12_1_sector_demeaned": {
"signal": "mom_12_1_sector_demeaned",
"weeks": 35,
"avg_cross_section": 496.7,
"mean_ic": 0.034,
"ic_t_stat": 1.32,
"ic_positive_pct": 62.9,
"mean_quintile_spread": 0.0154,
"reliable": true
},
"mom_3_1": {
"signal": "mom_3_1",
"weeks": 48,
"avg_cross_section": 3234.2,
"mean_ic": 0.0291,
"ic_t_stat": 1.54,
"ic_positive_pct": 66.7,
"mean_quintile_spread": -0.0356,
"reliable": true
},
"mom_12_1_resid": {
"signal": "mom_12_1_resid",
"weeks": 48,
"avg_cross_section": 2913.0,
"mean_ic": 0.0265,
"ic_t_stat": 1.37,
"ic_positive_pct": 66.7,
"mean_quintile_spread": -0.0405,
"reliable": true
},
"reversal_1m": {
"signal": "reversal_1m",
"weeks": 48,
"avg_cross_section": 3315.8,
"mean_ic": -0.0126,
"ic_t_stat": -0.72,
"ic_positive_pct": 45.8,
"mean_quintile_spread": -0.0387,
"reliable": true
},
"vol_6m": {
"signal": "vol_6m",
"weeks": 48,
"avg_cross_section": 3122.6,
"mean_ic": -0.1623,
"ic_t_stat": -4.94,
"ic_positive_pct": 22.9,
"mean_quintile_spread": 0.0059,
"reliable": true
}
}
},
"params": {
"step_days": 5,
"step_sessions": 5,
"entry_cadence": "weekly",
"signal_eval_cadence": "weekly",
"horizon_days": 30,
"min_lookback": 60,
"cost_per_side_pct": 0.1,
"target_model": "production_gtl",
"target_model_label": "Live GTL (production)",
"is_production_target_model": true,
"production_reentry_policy": "gate_reset",
"liquid_breadth_top_n": null,
"liquid_min_price": null,
"signal_eval_only": true
},
"tickers": 4655,
"generated_at_run": "2026-07-19T08:27:27.206022+00:00"
},
"verdict": "PENDING_HUMAN",
"verdict_detail": "Harness complete \u2014 human interprets relative IC / era stability. No production retune from this artifact.",
"human_next": "- Compare sector residual vs market residual across eras.\n- If pre-2021 IC collapses, park Task 1 wire-in.\n- Do not retune production knobs on deep history levels.",
"report_path": "reports/history-depth-20260719-103315.json"
}
reports/history-depth-20260719-103315.md
+208
View File
@@ -0,0 +1,208 @@
# History-depth extension (Tier-1 alpha research)
**Status:** PRE-REGISTERED — run on MacBook (heavy I/O + full harness).
**Branch:** `research/history-depth-extension` (create from latest research stack).
**Production impact:** none. **Do not retune any production knob on deep history.**
---
## Pre-registration (locked before rebuild)
### Motivation
All current conclusions rest on ~35 non-overlapping weekly windows in essentially
one post-2021 regime. Extending history toward max Alpaca daily-bar depth adds
the 2018 vol shock and full 2020 crash (where the feed allows).
### Protocol
1. **Empirical coverage first** — bars per calendar year per symbol; document
where the feed thins out. Do **not** assume a uniform start date.
2. **Rebuild the research snapshot completely** from prod source + max history
per symbol (`Adjustment.SPLIT`, ~200 req/min pacing via existing extender).
3. **Race guard (rule 6)** — refuse analysis until completion manifest is
`complete=true` and live counts match.
4. **Re-run full signal harness** (all existing signals incl. sector residual /
SUE if present) on the extended window.
5. **Report per signal:** mean IC, t, window count, and **era split**
(pre-/post-2021) — diagnostic only, **not a tuning input**.
6. **Log prominently:** survivorship bias grows with depth (today’s constituents
backfilled). Absolute Sharpe/CAGR on deep history is optimistic; payload is
**relative** signal comparisons and IC stability, not levels.
7. **Do not retune** production knobs. If a knob’s confirmation looks
overturned on deep history → report only; human decides.
### Success / interpretation (not promotion of a new signal)
| outcome | meaning |
|---|---|
| Sector residual still ≥ market residual on deep IC + stable sign | strengthens Task 1 PROMOTE case |
| Sector residual collapses pre-2021 | **PARK** Task 1 wire-in |
| SUE remains weak after full earnings + depth | **DEAD** SUE for this stack |
| Any production knob looks worse deep | report; no auto-retune |
---
## MacBook runbook
Prefer the bundled script (one entry point):
```bash
git fetch origin && git checkout research/earnings-gap-and-sue
# .env: ALPACA_* required; FMP_* if resuming earnings
# copy backtest_snapshots/prod.sqlite if not already local
chmod +x scripts/run_tier1_macbook.sh
# Default: coverage → deep rebuild → harness (+ era split)
./scripts/run_tier1_macbook.sh
# Optional variants
./scripts/run_tier1_macbook.sh --all # + earnings resume first
./scripts/run_tier1_macbook.sh --earnings-only # multi-day FMP + 2a/2b only
./scripts/run_tier1_macbook.sh --harness-only # skip rebuild
./scripts/run_tier1_macbook.sh --coverage-only
# Tunables
WORKERS=12 HISTORY_DAYS=5000 ./scripts/run_tier1_macbook.sh
./scripts/run_tier1_macbook.sh --workers 12 --fmp-limit 250
```
Then commit `reports/` + updated research docs, or copy them back to Windows.
---
## Data provenance
*(filled at run time)*
---
## Results
Generated: `2026-07-19T10:33:15.322673`
> **SURVIVORSHIP BIAS: today's constituents backfilled historically. Absolute Sharpe/CAGR levels on deep history are optimistic. Use RELATIVE signal IC comparisons and era stability only — not levels.**
### Coverage
```json
{}
```
### Race guard
```json
{
"manifest": {
"schema_version": 1,
"snapshot": "research.sqlite",
"snapshot_resolved": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/research.sqlite",
"complete": true,
"finished_at": "2026-07-19T08:19:02.992206+00:00",
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149,
"sources": {
"nasdaq_all": "nasdaq_trader",
"sp500": "wikipedia_sp500"
},
"history_days": 5000,
"min_bars": 260,
"fetch_ok": 4152,
"fetch_fail": 0,
"limit": null,
"extra": {
"prod_symbols_at_start": 506,
"pool_size": 4648,
"to_fetch": 4152
},
"live_counts": {
"ticker_count": 4655,
"ohlcv_row_count": 6609926,
"rank_only_count": 4149
}
},
"ok": true
}
```
### Signal IC (full extended window)
| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable |
|---|---:|---:|---:|---:|---|
| high_52w | 0.1111 | 6.41 | 84 | 2280.3 | True |
| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 | True |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | True |
| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 | True |
| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 | True |
| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 | True |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 | True |
| fip_id | 0.0267 | 3.25 | 83 | 2277.5 | True |
| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 | True |
| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 | True |
| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 | True |
### Era split (diagnostic only)
#### full
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0267 | 3.25 | 83 | 2277.5 |
| high_52w | 0.1111 | 6.41 | 84 | 2280.3 |
| mom_12_1 | 0.0663 | 5.11 | 83 | 2277.5 |
| mom_12_1_resid | 0.0256 | 2.21 | 83 | 2277.5 |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
| mom_3_1 | 0.0363 | 3.58 | 90 | 2425.6 |
| mom_6_1 | 0.0493 | 4.91 | 88 | 2375.7 |
| reversal_1m | 0.003 | 0.31 | 91 | 2443.2 |
| trend_200 | 0.0546 | 4.3 | 85 | 2308.1 |
| vol_6m | -0.1226 | -6.34 | 88 | 2375.7 |
#### pre_2021
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0116 | 1.35 | 36 | 1424.8 |
| high_52w | 0.0494 | 2.0 | 36 | 1422.2 |
| mom_12_1 | 0.0413 | 2.94 | 36 | 1424.8 |
| mom_12_1_resid | 0.0226 | 1.68 | 36 | 1424.8 |
| mom_3_1 | 0.0217 | 1.84 | 42 | 1480.0 |
| mom_6_1 | 0.0205 | 1.63 | 40 | 1461.2 |
| reversal_1m | 0.0019 | 0.15 | 44 | 1501.4 |
| trend_200 | 0.0322 | 2.23 | 38 | 1440.9 |
| vol_6m | -0.056 | -2.16 | 40 | 1461.2 |
#### post_2021
| signal | mean_ic | t | weeks | N |
|---|---:|---:|---:|---:|
| fip_id | 0.0366 | 2.91 | 48 | 2913.0 |
| high_52w | 0.1375 | 4.34 | 48 | 2914.5 |
| mom_12_1 | 0.0791 | 3.64 | 48 | 2913.0 |
| mom_12_1_resid | 0.0265 | 1.37 | 48 | 2913.0 |
| mom_12_1_sector_demeaned | 0.034 | 1.32 | 35 | 496.7 |
| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 |
| mom_3_1 | 0.0291 | 1.54 | 48 | 3234.2 |
| mom_6_1 | 0.0779 | 4.36 | 48 | 3122.6 |
| reversal_1m | -0.0126 | -0.72 | 48 | 3315.8 |
| trend_200 | 0.0585 | 2.68 | 48 | 3001.5 |
| vol_6m | -0.1623 | -4.94 | 48 | 3122.6 |
## Verdict
**PENDING_HUMAN**
Harness complete — human interprets relative IC / era stability. No production retune from this artifact.
## What a human must decide next
- Compare sector residual vs market residual across eras.
- If pre-2021 IC collapses, park Task 1 wire-in.
- Do not retune production knobs on deep history levels.
Artifacts: `reports/history-depth-20260719-103315.json`
scripts/run_tier1_macbook.sh
Regular → Executable
View File