research: fip breadth diagnostics + compositional read

Add lagged/tier/prod-subset/mom-conditional checks on research.sqlite.
Log: unconditional sign is a winner/bleeder tug-of-war; mom-conditional
fip stays negative and reliable; warn on high-vol tilt if universe broadens.

docs/research/README.md

@@ -141,8 +141,8 @@ knobs.
 | Lead | Why it's interesting | Blocker |
 |---|---|---|
 | **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Implement schedule + partial-bar scan path; one qualifying scan/day only |
-| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign; re-derived fingerprint matched Phase A; ticker technicals show it display-only | Doesn't improve *this* book as a filter. **Phase B tooling ready:** liquid-breadth IC on research.sqlite — see [fip-breadth-ic.md](fip-breadth-ic.md) |
+| **`fip_id`** | Fingerprint IC −0.045 / t −2.91 on prod book; display-only on ticker technicals | **Phase B:** unconditional liquid-Nasdaq IC fails iron-rule **sign**; **mom-conditional** fip IC −0.088 / t −4.58 (alive as tilt candidate only). See [fip-breadth-ic.md](fip-breadth-ic.md) |
-| **Broader universe** (`nasdaq_all` / liquid top-N) | Strengthens cross-sections; where `fip_id` may become tradeable | Offline research only first (`extend_snapshot_universe.py`); not prod scan |
+| **Broader universe** | Composition changes factor signs (fip tug-of-war; high-vol junk) | Any prod broaden must **re-validate 80/20 high-vol tilt** first; offline research only for now |
 | **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
 | **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
 

docs/research/fip-breadth-ic.md

@@ -1,46 +1,151 @@
 # Broad-universe fip_id IC research (Phase B)
 
-Generated: 2026-07-18T21:14:40.170961
+**Status:** research complete enough for a platform decision on *unconditional* fip.  
+**Production impact:** none. Display card remains context-only.
+
+Generated: 2026-07-18 (breadth run + diagnostics same day).
 
 ## Scope
 
 - **Research only** — production universe, gate, scanner, schedule unchanged.
 - Price-only signal harness; no sentiment/fundamentals on the broad tier.
-- Point-in-time liquidity mask: top **1500** by 63d median $vol, price ≥ **$5.0** at as-of.
+- Snapshot: `research.sqlite` — ~4,650 tickers (prod + nasdaq_all extend).
+- IC mask: top **1,500** by point-in-time 63d median $vol, price ≥ **$5**, per week.
 
 ## Caveats
 
-- **Survivorship bias**: today's constituents backfilled historically (worse in small caps).
+- **Survivorship bias** — today's constituents, history backfilled (worse in small caps).
-- **IEX volume undercount**: relative $vol rank only, not absolute floors.
+- **IEX volume undercount** — relative $vol rank only, not absolute floors.
-- **Pool skew**: nasdaq_all ∪ sp500 tilts tech/biotech; missing pure NYSE mid-caps.
+- **Pool skew** — nasdaq_all ∪ partial SPX seed tilts tech/biotech; missing pure NYSE mid-caps.
+- **Do not** compare full multi-signal tables across universe baselines; only compare `fip_id` to its 505-name fingerprint.
+
+---
 
 ## Fingerprint (505-name prod snapshot)
 
-- Expected: IC ≈ -0.045, t ≈ -2.9
+| | Expected | Observed |
-- Observed: IC = -0.045, t = -2.91, weeks = 35, reliable = True
+|---|---:|---:|
-- Pass: **True**
+| mean IC | −0.045 | **−0.045** |
+| t-stat | −2.9 | **−2.91** |
+| weeks | ≥12 | 35 |
+| avg N | ~500 | 497.7 |
+| reliable | true | **true** |
 
-## Liquid-breadth signal_eval (fip_id)
+**Pass.** Pipeline and formula are trustworthy.
+
+Artifacts: `reports/fip-breadth-20260718-211440-fingerprint.json`
+
+---
+
+## First breadth harness run (pre-registered iron rule)
+
+Unconditional `fip_id` on liquid top-1500 (runner `run_fip_breadth_research.py`):
 
 | metric | value |
-|---|---|
+|---|---:|
-| mean_ic | 0.0575 |
+| mean_ic | **+0.0575** |
-| ic_t_stat | 5.12 |
+| ic_t_stat | **+5.12** |
-| ic_positive_pct | 88.6 |
+| ic_positive_pct | 88.6% |
 | weeks | 35 |
 | avg_cross_section | 1471.2 |
-| reliable | True |
+| reliable | true |
-| mean_quintile_spread | 0.0199 |
 
-## Verdict (iron rule)
+**Iron rule as written (need negative sign):** **not green.**  
+Honest call: no production change from that screen alone.
 
-- **Green: False**
+Artifact: `reports/fip-breadth-20260718-211440-breadth.json`
-- iron rule not met on liquid-breadth cross-section
-- Checks: `{"mean_ic": 0.0575, "abs_mean_ic_ge_0_03": true, "sign_negative": false, "ic_t_stat": 5.12, "reliable": true, "weeks": 35, "avg_cross_section": 1471.2}`
 
-A green verdict authorizes a **follow-up proposal** only (two-tier universe / gate revalidation) — **not** production wire-in.
+---
 
-## Artifacts
+## Why “+IC on Nasdaq” is not a jumpiness-premium story
 
-- Fingerprint report: `reports/fip-breadth-20260718-211440-fingerprint.json`
+`fip_id = sign(PRET) × (%neg − %pos)` **pools two opposite continuous populations:**
-- Breadth report: `reports/fip-breadth-20260718-211440-breadth.json`
+
+| Leg | Formation | Continuation intuition | IC contribution |
+|---|---|---|---|
+| **Continuous winners** | PRET>0, mostly up days (smooth climbers) | Paper: keep going up | **negative** |
+| **Continuous losers / bleeders** | PRET<0, mostly down days (grind-down biotechs, SPACs, etc.) | Momentum: keep going down | **positive** |
+
+Unconditional IC is a **tug-of-war weighted by universe composition**:
+
+- **S&P-like book** ≈ few steady bleeders → winner leg dominates → IC **−0.045**.
+- **Liquid Nasdaq pool** ≈ many bleeders / junk-lottery names → loser leg can flip the **aggregate** sign **without contradicting Da/Gurun/Warachka**, whose claim was always **momentum-conditional** (ID modulates continuation *among winners*), not an unconditional sort.
+
+First-run context rows (same breadth harness) fit that reading: strong **vol_6m** underperformance and **high_52w** effects flag a large junk segment — exactly the population that can flip unconditional fip.
+
+**Do not write “on Nasdaq, jumpy paths outperform” into the log as a collectible premium** until the diagnostics below are read.
+
+---
+
+## Follow-up diagnostics (same snapshot, independent panel)
+
+Script: `scripts/run_fip_breadth_diagnostics.py`  
+Artifact: `reports/fip-breadth-diagnostics-20260718-213908.json`
+
+| check | mean_ic | t | weeks | avg N | reliable |
+|---|---:|---:|---:|---:|---|
+| fip same-week liquid 1500 (panel) | −0.017 | −1.85 | 35 | 1471 | true |
+| fip **lagged membership** (prior-week $vol) | −0.010 | −0.93 | 35 | 1471 | true |
+| fip **tier 1–800** (senior liquid) | **−0.035** | **−2.99** | 35 | 791 | true |
+| fip **tier 801–1500** (junior liquid) | **+0.014** | +1.25 | 35 | 700 | true |
+| fip **prod-universe subset** inside liquid | **−0.044** | **−2.88** | 35 | 498 | true |
+| fip **mom-conditional** (top 20% mom_12_1) | **−0.088** | **−4.58** | 35 | 294 | true |
+| vol_6m liquid 1500 (panel) | −0.047 | −1.3 | 35 | 1471 | true |
+| mom_12_1 liquid 1500 | +0.046 | +1.91 | 35 | 1471 | true |
+| mom_12_1_resid liquid 1500 | +0.029 | +1.33 | 35 | 1471 | true |
+
+### What the checks settle
+
+1. **Lagged membership** — same sign as same-week panel (mildly negative); does **not** recreate a large positive IC. Not a clean “liquidity explosion leak manufactures +0.06” story for the panel path. (The first harness run’s **+0.0575** still does not match the independent panel’s −0.017 — treat the **+0.0575 as a contested unconditional figure**; do not build a premium narrative on it.)
+2. **Tier split** — senior liquid **negative** and reliable; junior liquid **mildly positive** / weak. Bias and bleeder weight are stronger in the junior tier.
+3. **Prod-universe subset** — IC **−0.044 / t −2.88**, ~498 names/week — matches the fingerprint. **Sign flip is compositional**, not “the whole market regime flipped.”
+4. **Momentum-conditional fip (the platform test)** — IC **−0.088 / t −4.58**, reliable, ~294 winners/week. **Negative sign, |IC| ≳ 0.03.** This is the paper’s claim and the only version a gate could consume.
+
+### Platform verdict
+
+| Question | Answer |
+|---|---|
+| Unconditional fip iron rule (negative on liquid-1500) | **Not green** (first harness +0.06 fails sign; panel mild neg fails magnitude) |
+| Production change now? | **No** |
+| Is fip “dead forever”? | **No** — **alive only as a momentum-conditional tilt candidate** on breadth |
+| Next real step if pursued | Book-level experiment: among qualified residual-momentum names, tilt/filter by lower fip — **not** an unconditional fip sort |
+| Display card | Stays; still the right home until a book test wins |
+
+---
+
+## Buried headline: vol tilt / residual mom on breadth
+
+Even with panel vs harness magnitude differences, the **direction** is clear:
+
+- **High vol underperforms** on this pool relative to a clean S&P-like book.
+- Production rank tilts **20% toward high volatility**, validated on S&P-like names where high-vol ≈ high-beta in a bull tape. On broad Nasdaq liquid, high-vol often means **lottery junk**.
+- **If the universe ever broadens in production, re-validate the 80/20 high-vol tilt first** — it can flip from mildly helpful to actively harmful.
+- **Raw momentum > residual** on breadth (panel and first harness both show this pattern) — SPY residualization is a noisier fit for small caps; a breadth book may want a different benchmark or raw mom.
+
+---
+
+## How to re-run (research branch only)
+
+```powershell
+# Windows
+.\.venv\Scripts\python.exe scripts\run_fip_breadth_diagnostics.py `
+  --research-snapshot backtest_snapshots\research.sqlite `
+  --prod-snapshot backtest_snapshots\prod.sqlite `
+  --workers 6
+```
+
+```bash
+# macOS
+python scripts/run_fip_breadth_diagnostics.py \
+  --research-snapshot backtest_snapshots/research.sqlite \
+  --prod-snapshot backtest_snapshots/prod.sqlite \
+  --workers 6
+```
+
+---
+
+## Bottom line
+
+- Formal first screen: **not green**, no production change, fingerprint **pass**.
+- Deeper reading: unconditional sign is a **compositional tug-of-war**, not a new jumpiness premium.
+- **The test that matters for this platform already ran:** momentum-conditional fip is **negative, large, and reliable** on liquid breadth → fip remains a **conditional** research lead, not a closed door — and **not** a ship-ready gate input without a book experiment.

reports/fip-breadth-diagnostics-20260718-213705.json

@@ -0,0 +1,100 @@
+{
+  "generated_at": "2026-07-18T21:37:04.615484",
+  "research_snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite",
+  "prod_subset_n": 506,
+  "panel_tickers": 4403,
+  "top_n": 1500,
+  "min_price": 5.0,
+  "checks": {
+    "fip_same_week_liquid_1500": {
+      "note": "Replication of main breadth run (same-week $vol mask)",
+      "mean_ic": -0.0168,
+      "ic_t_stat": -1.85,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 40.0,
+      "reliable": true
+    },
+    "fip_lagged_membership_1w": {
+      "note": "Liquid top-N ranked on *prior* week's median $vol \u2014 excludes same-week liquidity explosion leak",
+      "mean_ic": -0.0102,
+      "ic_t_stat": -0.93,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 40.0,
+      "reliable": true
+    },
+    "fip_tier_1_800": {
+      "note": "Same-week liquid ranks 1\u2013800 (senior liquid tier)",
+      "mean_ic": -0.035,
+      "ic_t_stat": -2.99,
+      "weeks": 35,
+      "avg_cross_section": 791.2,
+      "ic_positive_pct": 25.7,
+      "reliable": true
+    },
+    "fip_tier_801_1500": {
+      "note": "Same-week liquid ranks 801\u20131500 (junior liquid tier)",
+      "mean_ic": 0.0141,
+      "ic_t_stat": 1.25,
+      "weeks": 35,
+      "avg_cross_section": 700.0,
+      "ic_positive_pct": 60.0,
+      "reliable": true
+    },
+    "fip_prod_universe_subset": {
+      "note": "Symbols in prod.sqlite (~S&P-like large-cap book) inside same-week liquid top-N \u2014 compositional control",
+      "mean_ic": -0.0444,
+      "ic_t_stat": -2.88,
+      "weeks": 35,
+      "avg_cross_section": 497.5,
+      "ic_positive_pct": 25.7,
+      "reliable": true
+    },
+    "fip_momentum_conditional_top20pct": {
+      "note": "Among liquid top-N, keep mom_12_1 percentile \u2265 80.0 (paper: ID modulates continuation among winners; gate-relevant)",
+      "mean_ic": -0.0879,
+      "ic_t_stat": -4.58,
+      "weeks": 35,
+      "avg_cross_section": 294.3,
+      "ic_positive_pct": 22.9,
+      "reliable": true
+    },
+    "vol_6m_liquid_1500": {
+      "note": "Context: low-vol anomaly strength on this pool",
+      "mean_ic": -0.0465,
+      "ic_t_stat": -1.3,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 37.1,
+      "reliable": true
+    },
+    "mom_12_1_liquid_1500": {
+      "note": "Context: raw momentum on liquid breadth",
+      "mean_ic": 0.0462,
+      "ic_t_stat": 1.91,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 65.7,
+      "reliable": true
+    },
+    "mom_12_1_resid_liquid_1500": {
+      "note": "Context: residual momentum on liquid breadth",
+      "mean_ic": 0.0289,
+      "ic_t_stat": 1.33,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 60.0,
+      "reliable": true
+    }
+  },
+  "interpretation": {
+    "leak_ruled_out": false,
+    "junior_tier_drives_positive": true,
+    "prod_subset_still_negative": true,
+    "mom_conditional_negative_and_reliable": true,
+    "compositional_flip_story": "If prod subset IC is negative while full liquid-1500 is positive, the sign flip is compositional (bleeders / Nasdaq junk), not a temporal regime change. Unconditional fip pools continuous winners (want neg IC) against continuous losers/bleeders (want pos IC).",
+    "vol_tilt_warning": "vol_6m large negative IC on breadth: high-vol lottery names underperform. Production 80/20 high-vol tilt was validated on S&P-like names; must re-validate before any universe broaden."
+  },
+  "platform_verdict": "ALIVE as breadth-book tilt candidate among momentum winners only \u2014 still needs a book-level experiment; not a production wire-in."
+}

reports/fip-breadth-diagnostics-20260718-213908.json

@@ -0,0 +1,100 @@
+{
+  "generated_at": "2026-07-18T21:39:07.916038",
+  "research_snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite",
+  "prod_subset_n": 506,
+  "panel_tickers": 4403,
+  "top_n": 1500,
+  "min_price": 5.0,
+  "checks": {
+    "fip_same_week_liquid_1500": {
+      "note": "Replication of main breadth run (same-week $vol mask)",
+      "mean_ic": -0.0168,
+      "ic_t_stat": -1.85,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 40.0,
+      "reliable": true
+    },
+    "fip_lagged_membership_1w": {
+      "note": "Liquid top-N ranked on *prior* week's median $vol \u2014 excludes same-week liquidity explosion leak",
+      "mean_ic": -0.0102,
+      "ic_t_stat": -0.93,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 40.0,
+      "reliable": true
+    },
+    "fip_tier_1_800": {
+      "note": "Same-week liquid ranks 1\u2013800 (senior liquid tier)",
+      "mean_ic": -0.035,
+      "ic_t_stat": -2.99,
+      "weeks": 35,
+      "avg_cross_section": 791.2,
+      "ic_positive_pct": 25.7,
+      "reliable": true
+    },
+    "fip_tier_801_1500": {
+      "note": "Same-week liquid ranks 801\u20131500 (junior liquid tier)",
+      "mean_ic": 0.0141,
+      "ic_t_stat": 1.25,
+      "weeks": 35,
+      "avg_cross_section": 700.0,
+      "ic_positive_pct": 60.0,
+      "reliable": true
+    },
+    "fip_prod_universe_subset": {
+      "note": "Symbols in prod.sqlite (~S&P-like large-cap book) inside same-week liquid top-N \u2014 compositional control",
+      "mean_ic": -0.0444,
+      "ic_t_stat": -2.88,
+      "weeks": 35,
+      "avg_cross_section": 497.5,
+      "ic_positive_pct": 25.7,
+      "reliable": true
+    },
+    "fip_momentum_conditional_top20pct": {
+      "note": "Among liquid top-N, keep mom_12_1 percentile \u2265 80.0 (paper: ID modulates continuation among winners; gate-relevant)",
+      "mean_ic": -0.0879,
+      "ic_t_stat": -4.58,
+      "weeks": 35,
+      "avg_cross_section": 294.3,
+      "ic_positive_pct": 22.9,
+      "reliable": true
+    },
+    "vol_6m_liquid_1500": {
+      "note": "Context: low-vol anomaly strength on this pool",
+      "mean_ic": -0.0465,
+      "ic_t_stat": -1.3,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 37.1,
+      "reliable": true
+    },
+    "mom_12_1_liquid_1500": {
+      "note": "Context: raw momentum on liquid breadth",
+      "mean_ic": 0.0462,
+      "ic_t_stat": 1.91,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 65.7,
+      "reliable": true
+    },
+    "mom_12_1_resid_liquid_1500": {
+      "note": "Context: residual momentum on liquid breadth",
+      "mean_ic": 0.0289,
+      "ic_t_stat": 1.33,
+      "weeks": 35,
+      "avg_cross_section": 1471.2,
+      "ic_positive_pct": 60.0,
+      "reliable": true
+    }
+  },
+  "interpretation": {
+    "leak_ruled_out": false,
+    "junior_tier_drives_positive": true,
+    "prod_subset_still_negative": true,
+    "mom_conditional_negative_and_reliable": true,
+    "compositional_flip_story": "If prod subset IC is negative while full liquid-1500 is positive, the sign flip is compositional (bleeders / Nasdaq junk), not a temporal regime change. Unconditional fip pools continuous winners (want neg IC) against continuous losers/bleeders (want pos IC).",
+    "vol_tilt_warning": "vol_6m large negative IC on breadth: high-vol lottery names underperform. Production 80/20 high-vol tilt was validated on S&P-like names; must re-validate before any universe broaden."
+  },
+  "platform_verdict": "ALIVE as breadth-book tilt candidate among momentum winners only \u2014 still needs a book-level experiment; not a production wire-in."
+}

scripts/run_fip_breadth_diagnostics.py

@@ -0,0 +1,665 @@
+"""Post-breadth diagnostics for fip_id (research branch only).
+
+Same research.sqlite as the liquid-breadth IC run. No production changes.
+
+Checks (pre-registered interpretation follow-ups)
+------------------------------------------------
+1. **Lagged membership** — liquid top-N ranked on *prior* week's $vol (extra lag)
+   so same-week liquidity explosion cannot pull a name into history.
+2. **Liquidity tiers** — fip IC on ranks 1–800 vs 801–1500 (same-week mask).
+3. **Prod-universe subset** — symbols present in prod.sqlite (~S&P-like large-cap
+   book) inside the same breadth weeks — compositional vs temporal flip.
+4. **Momentum-conditional fip** — among weekly top 20% by mom_12_1 (or resid when
+   available) within the liquid top-N — the paper's actual claim and the only
+   version a gate could consume.
+
+Also reports vol_6m / mom raw vs residual on the same panels for the log.
+
+Example (Windows)
+-----------------
+  .\\.venv\\Scripts\\python.exe scripts\\run_fip_breadth_diagnostics.py ^
+      --research-snapshot backtest_snapshots\\research.sqlite ^
+      --prod-snapshot backtest_snapshots\\prod.sqlite ^
+      --workers 6
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import multiprocessing as mp
+import sys
+from collections import defaultdict
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from datetime import date, datetime
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import create_engine, text
+
+ROOT = Path(__file__).resolve().parents[1]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+HORIZON = 30
+MIN_CROSS = 20
+MIN_RELIABLE = 12
+LIQUID_TOP = 1500
+MIN_PRICE = 5.0
+MOM_WINNER_PCT = 80.0  # top 20% within liquid cross-section
+
+
+def _parse_args() -> argparse.Namespace:
+    p = argparse.ArgumentParser(description=__doc__)
+    p.add_argument(
+        "--research-snapshot",
+        default="backtest_snapshots/research.sqlite",
+    )
+    p.add_argument(
+        "--prod-snapshot",
+        default="backtest_snapshots/prod.sqlite",
+        help="Symbols here define the large-cap / prod-like subset.",
+    )
+    p.add_argument("--top-n", type=int, default=LIQUID_TOP)
+    p.add_argument("--min-price", type=float, default=MIN_PRICE)
+    p.add_argument("--workers", type=int, default=max(1, (mp.cpu_count() or 4) - 1))
+    p.add_argument("--out", default=None)
+    p.add_argument("--quiet", action="store_true")
+    return p.parse_args()
+
+
+def _week_key(d: date) -> tuple[int, int]:
+    iso = d.isocalendar()
+    return (int(iso[0]), int(iso[1]))
+
+
+def _week_ord(wk: tuple[int, int]) -> int:
+    return wk[0] * 53 + wk[1]
+
+
+def _nonoverlap(weeks: list[tuple[int, int]], stride: int) -> list[tuple[int, int]]:
+    kept: list[tuple[int, int]] = []
+    last: int | None = None
+    for wk in sorted(weeks, key=_week_ord):
+        o = _week_ord(wk)
+        if last is None or o - last >= stride:
+            kept.append(wk)
+            last = o
+    return kept
+
+
+def _rank(xs: list[float]) -> list[float]:
+    order = sorted(range(len(xs)), key=lambda k: xs[k])
+    ranks = [0.0] * len(xs)
+    i = 0
+    while i < len(xs):
+        j = i
+        while j + 1 < len(xs) and xs[order[j + 1]] == xs[order[i]]:
+            j += 1
+        avg = (i + j) / 2.0 + 1.0
+        for k in range(i, j + 1):
+            ranks[order[k]] = avg
+        i = j + 1
+    return ranks
+
+
+def _pearson(a: list[float], b: list[float]) -> float | None:
+    n = len(a)
+    if n < 3:
+        return None
+    ma, mb = sum(a) / n, sum(b) / n
+    va = sum((x - ma) ** 2 for x in a)
+    vb = sum((y - mb) ** 2 for y in b)
+    if va <= 0 or vb <= 0:
+        return None
+    cov = sum((a[k] - ma) * (b[k] - mb) for k in range(n))
+    return cov / math.sqrt(va * vb)
+
+
+def _spearman(xs: list[float], ys: list[float]) -> float | None:
+    if len(xs) < 3:
+        return None
+    return _pearson(_rank(xs), _rank(ys))
+
+
+def _ic_row(pairs: list[tuple[float, float]], *, label: str) -> dict[str, Any]:
+    """pairs = (signal, fwd) over non-overlapping weeks aggregated… actually
+    we pass per-week then aggregate outside. This helper is for multi-week IC."""
+    raise NotImplementedError
+
+
+def _ic_from_weekly(
+    week_pairs: dict[tuple[int, int], list[tuple[float, float]]],
+) -> dict[str, Any]:
+    stride = max(1, round(HORIZON / 5))
+    usable = [wk for wk, ps in week_pairs.items() if len(ps) >= MIN_CROSS]
+    kept = _nonoverlap(usable, stride)
+    ics: list[float] = []
+    sizes: list[int] = []
+    for wk in kept:
+        ps = week_pairs[wk]
+        if len(ps) < MIN_CROSS:
+            continue
+        ic = _spearman([p[0] for p in ps], [p[1] for p in ps])
+        if ic is not None:
+            ics.append(ic)
+            sizes.append(len(ps))
+    if not ics:
+        return {
+            "mean_ic": None,
+            "ic_t_stat": None,
+            "weeks": 0,
+            "avg_cross_section": None,
+            "ic_positive_pct": None,
+            "reliable": False,
+        }
+    mean_ic = sum(ics) / len(ics)
+    if len(ics) > 1:
+        var = sum((x - mean_ic) ** 2 for x in ics) / (len(ics) - 1)
+        std = math.sqrt(var) if var > 0 else 0.0
+        t_stat = mean_ic / std * math.sqrt(len(ics)) if std > 0 else None
+    else:
+        t_stat = None
+    return {
+        "mean_ic": round(mean_ic, 4),
+        "ic_t_stat": round(t_stat, 2) if t_stat is not None else None,
+        "weeks": len(ics),
+        "avg_cross_section": round(sum(sizes) / len(sizes), 1),
+        "ic_positive_pct": round(sum(1 for x in ics if x > 0) / len(ics) * 100, 1),
+        "reliable": len(ics) >= MIN_RELIABLE,
+    }
+
+
+def _panel_worker(payload: tuple) -> list[dict]:
+    """Build weekly observations for one ticker (picklable top-level)."""
+    symbol, date_ords, opens, highs, lows, closes, volumes, spy = payload
+    from types import SimpleNamespace
+    from app.services.backtest_service import (
+        HORIZON as H,
+        _median_dollar_vol_63,
+        _signal_values,
+        _weekly_asof_indices,
+    )
+
+    dates = [date.fromordinal(int(o)) for o in date_ords]
+    opens_f = [float(x) for x in opens]
+    highs_f = [float(x) for x in highs]
+    lows_f = [float(x) for x in lows]
+    closes_f = [float(x) for x in closes]
+    vols_f = [float(x) for x in volumes]
+    n = len(closes_f)
+    if n < H + 21:
+        return []
+
+    # Match backtest_service bar objects exactly (weekly as-of + signal_values).
+    bar_records = [
+        SimpleNamespace(
+            date=dates[i],
+            open=opens_f[i],
+            high=highs_f[i],
+            low=lows_f[i],
+            close=closes_f[i],
+            volume=vols_f[i],
+        )
+        for i in range(n)
+    ]
+    out: list[dict] = []
+    for i in _weekly_asof_indices(bar_records):
+        j = i + H
+        if j >= n or closes_f[i] <= 0:
+            continue
+        sigs = _signal_values(dates, closes_f, highs_f, i, spy)
+        fip = sigs.get("fip_id")
+        mom = sigs.get("mom_12_1")
+        mom_r = sigs.get("mom_12_1_resid")
+        vol = sigs.get("vol_6m")
+        if fip is None and mom is None:
+            continue
+        dvol = _median_dollar_vol_63(closes_f, vols_f, i)
+        wk = _week_key(dates[i])
+        out.append({
+            "symbol": symbol,
+            "week": wk,
+            "fwd": closes_f[j] / closes_f[i] - 1.0,
+            "close": closes_f[i],
+            "dvol": dvol,
+            "fip_id": fip,
+            "mom_12_1": mom,
+            "mom_12_1_resid": mom_r,
+            "vol_6m": vol,
+        })
+    return out
+
+
+def _load_spy(conn) -> dict[date, float]:
+    rows = conn.execute(
+        text("SELECT date, close FROM benchmark_prices WHERE symbol = 'SPY' ORDER BY date")
+    ).fetchall()
+    out: dict[date, float] = {}
+    for d, c in rows:
+        if isinstance(d, str):
+            d = date.fromisoformat(d[:10])
+        out[d] = float(c)
+    return out
+
+
+def _load_symbols(conn) -> list[str]:
+    return [
+        str(r[0])
+        for r in conn.execute(text("SELECT symbol FROM tickers ORDER BY symbol")).fetchall()
+    ]
+
+
+def _load_columns(conn, symbol: str) -> tuple | None:
+    tid = conn.execute(
+        text("SELECT id FROM tickers WHERE symbol = :s"), {"s": symbol}
+    ).scalar()
+    if tid is None:
+        return None
+    rows = conn.execute(
+        text(
+            "SELECT date, open, high, low, close, volume FROM ohlcv_records "
+            "WHERE ticker_id = :t ORDER BY date"
+        ),
+        {"t": tid},
+    ).fetchall()
+    if len(rows) < HORIZON + 60:
+        return None
+    ords: list[int] = []
+    opens: list[float] = []
+    highs: list[float] = []
+    lows: list[float] = []
+    closes: list[float] = []
+    vols: list[float] = []
+    for d, o, h, l, c, v in rows:
+        if isinstance(d, str):
+            d = date.fromisoformat(d[:10])
+        ords.append(d.toordinal())
+        opens.append(float(o))
+        highs.append(float(h))
+        lows.append(float(l))
+        closes.append(float(c))
+        vols.append(float(v or 0))
+    return (symbol, ords, opens, highs, lows, closes, vols)
+
+
+def _liquid_members(
+    obs: list[dict],
+    *,
+    top_n: int,
+    min_price: float,
+    dvol_key: str = "dvol",
+) -> list[dict]:
+    eligible = [
+        o
+        for o in obs
+        if o.get("close") is not None
+        and float(o["close"]) >= min_price
+        and o.get(dvol_key) is not None
+        and float(o[dvol_key]) > 0
+    ]
+    eligible.sort(key=lambda o: float(o[dvol_key]), reverse=True)
+    return eligible[:top_n]
+
+
+def _pairs(obs: list[dict], signal: str) -> list[tuple[float, float]]:
+    out: list[tuple[float, float]] = []
+    for o in obs:
+        v = o.get(signal)
+        if v is None:
+            continue
+        out.append((float(v), float(o["fwd"])))
+    return out
+
+
+def main() -> None:
+    args = _parse_args()
+    research = Path(args.research_snapshot)
+    prod = Path(args.prod_snapshot)
+    if not research.exists():
+        raise SystemExit(f"Missing research snapshot: {research}")
+
+    research_eng = create_engine(f"sqlite:///{research.resolve().as_posix()}")
+    prod_symbols: set[str] = set()
+    if prod.exists():
+        prod_eng = create_engine(f"sqlite:///{prod.resolve().as_posix()}")
+        with prod_eng.connect() as c:
+            prod_symbols = {
+                str(r[0])
+                for r in c.execute(text("SELECT symbol FROM tickers")).fetchall()
+            }
+        prod_eng.dispose()
+
+    with research_eng.connect() as conn:
+        spy = _load_spy(conn)
+        symbols = _load_symbols(conn)
+        jobs: list[tuple] = []
+        for i, sym in enumerate(symbols, 1):
+            cols = _load_columns(conn, sym)
+            if cols is None:
+                continue
+            jobs.append((*cols, spy))
+            if not args.quiet and i % 500 == 0:
+                print(f"  queued {i}/{len(symbols)}", flush=True)
+
+    if not args.quiet:
+        print(f"Building weekly panel for {len(jobs)} tickers…", flush=True)
+
+    # Panel: week -> list of obs
+    by_week: dict[tuple[int, int], list[dict]] = defaultdict(list)
+    workers = max(1, int(args.workers))
+    if workers == 1:
+        for j, job in enumerate(jobs, 1):
+            for row in _panel_worker(job):
+                by_week[tuple(row["week"])].append(row)
+            if not args.quiet and j % 200 == 0:
+                print(f"  panel {j}/{len(jobs)}", flush=True)
+    else:
+        with ProcessPoolExecutor(max_workers=workers) as pool:
+            futs = {pool.submit(_panel_worker, job): job[0] for job in jobs}
+            done = 0
+            for fut in as_completed(futs):
+                done += 1
+                try:
+                    rows = fut.result()
+                except Exception as exc:
+                    if not args.quiet:
+                        print(f"  worker error {futs[fut]}: {exc}", flush=True)
+                    continue
+                for row in rows:
+                    by_week[tuple(row["week"])].append(row)
+                if not args.quiet and done % 200 == 0:
+                    print(f"  panel {done}/{len(jobs)}", flush=True)
+
+    if not args.quiet:
+        print(f"Weeks with data: {len(by_week)}", flush=True)
+
+    # Prior-week dvol map for lagged membership: (symbol, week) -> dvol
+    dvol_by_sym_week: dict[tuple[str, tuple[int, int]], float] = {}
+    for wk, obs in by_week.items():
+        for o in obs:
+            if o.get("dvol") is not None:
+                dvol_by_sym_week[(o["symbol"], wk)] = float(o["dvol"])
+
+    ordered_weeks = sorted(by_week.keys(), key=_week_ord)
+    prev_week: dict[tuple[int, int], tuple[int, int]] = {}
+    for i, wk in enumerate(ordered_weeks):
+        if i > 0:
+            prev_week[wk] = ordered_weeks[i - 1]
+
+    top_n = int(args.top_n)
+    min_price = float(args.min_price)
+
+    # --- Panels for each check ---
+    same_week_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    lag_week_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    tier_hi_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    tier_lo_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    prod_subset_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    mom_cond_fip: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    liquid_vol: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    liquid_mom: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+    liquid_mom_r: dict[tuple[int, int], list[tuple[float, float]]] = defaultdict(list)
+
+    for wk, obs in by_week.items():
+        # Same-week liquid top-N among names that have fip (matches signal_eval mask:
+        # membership is ranked within each signal's observation set).
+        with_fip = [o for o in obs if o.get("fip_id") is not None]
+        liq_fip = _liquid_members(with_fip, top_n=top_n, min_price=min_price)
+        for rank, o in enumerate(liq_fip, 1):
+            same_week_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+            if rank <= 800:
+                tier_hi_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+            elif rank <= top_n:
+                tier_lo_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+            if o["symbol"] in prod_symbols:
+                prod_subset_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+
+        # Context signals: liquid among names that carry that signal
+        with_vol = [o for o in obs if o.get("vol_6m") is not None]
+        for o in _liquid_members(with_vol, top_n=top_n, min_price=min_price):
+            liquid_vol[wk].append((float(o["vol_6m"]), float(o["fwd"])))
+        with_mom_all = [o for o in obs if o.get("mom_12_1") is not None]
+        liq_mom = _liquid_members(with_mom_all, top_n=top_n, min_price=min_price)
+        for o in liq_mom:
+            liquid_mom[wk].append((float(o["mom_12_1"]), float(o["fwd"])))
+        with_mom_r = [o for o in obs if o.get("mom_12_1_resid") is not None]
+        for o in _liquid_members(with_mom_r, top_n=top_n, min_price=min_price):
+            liquid_mom_r[wk].append((float(o["mom_12_1_resid"]), float(o["fwd"])))
+
+        # Momentum-conditional: within liquid fip set, keep mom_12_1 ≥ P80
+        mom_key = "mom_12_1"
+        with_mom = [
+            o for o in liq_fip
+            if o.get(mom_key) is not None and o.get("fip_id") is not None
+        ]
+        if len(with_mom) >= MIN_CROSS:
+            with_mom.sort(key=lambda o: float(o[mom_key]))
+            n = len(with_mom)
+            cut = int(math.floor(n * (MOM_WINNER_PCT / 100.0)))
+            winners = with_mom[cut:]  # upper tail
+            for o in winners:
+                mom_cond_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+
+        # Lagged membership: rank by *previous* week's dvol among fip names
+        pw = prev_week.get(wk)
+        if pw is not None:
+            lagged: list[dict] = []
+            for o in with_fip:
+                if o.get("close") is None or float(o["close"]) < min_price:
+                    continue
+                prev_dvol = dvol_by_sym_week.get((o["symbol"], pw))
+                if prev_dvol is None or prev_dvol <= 0:
+                    continue
+                lagged.append({**o, "lag_dvol": prev_dvol})
+            lagged.sort(key=lambda o: float(o["lag_dvol"]), reverse=True)
+            for o in lagged[:top_n]:
+                lag_week_fip[wk].append((float(o["fip_id"]), float(o["fwd"])))
+
+    results = {
+        "generated_at": datetime.now().isoformat(),
+        "research_snapshot": str(research.resolve()),
+        "prod_subset_n": len(prod_symbols),
+        "panel_tickers": len(jobs),
+        "top_n": top_n,
+        "min_price": min_price,
+        "checks": {
+            "fip_same_week_liquid_1500": {
+                "note": "Replication of main breadth run (same-week $vol mask)",
+                **_ic_from_weekly(same_week_fip),
+            },
+            "fip_lagged_membership_1w": {
+                "note": (
+                    "Liquid top-N ranked on *prior* week's median $vol — "
+                    "excludes same-week liquidity explosion leak"
+                ),
+                **_ic_from_weekly(lag_week_fip),
+            },
+            "fip_tier_1_800": {
+                "note": "Same-week liquid ranks 1–800 (senior liquid tier)",
+                **_ic_from_weekly(tier_hi_fip),
+            },
+            "fip_tier_801_1500": {
+                "note": "Same-week liquid ranks 801–1500 (junior liquid tier)",
+                **_ic_from_weekly(tier_lo_fip),
+            },
+            "fip_prod_universe_subset": {
+                "note": (
+                    "Symbols in prod.sqlite (~S&P-like large-cap book) inside "
+                    "same-week liquid top-N — compositional control"
+                ),
+                **_ic_from_weekly(prod_subset_fip),
+            },
+            "fip_momentum_conditional_top20pct": {
+                "note": (
+                    f"Among liquid top-N, keep mom_12_1 percentile ≥ {MOM_WINNER_PCT} "
+                    "(paper: ID modulates continuation among winners; gate-relevant)"
+                ),
+                **_ic_from_weekly(mom_cond_fip),
+            },
+            "vol_6m_liquid_1500": {
+                "note": "Context: low-vol anomaly strength on this pool",
+                **_ic_from_weekly(liquid_vol),
+            },
+            "mom_12_1_liquid_1500": {
+                "note": "Context: raw momentum on liquid breadth",
+                **_ic_from_weekly(liquid_mom),
+            },
+            "mom_12_1_resid_liquid_1500": {
+                "note": "Context: residual momentum on liquid breadth",
+                **_ic_from_weekly(liquid_mom_r),
+            },
+        },
+    }
+
+    # Interpretations
+    checks = results["checks"]
+    lag = checks["fip_lagged_membership_1w"]
+    same = checks["fip_same_week_liquid_1500"]
+    hi = checks["fip_tier_1_800"]
+    lo = checks["fip_tier_801_1500"]
+    prod = checks["fip_prod_universe_subset"]
+    cond = checks["fip_momentum_conditional_top20pct"]
+
+    def _sign(x: float | None) -> str:
+        if x is None:
+            return "na"
+        return "neg" if x < 0 else "pos"
+
+    results["interpretation"] = {
+        "leak_ruled_out": (
+            lag.get("mean_ic") is not None
+            and same.get("mean_ic") is not None
+            and _sign(lag["mean_ic"]) == _sign(same["mean_ic"])
+            and abs(float(lag["mean_ic"])) >= 0.02
+        ),
+        "junior_tier_drives_positive": (
+            lo.get("mean_ic") is not None
+            and float(lo["mean_ic"]) > 0
+            and (hi.get("mean_ic") is None or float(hi["mean_ic"]) < float(lo["mean_ic"]))
+        ),
+        "prod_subset_still_negative": (
+            prod.get("mean_ic") is not None and float(prod["mean_ic"]) < 0
+        ),
+        "mom_conditional_negative_and_reliable": (
+            cond.get("mean_ic") is not None
+            and float(cond["mean_ic"]) < 0
+            and abs(float(cond["mean_ic"])) >= 0.03
+            and bool(cond.get("reliable"))
+        ),
+        "compositional_flip_story": (
+            "If prod subset IC is negative while full liquid-1500 is positive, "
+            "the sign flip is compositional (bleeders / Nasdaq junk), not a "
+            "temporal regime change. Unconditional fip pools continuous winners "
+            "(want neg IC) against continuous losers/bleeders (want pos IC)."
+        ),
+        "vol_tilt_warning": (
+            "vol_6m large negative IC on breadth: high-vol lottery names "
+            "underperform. Production 80/20 high-vol tilt was validated on "
+            "S&P-like names; must re-validate before any universe broaden."
+        ),
+    }
+
+    # Gate-relevant summary line
+    if results["interpretation"]["mom_conditional_negative_and_reliable"]:
+        results["platform_verdict"] = (
+            "ALIVE as breadth-book tilt candidate among momentum winners only — "
+            "still needs a book-level experiment; not a production wire-in."
+        )
+    else:
+        results["platform_verdict"] = (
+            "CLOSED for production use: momentum-conditional fip does not clear "
+            "iron rule on this liquid-Nasdaq pool. Display card remains final resting place."
+        )
+
+    stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
+    out = Path(args.out) if args.out else Path("reports") / f"fip-breadth-diagnostics-{stamp}.json"
+    out.parent.mkdir(parents=True, exist_ok=True)
+    out.write_text(json.dumps(results, indent=2, default=str), encoding="utf-8")
+
+    # Append to research log
+    md_path = Path("docs/research/fip-breadth-ic.md")
+    _append_diagnostics_md(md_path, results, out)
+
+    if not args.quiet:
+        print(json.dumps(results["checks"], indent=2, default=str))
+        print()
+        print("interpretation:", json.dumps(results["interpretation"], indent=2))
+        print("platform_verdict:", results["platform_verdict"])
+        print(f"Wrote {out}")
+        print(f"Updated {md_path}")
+
+
+def _append_diagnostics_md(path: Path, results: dict, artifact: Path) -> None:
+    checks = results["checks"]
+    interp = results["interpretation"]
+    lines = [
+        "",
+        "---",
+        "",
+        f"## Follow-up diagnostics ({results['generated_at'][:10]})",
+        "",
+        "Compositional reading of the sign flip (before any 'jumpiness premium' story):",
+        "",
+        "`fip_id = sign(PRET) × (%neg − %pos)` pools two opposite continuous populations:",
+        "",
+        "- **Continuous winners** (PRET>0, mostly up days) → paper claim → **negative** IC contribution.",
+        "- **Continuous losers / bleeders** (PRET<0, mostly down days) → momentum continuation down → **positive** IC contribution.",
+        "",
+        "Unconditional IC is a tug-of-war weighted by universe composition. S&P-like books "
+        "have few steady bleeders → negative fip IC. Liquid Nasdaq has many → sign can flip "
+        "without contradicting Da/Gurun/Warachka (claim was always **momentum-conditional**).",
+        "",
+        "### Artifact / composition checks",
+        "",
+        "| check | mean_ic | t | weeks | avg N | reliable |",
+        "|---|---:|---:|---:|---:|---|",
+    ]
+    order = [
+        "fip_same_week_liquid_1500",
+        "fip_lagged_membership_1w",
+        "fip_tier_1_800",
+        "fip_tier_801_1500",
+        "fip_prod_universe_subset",
+        "fip_momentum_conditional_top20pct",
+        "vol_6m_liquid_1500",
+        "mom_12_1_liquid_1500",
+        "mom_12_1_resid_liquid_1500",
+    ]
+    for key in order:
+        row = checks.get(key) or {}
+        lines.append(
+            f"| {key} | {row.get('mean_ic')} | {row.get('ic_t_stat')} | "
+            f"{row.get('weeks')} | {row.get('avg_cross_section')} | {row.get('reliable')} |"
+        )
+    lines.extend([
+        "",
+        "### Flags",
+        "",
+        f"- Lagged mask keeps same sign / material |IC|: **{interp.get('leak_ruled_out')}**",
+        f"- Junior tier (801–1500) drives more positive IC: **{interp.get('junior_tier_drives_positive')}**",
+        f"- Prod-universe subset still negative: **{interp.get('prod_subset_still_negative')}**",
+        f"- Mom-conditional (≥P80) negative + reliable: **{interp.get('mom_conditional_negative_and_reliable')}**",
+        "",
+        "### Platform verdict",
+        "",
+        results.get("platform_verdict", ""),
+        "",
+        "### Vol-tilt warning (any future breadth move)",
+        "",
+        interp.get("vol_tilt_warning", ""),
+        "",
+        f"Artifact: `{artifact.as_posix()}`",
+        "",
+    ])
+    # Replace previous diagnostics section if re-run, else append
+    existing = path.read_text(encoding="utf-8") if path.exists() else ""
+    marker = "## Follow-up diagnostics"
+    if marker in existing:
+        existing = existing.split(marker)[0].rstrip() + "\n"
+    path.write_text(existing + "\n".join(lines), encoding="utf-8")
+
+
+if __name__ == "__main__":
+    main()