research: park Phase B fip breadth; race guard and compact evidence

Log the 21:14 orphan as a snapshot-build race, rewrite the context table to
authoritative ICs only, and soften the vol-tilt warning. Add extender completion
manifest + breadth refuse guard; strip intermediate/orphaned reports; park the
thread (no book sim, no deploy).
This commit is contained in:
2026-07-19 00:32:20 +02:00
parent 7d60e54f5a
commit 2311999e57
16 changed files with 562 additions and 8103 deletions
+2 -2
View File
@@ -263,7 +263,7 @@ A systematic single-variable sweep (offline prod snapshot, production gate/rank/
Two findings future sessions must not re-litigate: Two findings future sessions must not re-litigate:
- **The "inverse-vol sizing win" (July 2026) was mis-attributed — do not resurrect.** The diagnostic sized `notional = equity × 1% / vol_6m`, and the 20% notional cap bound on 95% of entries, so it actually measured "~5 positions × 20% notional each" — a concentration/risk-appetite bump economically equivalent to raising risk to 1.5%, not vol-managed sizing. Genuine inverse-vol sizing (risk budget × median-vol/vol) cuts max drawdown to 18.2% but costs ~58pp total return at flat Sharpe: a risk-preference trade, not edge. - **The "inverse-vol sizing win" (July 2026) was mis-attributed — do not resurrect.** The diagnostic sized `notional = equity × 1% / vol_6m`, and the 20% notional cap bound on 95% of entries, so it actually measured "~5 positions × 20% notional each" — a concentration/risk-appetite bump economically equivalent to raising risk to 1.5%, not vol-managed sizing. Genuine inverse-vol sizing (risk budget × median-vol/vol) cuts max drawdown to 18.2% but costs ~58pp total return at flat Sharpe: a risk-preference trade, not edge.
- **`fip_id` — Da/Gurun/Warachka information discreteness over the 12-1 formation window — is the strongest cross-sectional signal measured on this universe: IC 0.045, t = 2.91, correct sign (continuous-information winners outperform).** It clears the iron-rule bar in isolation but does not improve this book (the momentum gate already captures the effect in-sample). It is the prime ranking/gate candidate **if the universe broadens** (e.g. `nasdaq_all`). - **`fip_id` — Da/Gurun/Warachka information discreteness over the 12-1 formation window — is the strongest cross-sectional signal on the *production* universe: IC 0.045, t = 2.91, correct sign (continuous-information winners outperform).** It clears the iron-rule bar in isolation but does not improve this book (the momentum gate already captures the effect in-sample). **Phase B (liquid-1500, research branch only):** unconditional fip fails iron rule (0.017 / t 1.85); mom-conditional fip (0.088 / t 4.58) is a *book-tilt candidate only* after a baseline breadth mom book is proven. Do **not** cite the orphaned 21:14 row (+0.0575) — it raced a partial `research.sqlite`. See `docs/research/fip-breadth-ic.md`.
### The iron rule for strategy changes ### The iron rule for strategy changes
@@ -281,7 +281,7 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m
1. **Forward monitor the promoted strategy** — the production UI now behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison. Forward paper-trade months are the only evidence the snapshot cannot provide; the July 2026 tuning pass closed every in-sample lead. (Trailing-stop sensitivity and the max-15 capacity check are done — see the tuning table above.) 1. **Forward monitor the promoted strategy** — the production UI now behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison. Forward paper-trade months are the only evidence the snapshot cannot provide; the July 2026 tuning pass closed every in-sample lead. (Trailing-stop sensitivity and the max-15 capacity check are done — see the tuning table above.)
2. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only. 2. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only.
3. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. Now doubly motivated: it is also where the strong `fip_id` signal (see tuning findings) could become tradeable. (Deeper history was considered and declined.) 3. **Breadth is no longer free leverage** — Phase B found residual-mom t-stat *fell* on liquid-1500 vs the 505-name fingerprint (0.055/1.98 → 0.029/1.33). Any breadth book must clear a pre-registered baseline arm before fip tilts mean anything. (Deeper history was considered and declined.)
## Key Use Cases ## Key Use Cases
+8 -3
View File
@@ -140,9 +140,9 @@ knobs.
| Lead | Why it's interesting | Blocker | | Lead | Why it's interesting | Blocker |
|---|---|---| |---|---|---|
| **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Implement schedule + partial-bar scan path; one qualifying scan/day only | | **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Schedule + fill_mode shipped; live paper validation ongoing |
| **`fip_id`** | Fingerprint IC 0.045 / t 2.91 on prod book; display-only on ticker technicals | **Phase B:** unconditional liquid-Nasdaq IC fails iron-rule **sign**; **mom-conditional** fip IC 0.088 / t 4.58 (alive as tilt candidate only). See [fip-breadth-ic.md](fip-breadth-ic.md) | | **`fip_id` / liquid breadth** | Fingerprint 0.045 / t 2.91; liquid unconditional **0.017 / t 1.85** (not green); mom-conditional **0.088 / t 4.58** | **Parked.** Orphan +0.0575 died (snapshot race). Breadth did not strengthen resid-mom t-stat. Optional reopen = pre-registered two-arm liquid-1500 book first. See [fip-breadth-ic.md](fip-breadth-ic.md) |
| **Broader universe** | Composition changes factor signs (fip tug-of-war; high-vol junk) | Any prod broaden must **re-validate 80/20 high-vol tilt** first; offline research only for now | | **Broader universe** | Composition changes factor signs (fip tug-of-war); vol-tilt on breadth is only a **directional hypothesis** (auth. 0.048 / t 1.36) | Any prod broaden must re-validate 80/20 tilt; offline research only; research.sqlite requires completion manifest |
| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships | | **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
| **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR | | **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
@@ -169,6 +169,11 @@ knobs.
6. **Fill timing is part of the strategy.** Close-fill reports are not deployable 6. **Fill timing is part of the strategy.** Close-fill reports are not deployable
numbers for an overnight scanner. Grade promotion under the fill mode you will numbers for an overnight scanner. Grade promotion under the fill mode you will
actually trade. actually trade.
7. **Incomplete research artifacts are not results.** The Phase B +0.0575 / t +5.12
liquid-fip row was orphaned within hours: it raced a partially built
`research.sqlite`. Extender now writes a completion manifest; breadth mode
refuses without a match. Same class of protection as calendar-truncation
asserts — do not re-mythologize numbers computed on half a universe.
--- ---
+97 -26
View File
@@ -1,13 +1,15 @@
# Broad-universe fip_id IC research (Phase B) # Broad-universe fip_id IC research (Phase B)
**Status:** unconditional fip closed; mom-conditional lead confirmed on single-sourced path. **Status:** **Parked / closed for now.** Unconditional fip not green; mom-conditional lead logged; breadth-momentum thesis challenged. No book sim until reopen.
**Production impact:** none. Display card remains context-only. **Production impact:** none. Display card remains context-only. No deploy from this work.
**Artifacts:** research log + compact reports + env-gated harness hooks; tooling stays for a future reopen.
## Scope ## Scope
- Research only — production universe, gate, scanner, schedule unchanged. - Research only — production universe, gate, scanner, schedule unchanged.
- Snapshot: `research.sqlite` (~4,650 tickers = prod + nasdaq_all extend). - Snapshot: `research.sqlite` (~4,650 tickers = prod + nasdaq_all extend).
- Liquid mask: top **1,500** by point-in-time 63d median $vol, price ≥ **$5**/week. - Liquid mask: top **1,500** by point-in-time 63d median $vol, price ≥ **$5**/week.
- **Completion manifest required:** extender writes `<snapshot>.manifest.json`; breadth runners refuse without a matching complete manifest (see §Race guard).
## Caveats ## Caveats
@@ -15,6 +17,7 @@
- IEX volume undercount → relative $vol rank only. - IEX volume undercount → relative $vol rank only.
- Pool skew: Nasdaq-heavy; missing pure NYSE mid-caps. - Pool skew: Nasdaq-heavy; missing pure NYSE mid-caps.
- Do not mix multi-signal tables across universe baselines. - Do not mix multi-signal tables across universe baselines.
- **Do not cite orphaned 21:14 numbers** (see below).
--- ---
@@ -28,47 +31,83 @@
**Pass.** Formula + pipeline trustworthy. **Pass.** Formula + pipeline trustworthy.
Residual momentum on the same fingerprint (what the production book ranks on): **IC +0.055 / t +1.98**.
--- ---
## Discrepancy (must not be papered over) ## The orphan (21:14) — root cause
| Source | fip IC (liquid ~1500) | t | | Source | fip IC (liquid ~1500) | t |
|---|---:|---:| |---|---:|---:|
| Report `fip-breadth-20260718-211440-breadth.json` | **+0.0575** | **+5.12** | | Orphan run 21:14 (removed from tree; was `fip-breadth-20260718-211440-breadth.json`) | **+0.0575** | **+5.12** |
| Single-sourced recompute (2026-07-19) | **0.0168** | **1.85** | | Single-sourced recompute on complete snapshot (2026-07-19) | **0.0168** | **1.85** |
That is a **sign disagreement** on the same intended quantity. Method rule: the number you cannot reconcile is the number you cannot use. That is a **sign disagreement** on the same intended quantity. Method rule: the number you cannot reconcile is the number you cannot use.
### What we did ### Verdict: orphaned — raced the snapshot build
1. **Single-sourced the mask** — diagnostics call harness `_signal_series` + `_filter_liquid_breadth_week_rich` only (no parallel mask). **Not** “orphaned, unexplained.” The mechanism is derivable from the table itself:
2. **Documented avg_cross_section semantics** — always **post-mask** IC sample size.
3. **Logged pre-mask stats** so “did top-N bind?” is answerable.
### Authoritative unconditional liquid fip (post-reconciliation) 1. **Code was not the difference.** Reconcile shows the old harness path and the new shared filter produce **identical** results on current data (0.0168 / 1.85). The implementation fork is closed.
2. **Data was the difference.** On todays complete snapshot the liquid mask **binds in 97.1% of weeks** at top-N = 1,500. Dense signals (e.g. `vol_6m`) post-mask at **exactly 1,500**. The orphaned reports `vol_6m` averaged **~1,475** cross-section — a masked run on complete data cannot do that. At 21:14 the eligible pool was smaller than 1,500 and the mask never bound.
3. **Timeline fits.** Extender fixes landed ~20:32 / 20:34; full fetch takes ~30 minutes; breadth run fired **21:14** against a partially built `research.sqlite`. Every number in that report was computed on an incomplete universe.
**Do not cite +0.0575 / t +5.12.** It survived less than six hours of contact with project discipline — that is the system working, not time wasted. The orphan JSON was **deleted from the tree** (still in Git history) so it cannot be re-imported as evidence.
**Kept artifacts**
| File | Role |
|---|---|
| `reports/fip-reconcile-20260719-000520.json` | Authoritative single-sourced ICs (compact; membership dumps stripped) |
| `reports/fip-breadth-20260718-211440-fingerprint.json` | Prod fingerprint pass |
### Race guard (same class as calendar truncation)
| Piece | Behavior |
|---|---|
| `extend_snapshot_universe.py` | Clears any prior manifest on start; on full completion writes `<output>.manifest.json` with `complete=true`, ticker / OHLCV / rank_only counts, `finished_at`. `--limit` smoke runs write `complete=false`. |
| `run_fip_breadth_research.py` / `run_fip_breadth_diagnostics.py` | **Refuse** breadth mode unless a matching complete manifest exists and live counts equal the recorded totals. |
Helper: `scripts/research_snapshot_manifest.py`.
---
## Authoritative unconditional liquid fip (post-reconciliation)
| metric | value | | metric | value |
|---|---:| |---|---:|
| mean_ic | **0.0168** | | mean_ic | **0.0168** |
| ic_t_stat | **1.85** | | ic_t_stat | **1.85** |
| weeks | 35 | | weeks | 35 |
| avg_cross_section (**post-mask**) | 1471.2 | | avg_cross_section (**post-mask IC sample**) | 1471.2 |
| avg_raw_pool | 3214.4 | | avg_raw_pool | 3214.4 |
| avg_eligible_pre_mask | **2338.4** | | avg_eligible_pre_mask | **2338.4** |
| mask_binds_pct | **97.1%** | | mask_binds_pct | **97.1%** |
| reliable | true | | reliable | true |
**Mask binds hard** (eligible ≫ 1500). The hypothesis that “1471 meant the mask never bound / unmasked +5σ” is **false**. **Mask binds hard** on complete data (eligible ≫ 1500). Post-mask IC N for fip is ~1471 because not every liquid name has a valid 12-1 fip path — that is signal availability, not a non-binding mask. Contrast orphan `vol_6m` avg N ~1475 vs complete-data `vol_6m` avg N **1500**.
Harness `_signal_evaluation` vs manual IC through the same filter: **exact match** (0.0168 / 1.85). Harness `_signal_evaluation` vs manual IC through the same filter: **exact match** (0.0168 / 1.85).
### Verdict on the orphan **Iron rule unconditional:** **not green** (|IC| 0.017 < 0.03), correct mild-negative sign.
The **+0.0575 / t +5.12** row is **orphaned**. Do not cite it. Root cause of that single run is not fully forensic-reconstructed (no dual dump from the original process remains), but every single-sourced recompute on this snapshot lands near **0.017**, and the tier blend (≈800×−0.035 + ≈670×+0.014)/1471 ≈ **0.013** is internally consistent with that number—not with +0.058. ---
**Iron rule unconditional:** still **not green** (|IC| 0.017 < 0.03), and now with the correct mild-negative sign. ## Context table (orphaned 21:14 vs authoritative) — kill the myth numbers
Artifact: `reports/fip-reconcile-20260719-000520.json` The context table died with the orphan. **0.16 must not survive in the log.**
| signal (liquid ~1500) | orphaned (21:14) | authoritative (shared filter) | consequence |
|---|---:|---:|---|
| **vol_6m** | 0.16 / t **6.1** | **0.048 / t 1.36** | “High-vol tilt harmful on breadth” **downgrades from finding to directional hypothesis** — not significant |
| **raw mom** (`mom_12_1`) | +0.10 / t +4.6 | **+0.046 / t +1.91** | Below iron-rule bar on this pool |
| **resid mom** (`mom_12_1_resid`) | +0.04 / t +2.3 | **+0.029 / t +1.33** | Ditto, and weaker than raw |
### Breadth-momentum thesis — challenged
That last pair is the sobering one. Momentum on liquid breadth is **marginal**. The “more breadth strengthens the momentum t-stat” thesis that motivated Phase B is **empirically wrong on this pool**: same 35 weeks, triple the names, residual-mom t-stat **fell** versus the 505-name fingerprint (**0.055 / 1.98** → **0.029 / 1.33**). The clean momentum edge lives in the large-cap universe already traded.
Meanwhile the strongest reliable signal on liquid breadth is now **mom-conditional fip** (0.088 / 4.58) — but a fip tilt presupposes a breadth momentum book worth tilting, and that is no longer free.
--- ---
@@ -107,27 +146,57 @@ Computed on the **same single-sourced path** as the authoritative 0.017. This
| Decision | | | Decision | |
|---|---| |---|---|
| Unconditional fip | **Closed** for production | | Unconditional fip | **Closed** for production |
| Mom-conditional fip | **Alive as book-tilt candidate only**book sim before any gate talk | | Mom-conditional fip | **Alive as book-tilt candidate only**and only after a baseline breadth book proves itself |
| Display card | Stays | | Display card | Stays |
| Production change | **None** | | Production change | **None** |
--- ---
## Vol-tilt / residual-mom warning (any future breadth move) ## Vol-tilt warning (softened)
| signal (liquid, single-sourced) | IC | t | | signal (liquid, single-sourced) | IC | t |
|---|---:|---:| |---|---:|---:|
| vol_6m | 0.048 | 1.4 | | vol_6m | 0.048 | **1.36** |
| mom_12_1 | +0.046 | +1.9 | | mom_12_1 | +0.046 | +1.91 |
| mom_12_1_resid | +0.029 | +1.3 | | mom_12_1_resid | +0.029 | +1.33 |
High-vol names tend to underperform on this pool relative to a clean S&P-like book. Production **80/20 high-vol tilt** was validated on S&P-like names. **If the universe ever broadens in production, re-validate that tilt first** — it can flip from mildly helpful to harmful. Raw momentum also looks stronger than SPY residualization here (noisier fit for small caps). High-vol names **tend** to underperform on this pool relative to a clean S&P-like book — that is a **directional hypothesis**, not a finding. Production **80/20 high-vol tilt** was validated on S&P-like names. If the universe ever broadens in production, re-validate that tilt; do not treat the orphaned 0.16 / t 6.1 as evidence.
---
## What this means for the book experiment
A fip tilt presupposes a breadth momentum book worth tilting — **that is no longer free.**
**Caution against over-reacting the other way:** modest cross-sectional IC does not preclude a good book. The 505-name book turns resid-mom IC ~0.055 into Sharpe ~2 because the gate trades the **extreme tail**, not the linear sort. The breadth book might still work; it just has to **prove it** before the fip arm means anything. If the baseline cannot clearly beat the existing production books territory, fips future is a footnote regardless of 4.58.
### Parked next step (if reopened): pre-registered two-arm design
Not started — **design only**, pre-register before any sim:
| Arm | Definition |
|---|---|
| **A — baseline** | Top-quintile residual (or raw — pick one and lock) momentum book on liquid-1500; **no fip**; honest costs; next-open or near-close fills; production-like capacity / risk / stops |
| **B — +fip tilt** | Same book + mom-conditional fip tilt (among mom winners, prefer smoother paths / negative fip_id) |
| Grade on | Spec |
|---|---|
| Split | Entry-date train / validation (`BACKTEST_HOLDOUT_SPLIT` naming — not pristine holdout) |
| Metrics | Sharpe + Mertens/Lo SE, PSR, **DSR**; max DD; turnover; cost drag |
| Promote bar | Arm A must be in production-book territory first; Arm B must beat A on validation with DSR-aware multiple-testing honesty |
| Fail-closed | If A fails, fip is a footnote; do not shop tilts on a dead baseline |
--- ---
## How to re-run (research branch only) ## How to re-run (research branch only)
```powershell ```powershell
# 1) Full extend writes completion manifest (required)
.\.venv\Scripts\python.exe scripts\extend_snapshot_universe.py `
--source backtest_snapshots\prod.sqlite `
--output backtest_snapshots\research.sqlite
# 2) Breadth / diagnostics refuse without matching manifest
.\.venv\Scripts\python.exe scripts\run_fip_breadth_diagnostics.py ` .\.venv\Scripts\python.exe scripts\run_fip_breadth_diagnostics.py `
--research-snapshot backtest_snapshots\research.sqlite ` --research-snapshot backtest_snapshots\research.sqlite `
--prod-snapshot backtest_snapshots\prod.sqlite ` --prod-snapshot backtest_snapshots\prod.sqlite `
@@ -139,7 +208,9 @@ High-vol names tend to underperform on this pool relative to a clean S&P-like bo
## Bottom line ## Bottom line
1. Formal iron-rule screen: **not green** either before or after reconciliation. 1. Formal iron-rule screen: **not green** either before or after reconciliation.
2. **+0.0575 / +5.12 is orphaned** — authoritative unconditional liquid fip is **0.017 / 1.9**; mask binds (~97%). 2. **+0.0575 / +5.12 is orphaned: raced the snapshot build** — authoritative unconditional liquid fip is **0.017 / 1.9**; mask binds (~97%) on complete data.
3. Compositional tug-of-war is the right story; jumpiness premium is not. 3. Context-table myths die with the orphan: **vol 0.16 is not real**; authoritative vol is **0.048 / t 1.36** (directional only).
4. **Mom-conditional 0.088 / 4.6 stands on the single-sourced path** → optional next research step is a **book** A/B, not a gate wire-in. 4. Compositional tug-of-war is the right story; jumpiness premium is not.
5. Log any future reader who sees both numbers: trust the reconcile artifact, not the orphaned breadth headline. 5. **Breadth does not strengthen residual-mom t-stat** on this pool (0.055/1.98 → 0.029/1.33).
6. **Mom-conditional 0.088 / 4.6 stands** on the single-sourced path → optional next step is a **pre-registered two-arm breadth book** (baseline first), not a gate wire-in.
7. Manifest guard is in place so the race cannot recur silently.
+20
View File
@@ -41,3 +41,23 @@ rejected stop-adjustment path, and add no decision evidence beyond the final
daily matrix and narrative. Their matching one-off runners were removed too. daily matrix and narrative. Their matching one-off runners were removed too.
All remain recoverable from Git history. Rebuildable candidate pickle caches All remain recoverable from Git history. Rebuildable candidate pickle caches
are intentionally ignored and must not be committed. are intentionally ignored and must not be committed.
### Phase B fip breadth IC (2026-07-18/19) — compact evidence
Canonical artifacts:
- `fip-reconcile-20260719-000520.json` — single-sourced authoritative ICs
(unconditional liquid fip, tiers, prod-subset, mom-conditional, context
signals). Membership symbol dumps stripped after the decision; narrative in
[`docs/research/fip-breadth-ic.md`](../docs/research/fip-breadth-ic.md).
- `fip-breadth-20260718-211440-fingerprint.json` — prod-snapshot fingerprint
pass (fip IC 0.045 / t 2.91).
Removed as superseded / dangerous intermediate noise (recoverable from Git):
- `fip-breadth-20260718-211440-breadth.json` (+ wrapper) — **orphaned** +0.0575
/ t +5.12 from racing a partial `research.sqlite`. Kept out of the tree so it
cannot be re-mythologized.
- `fip-breadth-20260718-194828*.json` — fingerprint-only partial run.
- `fip-breadth-diagnostics-20260718-213705.json` and `…-213908.json` — dual-path
diagnostics superseded by the single-sourced reconcile.
@@ -1,578 +0,0 @@
{
"generated_at": "2026-07-18T18:21:28.359793+00:00",
"tickers": 506,
"rank_only_tickers": 0,
"candidates": 202765,
"qualified": 1086,
"params": {
"step_days": 5,
"step_sessions": 5,
"entry_cadence": "weekly",
"signal_eval_cadence": "weekly",
"horizon_days": 30,
"min_lookback": 60,
"cost_per_side_pct": 0.1,
"target_model": "production_gtl",
"target_model_label": "Live GTL (production)",
"is_production_target_model": true,
"production_reentry_policy": "gate_reset",
"liquid_breadth_top_n": null,
"liquid_min_price": null,
"signal_eval_only": true
},
"activation": {
"min_momentum_percentile": 80.0,
"min_rr": 2.0,
"min_confidence": 0.0,
"require_high_conviction": false,
"exclude_conflicts": false,
"exclude_neutral": true
},
"overall_qualified": {
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
"overall_all": {
"total": 202765,
"wins": 82220,
"losses": 113809,
"expired": 6736,
"hit_rate": 41.9,
"avg_r": -0.04,
"total_r": -8186.18,
"net_avg_r": -0.095,
"net_total_r": -19184.55,
"best_r": 9.24,
"worst_r": -16.42,
"avg_hold_days": 8.2,
"net_r_per_day": -0.0115,
"median_net_r": -1.035,
"profit_factor": 0.85,
"net_avg_r_ex_top5": -0.22
},
"by_direction": {
"long": {
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
"short": {
"total": 0,
"wins": 0,
"losses": 0,
"expired": 0,
"hit_rate": null,
"avg_r": null,
"total_r": null,
"net_avg_r": null,
"net_total_r": null,
"best_r": null,
"worst_r": null,
"avg_hold_days": null,
"net_r_per_day": null,
"median_net_r": null,
"profit_factor": null,
"net_avg_r_ex_top5": null
}
},
"min_momentum_percentile": 80.0,
"sweep": [
{
"min_momentum_percentile": 90.0,
"total": 497,
"wins": 177,
"losses": 269,
"expired": 51,
"hit_rate": 39.7,
"avg_r": 0.276,
"total_r": 137.05,
"net_avg_r": 0.235,
"net_total_r": 116.55,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 11.8,
"net_r_per_day": 0.0199,
"median_net_r": -1.026,
"profit_factor": 1.39,
"net_avg_r_ex_top5": 0.071
},
{
"min_momentum_percentile": 80.0,
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
{
"min_momentum_percentile": 70.0,
"total": 1841,
"wins": 597,
"losses": 1062,
"expired": 182,
"hit_rate": 36.0,
"avg_r": 0.152,
"total_r": 280.26,
"net_avg_r": 0.104,
"net_total_r": 190.75,
"best_r": 8.85,
"worst_r": -4.21,
"avg_hold_days": 11.8,
"net_r_per_day": 0.0088,
"median_net_r": -1.037,
"profit_factor": 1.16,
"net_avg_r_ex_top5": -0.055
},
{
"min_momentum_percentile": 60.0,
"total": 2772,
"wins": 873,
"losses": 1611,
"expired": 288,
"hit_rate": 35.1,
"avg_r": 0.126,
"total_r": 348.07,
"net_avg_r": 0.075,
"net_total_r": 209.25,
"best_r": 8.85,
"worst_r": -4.21,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0063,
"median_net_r": -1.04,
"profit_factor": 1.12,
"net_avg_r_ex_top5": -0.077
},
{
"min_momentum_percentile": 50.0,
"total": 3901,
"wins": 1182,
"losses": 2295,
"expired": 424,
"hit_rate": 34.0,
"avg_r": 0.089,
"total_r": 345.37,
"net_avg_r": 0.038,
"net_total_r": 146.96,
"best_r": 8.85,
"worst_r": -4.86,
"avg_hold_days": 12.1,
"net_r_per_day": 0.0031,
"median_net_r": -1.042,
"profit_factor": 1.06,
"net_avg_r_ex_top5": -0.114
},
{
"min_momentum_percentile": 0.0,
"total": 14588,
"wins": 3719,
"losses": 9271,
"expired": 1598,
"hit_rate": 28.6,
"avg_r": -0.065,
"total_r": -952.08,
"net_avg_r": -0.115,
"net_total_r": -1676.86,
"best_r": 9.24,
"worst_r": -15.94,
"avg_hold_days": 12.1,
"net_r_per_day": -0.0095,
"median_net_r": -1.043,
"profit_factor": 0.84,
"net_avg_r_ex_top5": -0.275
}
],
"gate_ablation": [
{
"variant": "all_floors",
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049,
"hold_days": 30,
"hold_avg_r": 0.631,
"hold_net_avg_r": 0.585,
"hold_total_r": 684.97
},
{
"variant": "no_confidence_floor",
"total": 1093,
"wins": 380,
"losses": 596,
"expired": 117,
"hit_rate": 38.9,
"avg_r": 0.25,
"total_r": 273.55,
"net_avg_r": 0.204,
"net_total_r": 222.99,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 11.9,
"net_r_per_day": 0.0171,
"median_net_r": -1.031,
"profit_factor": 1.33,
"net_avg_r_ex_top5": 0.045,
"hold_days": 30,
"hold_avg_r": 0.626,
"hold_net_avg_r": 0.58,
"hold_total_r": 684.08
},
{
"variant": "no_rr_floor",
"total": 6849,
"wins": 3235,
"losses": 3409,
"expired": 205,
"hit_rate": 48.7,
"avg_r": 0.112,
"total_r": 770.17,
"net_avg_r": 0.061,
"net_total_r": 418.96,
"best_r": 8.85,
"worst_r": -5.16,
"avg_hold_days": 7.9,
"net_r_per_day": 0.0078,
"median_net_r": -0.061,
"profit_factor": 1.11,
"net_avg_r_ex_top5": -0.061,
"hold_days": 30,
"hold_avg_r": 0.354,
"hold_net_avg_r": 0.303,
"hold_total_r": 2425.86
},
{
"variant": "no_neutral_exclusion",
"total": 2313,
"wins": 770,
"losses": 1279,
"expired": 264,
"hit_rate": 37.6,
"avg_r": 0.2,
"total_r": 462.35,
"net_avg_r": 0.154,
"net_total_r": 355.89,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.6,
"net_r_per_day": 0.0122,
"median_net_r": -1.031,
"profit_factor": 1.25,
"net_avg_r_ex_top5": 0.004,
"hold_days": 30,
"hold_avg_r": 0.583,
"hold_net_avg_r": 0.537,
"hold_total_r": 1348.89
},
{
"variant": "momentum_only",
"total": 14696,
"wins": 6827,
"losses": 7359,
"expired": 510,
"hit_rate": 48.1,
"avg_r": 0.114,
"total_r": 1669.64,
"net_avg_r": 0.064,
"net_total_r": 936.93,
"best_r": 8.85,
"worst_r": -5.71,
"avg_hold_days": 8.4,
"net_r_per_day": 0.0076,
"median_net_r": -1.013,
"profit_factor": 1.11,
"net_avg_r_ex_top5": -0.055,
"hold_days": 30,
"hold_avg_r": 0.395,
"hold_net_avg_r": 0.345,
"hold_total_r": 5798.82
}
],
"gate_ablation_note": "Each row re-qualifies the same candidates at the current momentum cutoff (80) with one floor removed (long-only while the momentum gate is active). If dropping a floor doesn't hurt net expectancy, that floor isn't pulling its weight. The Hold columns grade the same variants under the hold-to-horizon time exit instead of the S/R target \u2014 the view that matters if the exit policy moves to a fixed hold.",
"time_exit_sweep": [
{
"hold_days": 5,
"total": 1086,
"wins": 603,
"win_rate": 55.5,
"avg_r": 0.175,
"total_r": 190.16,
"net_avg_r": 0.129,
"net_total_r": 139.97,
"best_r": 5.09,
"worst_r": -2.51,
"avg_hold_days": 4.5,
"net_r_per_day": 0.0285,
"median_net_r": 0.115,
"profit_factor": 1.36,
"net_avg_r_ex_top5": -0.002
},
{
"hold_days": 10,
"total": 1086,
"wins": 559,
"win_rate": 51.5,
"avg_r": 0.357,
"total_r": 387.9,
"net_avg_r": 0.311,
"net_total_r": 337.7,
"best_r": 6.73,
"worst_r": -2.51,
"avg_hold_days": 7.9,
"net_r_per_day": 0.0395,
"median_net_r": 0.031,
"profit_factor": 1.67,
"net_avg_r_ex_top5": 0.112
},
{
"hold_days": 21,
"total": 1086,
"wins": 487,
"win_rate": 44.8,
"avg_r": 0.525,
"total_r": 570.33,
"net_avg_r": 0.479,
"net_total_r": 520.14,
"best_r": 9.86,
"worst_r": -3.38,
"avg_hold_days": 13.7,
"net_r_per_day": 0.0349,
"median_net_r": -1.027,
"profit_factor": 1.81,
"net_avg_r_ex_top5": 0.191
},
{
"hold_days": 30,
"total": 1086,
"wins": 434,
"win_rate": 40.0,
"avg_r": 0.631,
"total_r": 684.97,
"net_avg_r": 0.585,
"net_total_r": 634.78,
"best_r": 12.87,
"worst_r": -3.38,
"avg_hold_days": 17.8,
"net_r_per_day": 0.0329,
"median_net_r": -1.033,
"profit_factor": 1.9,
"net_avg_r_ex_top5": 0.212
}
],
"portfolio_sim": {
"params": {
"starting_capital": 10000.0,
"max_positions": 10,
"risk_per_trade_pct": 1.0,
"notional_cap_pct": 20.0,
"cost_per_side_pct": 0.1,
"hold_days": 30
},
"policies": [],
"note": "One capital-constrained book over the same qualified setups the tables above grade per-setup: at most 10 concurrent positions (one per ticker), best momentum first, fixed-fractional risk sizing with a no-leverage cap, entries at the detection close, stops filled at the worse of stop or open. 'target' races the S/R target against the stop (timeout at the horizon); 'hold' keeps the initial stop and exits at the horizon close. SPY return is price-only over the same window. In-sample; no dividends."
},
"strategy_variants": {
"variants": [],
"note": "Research-only hold-to-horizon portfolio variants. Production now uses residual 12-1 momentum at cutoff 80; the remaining rows compare the legacy raw rank, raw cutoff 90, one max-15 capacity check, and volatility overlays."
},
"exit_policy_variants": {
"variants": [],
"note": "Research-only exit policies over the residual/high-vol 80/20 entry candidate. Every row uses the same entry qualification/ranking and changes only the exit discipline."
},
"portfolio_monitor": null,
"production_cadence_comparison": null,
"holdout": null,
"min_rr_sweep": null,
"target_model_diagnostics": {
"target_model": "production_gtl",
"target_model_label": "Live GTL (production)",
"candidate_count": 202765,
"primary_source_counts": {
"pivot_point": 196290,
"range_grid": 180036
},
"primary_round_only": 0,
"primary_strength_100": 138596,
"avg_primary_strength": 80.109,
"avg_primary_distance_atr": 2.293,
"avg_primary_rejection_count": 41.908,
"avg_raw_level_count": 53.204,
"avg_gate_level_count": 53.204
},
"signal_eval": [
{
"signal": "vol_6m",
"weeks": 39,
"avg_cross_section": 498.2,
"mean_ic": 0.0609,
"ic_t_stat": 1.48,
"ic_positive_pct": 64.1,
"mean_quintile_spread": 0.0337,
"reliable": true
},
{
"signal": "mom_12_1_resid",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": 0.0552,
"ic_t_stat": 1.98,
"ic_positive_pct": 60.0,
"mean_quintile_spread": 0.0207,
"reliable": true
},
{
"signal": "mom_12_1",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": 0.0531,
"ic_t_stat": 1.61,
"ic_positive_pct": 65.7,
"mean_quintile_spread": 0.0206,
"reliable": true
},
{
"signal": "trend_200",
"weeks": 37,
"avg_cross_section": 497.9,
"mean_ic": 0.0161,
"ic_t_stat": 0.44,
"ic_positive_pct": 59.5,
"mean_quintile_spread": 0.006,
"reliable": true
},
{
"signal": "reversal_1m",
"weeks": 43,
"avg_cross_section": 498.7,
"mean_ic": 0.0059,
"ic_t_stat": 0.22,
"ic_positive_pct": 53.5,
"mean_quintile_spread": 0.0053,
"reliable": true
},
{
"signal": "mom_6_1",
"weeks": 39,
"avg_cross_section": 498.2,
"mean_ic": 0.0051,
"ic_t_stat": 0.21,
"ic_positive_pct": 56.4,
"mean_quintile_spread": 0.0087,
"reliable": true
},
{
"signal": "mom_3_1",
"weeks": 42,
"avg_cross_section": 498.5,
"mean_ic": -0.0064,
"ic_t_stat": -0.25,
"ic_positive_pct": 50.0,
"mean_quintile_spread": 0.0046,
"reliable": true
},
{
"signal": "high_52w",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": -0.0086,
"ic_t_stat": -0.26,
"ic_positive_pct": 54.3,
"mean_quintile_spread": -0.0088,
"reliable": true
},
{
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": -0.045,
"ic_t_stat": -2.91,
"ic_positive_pct": 25.7,
"mean_quintile_spread": -0.0168,
"reliable": true
}
],
"signal_eval_note": "Cross-sectional rank-IC of price-only signals vs the forward 30-day return (min 20 names/window). |IC| \u2273 0.03 with a consistent sign is a real (if small) edge; near 0 means ranking on it sorts nothing. Momentum factors and high_52w are expected positive; reversal_1m and vol_6m expected negative (mean-reversion / low-vol anomaly). IC is measured on non-overlapping windows; signals with fewer than 12 independent windows are flagged unreliable (too few regimes \u2014 deepen history with the Data Backfill job).",
"note": "Sentiment & fundamentals held neutral (no point-in-time history). Stops fill at the worse of the stop or the bar's open (gaps through the stop are modeled, so a loss can exceed \u22121R); targets never fill better than their level. ~6 months \u2248 one market regime \u2014 treat as directional, not gospel.",
"recommendation": {
"headline": "Trade the qualified list long-only; hold 30 trading days with the initial ATR stop.",
"items": [
{
"topic": "exit",
"text": "Legacy exit diagnostic: hold 30 trading days with the initial stop (+0.58R net/trade vs +0.21R for the S/R target exit)."
},
{
"topic": "gate",
"text": "Gate: the confidence floor adds nothing \u2014 dropping it costs +0.01R/trade and adds 7 trades."
},
{
"topic": "gate",
"text": "Gate: keep the R:R floor (worth +0.28R/trade under the hold exit)."
},
{
"topic": "gate",
"text": "Gate: keep the NEUTRAL exclusion (worth +0.05R/trade under the hold exit)."
},
{
"topic": "cutoff",
"text": "Residual-momentum cutoff: 90 has the best per-trade net (+0.23R over 497 setups)."
},
{
"topic": "robustness",
"text": "Robustness: expectancy survives removing the top 5% of winners (+0.21R net/trade under the recommended 30d hold) \u2014 the edge is not a handful of outliers."
}
],
"note": "Derived from this report's numbers on every run \u2014 the advice flips if the data does."
},
"research_recommendation": {
"items": [],
"note": "Strategy variants unavailable; re-run the backtest after benchmark data is present."
}
}
-21
View File
@@ -1,21 +0,0 @@
{
"generated_at": "2026-07-18T19:48:28.127710",
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0,
"fingerprint": {
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": -0.045,
"ic_t_stat": -2.91,
"ic_positive_pct": 25.7,
"mean_quintile_spread": -0.0168,
"reliable": true,
"pass": true,
"expected_ic": -0.045,
"expected_t": -2.9
},
"breadth": null,
"verdict": null,
"fingerprint_report_path": "reports\\fip-breadth-20260718-194828-fingerprint.json"
}
@@ -1,596 +0,0 @@
{
"generated_at": "2026-07-18T19:23:17.726528+00:00",
"tickers": 4650,
"rank_only_tickers": 4144,
"candidates": 202769,
"qualified": 1086,
"params": {
"step_days": 5,
"step_sessions": 5,
"entry_cadence": "weekly",
"signal_eval_cadence": "weekly",
"horizon_days": 30,
"min_lookback": 60,
"cost_per_side_pct": 0.1,
"target_model": "production_gtl",
"target_model_label": "Live GTL (production)",
"is_production_target_model": true,
"production_reentry_policy": "gate_reset",
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0,
"signal_eval_only": true
},
"activation": {
"min_momentum_percentile": 80.0,
"min_rr": 2.0,
"min_confidence": 0.0,
"require_high_conviction": false,
"exclude_conflicts": false,
"exclude_neutral": true
},
"overall_qualified": {
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
"overall_all": {
"total": 202769,
"wins": 82221,
"losses": 113812,
"expired": 6736,
"hit_rate": 41.9,
"avg_r": -0.04,
"total_r": -8188.17,
"net_avg_r": -0.095,
"net_total_r": -19186.68,
"best_r": 9.24,
"worst_r": -16.42,
"avg_hold_days": 8.2,
"net_r_per_day": -0.0115,
"median_net_r": -1.035,
"profit_factor": 0.85,
"net_avg_r_ex_top5": -0.22
},
"by_direction": {
"long": {
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
"short": {
"total": 0,
"wins": 0,
"losses": 0,
"expired": 0,
"hit_rate": null,
"avg_r": null,
"total_r": null,
"net_avg_r": null,
"net_total_r": null,
"best_r": null,
"worst_r": null,
"avg_hold_days": null,
"net_r_per_day": null,
"median_net_r": null,
"profit_factor": null,
"net_avg_r_ex_top5": null
}
},
"min_momentum_percentile": 80.0,
"sweep": [
{
"min_momentum_percentile": 90.0,
"total": 497,
"wins": 177,
"losses": 269,
"expired": 51,
"hit_rate": 39.7,
"avg_r": 0.276,
"total_r": 137.05,
"net_avg_r": 0.235,
"net_total_r": 116.55,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 11.8,
"net_r_per_day": 0.0199,
"median_net_r": -1.026,
"profit_factor": 1.39,
"net_avg_r_ex_top5": 0.071
},
{
"min_momentum_percentile": 80.0,
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049
},
{
"min_momentum_percentile": 70.0,
"total": 1841,
"wins": 597,
"losses": 1062,
"expired": 182,
"hit_rate": 36.0,
"avg_r": 0.152,
"total_r": 280.26,
"net_avg_r": 0.104,
"net_total_r": 190.75,
"best_r": 8.85,
"worst_r": -4.21,
"avg_hold_days": 11.8,
"net_r_per_day": 0.0088,
"median_net_r": -1.037,
"profit_factor": 1.16,
"net_avg_r_ex_top5": -0.055
},
{
"min_momentum_percentile": 60.0,
"total": 2772,
"wins": 873,
"losses": 1611,
"expired": 288,
"hit_rate": 35.1,
"avg_r": 0.126,
"total_r": 348.07,
"net_avg_r": 0.075,
"net_total_r": 209.25,
"best_r": 8.85,
"worst_r": -4.21,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0063,
"median_net_r": -1.04,
"profit_factor": 1.12,
"net_avg_r_ex_top5": -0.077
},
{
"min_momentum_percentile": 50.0,
"total": 3901,
"wins": 1182,
"losses": 2295,
"expired": 424,
"hit_rate": 34.0,
"avg_r": 0.089,
"total_r": 345.37,
"net_avg_r": 0.038,
"net_total_r": 146.96,
"best_r": 8.85,
"worst_r": -4.86,
"avg_hold_days": 12.1,
"net_r_per_day": 0.0031,
"median_net_r": -1.042,
"profit_factor": 1.06,
"net_avg_r_ex_top5": -0.114
},
{
"min_momentum_percentile": 0.0,
"total": 14589,
"wins": 3719,
"losses": 9272,
"expired": 1598,
"hit_rate": 28.6,
"avg_r": -0.065,
"total_r": -953.08,
"net_avg_r": -0.115,
"net_total_r": -1677.9,
"best_r": 9.24,
"worst_r": -15.94,
"avg_hold_days": 12.1,
"net_r_per_day": -0.0095,
"median_net_r": -1.043,
"profit_factor": 0.84,
"net_avg_r_ex_top5": -0.275
}
],
"gate_ablation": [
{
"variant": "all_floors",
"total": 1086,
"wins": 379,
"losses": 591,
"expired": 116,
"hit_rate": 39.1,
"avg_r": 0.255,
"total_r": 276.76,
"net_avg_r": 0.209,
"net_total_r": 226.56,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.0,
"net_r_per_day": 0.0174,
"median_net_r": -1.031,
"profit_factor": 1.34,
"net_avg_r_ex_top5": 0.049,
"hold_days": 30,
"hold_avg_r": 0.631,
"hold_net_avg_r": 0.585,
"hold_total_r": 684.97
},
{
"variant": "no_confidence_floor",
"total": 1093,
"wins": 380,
"losses": 596,
"expired": 117,
"hit_rate": 38.9,
"avg_r": 0.25,
"total_r": 273.55,
"net_avg_r": 0.204,
"net_total_r": 222.99,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 11.9,
"net_r_per_day": 0.0171,
"median_net_r": -1.031,
"profit_factor": 1.33,
"net_avg_r_ex_top5": 0.045,
"hold_days": 30,
"hold_avg_r": 0.626,
"hold_net_avg_r": 0.58,
"hold_total_r": 684.08
},
{
"variant": "no_rr_floor",
"total": 6849,
"wins": 3235,
"losses": 3409,
"expired": 205,
"hit_rate": 48.7,
"avg_r": 0.112,
"total_r": 770.17,
"net_avg_r": 0.061,
"net_total_r": 418.96,
"best_r": 8.85,
"worst_r": -5.16,
"avg_hold_days": 7.9,
"net_r_per_day": 0.0078,
"median_net_r": -0.061,
"profit_factor": 1.11,
"net_avg_r_ex_top5": -0.061,
"hold_days": 30,
"hold_avg_r": 0.354,
"hold_net_avg_r": 0.303,
"hold_total_r": 2425.86
},
{
"variant": "no_neutral_exclusion",
"total": 2313,
"wins": 770,
"losses": 1279,
"expired": 264,
"hit_rate": 37.6,
"avg_r": 0.2,
"total_r": 462.35,
"net_avg_r": 0.154,
"net_total_r": 355.89,
"best_r": 8.85,
"worst_r": -3.38,
"avg_hold_days": 12.6,
"net_r_per_day": 0.0122,
"median_net_r": -1.031,
"profit_factor": 1.25,
"net_avg_r_ex_top5": 0.004,
"hold_days": 30,
"hold_avg_r": 0.583,
"hold_net_avg_r": 0.537,
"hold_total_r": 1348.89
},
{
"variant": "momentum_only",
"total": 14696,
"wins": 6827,
"losses": 7359,
"expired": 510,
"hit_rate": 48.1,
"avg_r": 0.114,
"total_r": 1669.64,
"net_avg_r": 0.064,
"net_total_r": 936.93,
"best_r": 8.85,
"worst_r": -5.71,
"avg_hold_days": 8.4,
"net_r_per_day": 0.0076,
"median_net_r": -1.013,
"profit_factor": 1.11,
"net_avg_r_ex_top5": -0.055,
"hold_days": 30,
"hold_avg_r": 0.395,
"hold_net_avg_r": 0.345,
"hold_total_r": 5798.82
}
],
"gate_ablation_note": "Each row re-qualifies the same candidates at the current momentum cutoff (80) with one floor removed (long-only while the momentum gate is active). If dropping a floor doesn't hurt net expectancy, that floor isn't pulling its weight. The Hold columns grade the same variants under the hold-to-horizon time exit instead of the S/R target \u2014 the view that matters if the exit policy moves to a fixed hold.",
"time_exit_sweep": [
{
"hold_days": 5,
"total": 1086,
"wins": 603,
"win_rate": 55.5,
"avg_r": 0.175,
"total_r": 190.16,
"net_avg_r": 0.129,
"net_total_r": 139.97,
"best_r": 5.09,
"worst_r": -2.51,
"avg_hold_days": 4.5,
"net_r_per_day": 0.0285,
"median_net_r": 0.115,
"profit_factor": 1.36,
"net_avg_r_ex_top5": -0.002
},
{
"hold_days": 10,
"total": 1086,
"wins": 559,
"win_rate": 51.5,
"avg_r": 0.357,
"total_r": 387.9,
"net_avg_r": 0.311,
"net_total_r": 337.7,
"best_r": 6.73,
"worst_r": -2.51,
"avg_hold_days": 7.9,
"net_r_per_day": 0.0395,
"median_net_r": 0.031,
"profit_factor": 1.67,
"net_avg_r_ex_top5": 0.112
},
{
"hold_days": 21,
"total": 1086,
"wins": 487,
"win_rate": 44.8,
"avg_r": 0.525,
"total_r": 570.33,
"net_avg_r": 0.479,
"net_total_r": 520.14,
"best_r": 9.86,
"worst_r": -3.38,
"avg_hold_days": 13.7,
"net_r_per_day": 0.0349,
"median_net_r": -1.027,
"profit_factor": 1.81,
"net_avg_r_ex_top5": 0.191
},
{
"hold_days": 30,
"total": 1086,
"wins": 434,
"win_rate": 40.0,
"avg_r": 0.631,
"total_r": 684.97,
"net_avg_r": 0.585,
"net_total_r": 634.78,
"best_r": 12.87,
"worst_r": -3.38,
"avg_hold_days": 17.8,
"net_r_per_day": 0.0329,
"median_net_r": -1.033,
"profit_factor": 1.9,
"net_avg_r_ex_top5": 0.212
}
],
"portfolio_sim": {
"params": {
"starting_capital": 10000.0,
"max_positions": 10,
"risk_per_trade_pct": 1.0,
"notional_cap_pct": 20.0,
"cost_per_side_pct": 0.1,
"hold_days": 30
},
"policies": [],
"note": "One capital-constrained book over the same qualified setups the tables above grade per-setup: at most 10 concurrent positions (one per ticker), best momentum first, fixed-fractional risk sizing with a no-leverage cap, entries at the detection close, stops filled at the worse of stop or open. 'target' races the S/R target against the stop (timeout at the horizon); 'hold' keeps the initial stop and exits at the horizon close. SPY return is price-only over the same window. In-sample; no dividends."
},
"strategy_variants": {
"variants": [],
"note": "Research-only hold-to-horizon portfolio variants. Production now uses residual 12-1 momentum at cutoff 80; the remaining rows compare the legacy raw rank, raw cutoff 90, one max-15 capacity check, and volatility overlays."
},
"exit_policy_variants": {
"variants": [],
"note": "Research-only exit policies over the residual/high-vol 80/20 entry candidate. Every row uses the same entry qualification/ranking and changes only the exit discipline."
},
"portfolio_monitor": null,
"production_cadence_comparison": null,
"holdout": null,
"min_rr_sweep": null,
"target_model_diagnostics": {
"target_model": "production_gtl",
"target_model_label": "Live GTL (production)",
"candidate_count": 202769,
"primary_source_counts": {
"pivot_point": 196294,
"range_grid": 180039
},
"primary_round_only": 0,
"primary_strength_100": 138599,
"avg_primary_strength": 80.109,
"avg_primary_distance_atr": 2.293,
"avg_primary_rejection_count": 41.907,
"avg_raw_level_count": 53.204,
"avg_gate_level_count": 53.204
},
"signal_eval": [
{
"signal": "high_52w",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.1283,
"ic_t_stat": 4.28,
"ic_positive_pct": 85.7,
"mean_quintile_spread": -0.1009,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "mom_12_1",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.0997,
"ic_t_stat": 4.56,
"ic_positive_pct": 88.6,
"mean_quintile_spread": -0.1001,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "mom_6_1",
"weeks": 40,
"avg_cross_section": 1474.8,
"mean_ic": 0.0681,
"ic_t_stat": 3.45,
"ic_positive_pct": 77.5,
"mean_quintile_spread": -0.0322,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.0575,
"ic_t_stat": 5.12,
"ic_positive_pct": 88.6,
"mean_quintile_spread": 0.0199,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "trend_200",
"weeks": 37,
"avg_cross_section": 1472.8,
"mean_ic": 0.0538,
"ic_t_stat": 2.33,
"ic_positive_pct": 75.7,
"mean_quintile_spread": -0.0675,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "mom_3_1",
"weeks": 42,
"avg_cross_section": 1476.0,
"mean_ic": 0.0523,
"ic_t_stat": 3.27,
"ic_positive_pct": 73.8,
"mean_quintile_spread": -0.0194,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "mom_12_1_resid",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.0388,
"ic_t_stat": 2.28,
"ic_positive_pct": 74.3,
"mean_quintile_spread": -0.0542,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "reversal_1m",
"weeks": 43,
"avg_cross_section": 1476.6,
"mean_ic": 0.0155,
"ic_t_stat": 0.85,
"ic_positive_pct": 48.8,
"mean_quintile_spread": -0.0862,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
{
"signal": "vol_6m",
"weeks": 40,
"avg_cross_section": 1474.8,
"mean_ic": -0.1584,
"ic_t_stat": -6.05,
"ic_positive_pct": 12.5,
"mean_quintile_spread": 0.0164,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
}
],
"signal_eval_note": "Cross-sectional rank-IC of price-only signals vs the forward 30-day return (min 20 names/window). |IC| \u2273 0.03 with a consistent sign is a real (if small) edge; near 0 means ranking on it sorts nothing. Momentum factors and high_52w are expected positive; reversal_1m and vol_6m expected negative (mean-reversion / low-vol anomaly). IC is measured on non-overlapping windows; signals with fewer than 12 independent windows are flagged unreliable (too few regimes \u2014 deepen history with the Data Backfill job).",
"note": "Sentiment & fundamentals held neutral (no point-in-time history). Stops fill at the worse of the stop or the bar's open (gaps through the stop are modeled, so a loss can exceed \u22121R); targets never fill better than their level. ~6 months \u2248 one market regime \u2014 treat as directional, not gospel.",
"recommendation": {
"headline": "Trade the qualified list long-only; hold 30 trading days with the initial ATR stop.",
"items": [
{
"topic": "exit",
"text": "Legacy exit diagnostic: hold 30 trading days with the initial stop (+0.58R net/trade vs +0.21R for the S/R target exit)."
},
{
"topic": "gate",
"text": "Gate: the confidence floor adds nothing \u2014 dropping it costs +0.01R/trade and adds 7 trades."
},
{
"topic": "gate",
"text": "Gate: keep the R:R floor (worth +0.28R/trade under the hold exit)."
},
{
"topic": "gate",
"text": "Gate: keep the NEUTRAL exclusion (worth +0.05R/trade under the hold exit)."
},
{
"topic": "cutoff",
"text": "Residual-momentum cutoff: 90 has the best per-trade net (+0.23R over 497 setups)."
},
{
"topic": "robustness",
"text": "Robustness: expectancy survives removing the top 5% of winners (+0.21R net/trade under the recommended 30d hold) \u2014 the edge is not a handful of outliers."
}
],
"note": "Derived from this report's numbers on every run \u2014 the advice flips if the data does."
},
"research_recommendation": {
"items": [],
"note": "Strategy variants unavailable; re-run the backtest after benchmark data is present."
}
}
-59
View File
@@ -1,59 +0,0 @@
{
"generated_at": "2026-07-18T21:14:40.170961",
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0,
"fingerprint": {
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 497.7,
"mean_ic": -0.045,
"ic_t_stat": -2.91,
"ic_positive_pct": 25.7,
"mean_quintile_spread": -0.0168,
"reliable": true,
"pass": true,
"expected_ic": -0.045,
"expected_t": -2.9
},
"breadth": {
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.0575,
"ic_t_stat": 5.12,
"ic_positive_pct": 88.6,
"mean_quintile_spread": 0.0199,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
},
"verdict": {
"green": false,
"reason": "iron rule not met on liquid-breadth cross-section",
"checks": {
"mean_ic": 0.0575,
"abs_mean_ic_ge_0_03": true,
"sign_negative": false,
"ic_t_stat": 5.12,
"reliable": true,
"weeks": 35,
"avg_cross_section": 1471.2
},
"row": {
"signal": "fip_id",
"weeks": 35,
"avg_cross_section": 1471.2,
"mean_ic": 0.0575,
"ic_t_stat": 5.12,
"ic_positive_pct": 88.6,
"mean_quintile_spread": 0.0199,
"reliable": true,
"liquid_breadth_top_n": 1500,
"liquid_min_price": 5.0
}
},
"fingerprint_report_path": "reports/fip-breadth-20260718-211440-fingerprint.json",
"breadth_report_path": "reports/fip-breadth-20260718-211440-breadth.json",
"breadth_tickers": 4650,
"breadth_rank_only_tickers": 4144
}
@@ -1,100 +0,0 @@
{
"generated_at": "2026-07-18T21:37:04.615484",
"research_snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite",
"prod_subset_n": 506,
"panel_tickers": 4403,
"top_n": 1500,
"min_price": 5.0,
"checks": {
"fip_same_week_liquid_1500": {
"note": "Replication of main breadth run (same-week $vol mask)",
"mean_ic": -0.0168,
"ic_t_stat": -1.85,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 40.0,
"reliable": true
},
"fip_lagged_membership_1w": {
"note": "Liquid top-N ranked on *prior* week's median $vol \u2014 excludes same-week liquidity explosion leak",
"mean_ic": -0.0102,
"ic_t_stat": -0.93,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 40.0,
"reliable": true
},
"fip_tier_1_800": {
"note": "Same-week liquid ranks 1\u2013800 (senior liquid tier)",
"mean_ic": -0.035,
"ic_t_stat": -2.99,
"weeks": 35,
"avg_cross_section": 791.2,
"ic_positive_pct": 25.7,
"reliable": true
},
"fip_tier_801_1500": {
"note": "Same-week liquid ranks 801\u20131500 (junior liquid tier)",
"mean_ic": 0.0141,
"ic_t_stat": 1.25,
"weeks": 35,
"avg_cross_section": 700.0,
"ic_positive_pct": 60.0,
"reliable": true
},
"fip_prod_universe_subset": {
"note": "Symbols in prod.sqlite (~S&P-like large-cap book) inside same-week liquid top-N \u2014 compositional control",
"mean_ic": -0.0444,
"ic_t_stat": -2.88,
"weeks": 35,
"avg_cross_section": 497.5,
"ic_positive_pct": 25.7,
"reliable": true
},
"fip_momentum_conditional_top20pct": {
"note": "Among liquid top-N, keep mom_12_1 percentile \u2265 80.0 (paper: ID modulates continuation among winners; gate-relevant)",
"mean_ic": -0.0879,
"ic_t_stat": -4.58,
"weeks": 35,
"avg_cross_section": 294.3,
"ic_positive_pct": 22.9,
"reliable": true
},
"vol_6m_liquid_1500": {
"note": "Context: low-vol anomaly strength on this pool",
"mean_ic": -0.0465,
"ic_t_stat": -1.3,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 37.1,
"reliable": true
},
"mom_12_1_liquid_1500": {
"note": "Context: raw momentum on liquid breadth",
"mean_ic": 0.0462,
"ic_t_stat": 1.91,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 65.7,
"reliable": true
},
"mom_12_1_resid_liquid_1500": {
"note": "Context: residual momentum on liquid breadth",
"mean_ic": 0.0289,
"ic_t_stat": 1.33,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 60.0,
"reliable": true
}
},
"interpretation": {
"leak_ruled_out": false,
"junior_tier_drives_positive": true,
"prod_subset_still_negative": true,
"mom_conditional_negative_and_reliable": true,
"compositional_flip_story": "If prod subset IC is negative while full liquid-1500 is positive, the sign flip is compositional (bleeders / Nasdaq junk), not a temporal regime change. Unconditional fip pools continuous winners (want neg IC) against continuous losers/bleeders (want pos IC).",
"vol_tilt_warning": "vol_6m large negative IC on breadth: high-vol lottery names underperform. Production 80/20 high-vol tilt was validated on S&P-like names; must re-validate before any universe broaden."
},
"platform_verdict": "ALIVE as breadth-book tilt candidate among momentum winners only \u2014 still needs a book-level experiment; not a production wire-in."
}
@@ -1,100 +0,0 @@
{
"generated_at": "2026-07-18T21:39:07.916038",
"research_snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite",
"prod_subset_n": 506,
"panel_tickers": 4403,
"top_n": 1500,
"min_price": 5.0,
"checks": {
"fip_same_week_liquid_1500": {
"note": "Replication of main breadth run (same-week $vol mask)",
"mean_ic": -0.0168,
"ic_t_stat": -1.85,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 40.0,
"reliable": true
},
"fip_lagged_membership_1w": {
"note": "Liquid top-N ranked on *prior* week's median $vol \u2014 excludes same-week liquidity explosion leak",
"mean_ic": -0.0102,
"ic_t_stat": -0.93,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 40.0,
"reliable": true
},
"fip_tier_1_800": {
"note": "Same-week liquid ranks 1\u2013800 (senior liquid tier)",
"mean_ic": -0.035,
"ic_t_stat": -2.99,
"weeks": 35,
"avg_cross_section": 791.2,
"ic_positive_pct": 25.7,
"reliable": true
},
"fip_tier_801_1500": {
"note": "Same-week liquid ranks 801\u20131500 (junior liquid tier)",
"mean_ic": 0.0141,
"ic_t_stat": 1.25,
"weeks": 35,
"avg_cross_section": 700.0,
"ic_positive_pct": 60.0,
"reliable": true
},
"fip_prod_universe_subset": {
"note": "Symbols in prod.sqlite (~S&P-like large-cap book) inside same-week liquid top-N \u2014 compositional control",
"mean_ic": -0.0444,
"ic_t_stat": -2.88,
"weeks": 35,
"avg_cross_section": 497.5,
"ic_positive_pct": 25.7,
"reliable": true
},
"fip_momentum_conditional_top20pct": {
"note": "Among liquid top-N, keep mom_12_1 percentile \u2265 80.0 (paper: ID modulates continuation among winners; gate-relevant)",
"mean_ic": -0.0879,
"ic_t_stat": -4.58,
"weeks": 35,
"avg_cross_section": 294.3,
"ic_positive_pct": 22.9,
"reliable": true
},
"vol_6m_liquid_1500": {
"note": "Context: low-vol anomaly strength on this pool",
"mean_ic": -0.0465,
"ic_t_stat": -1.3,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 37.1,
"reliable": true
},
"mom_12_1_liquid_1500": {
"note": "Context: raw momentum on liquid breadth",
"mean_ic": 0.0462,
"ic_t_stat": 1.91,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 65.7,
"reliable": true
},
"mom_12_1_resid_liquid_1500": {
"note": "Context: residual momentum on liquid breadth",
"mean_ic": 0.0289,
"ic_t_stat": 1.33,
"weeks": 35,
"avg_cross_section": 1471.2,
"ic_positive_pct": 60.0,
"reliable": true
}
},
"interpretation": {
"leak_ruled_out": false,
"junior_tier_drives_positive": true,
"prod_subset_still_negative": true,
"mom_conditional_negative_and_reliable": true,
"compositional_flip_story": "If prod subset IC is negative while full liquid-1500 is positive, the sign flip is compositional (bleeders / Nasdaq junk), not a temporal regime change. Unconditional fip pools continuous winners (want neg IC) against continuous losers/bleeders (want pos IC).",
"vol_tilt_warning": "vol_6m large negative IC on breadth: high-vol lottery names underperform. Production 80/20 high-vol tilt was validated on S&P-like names; must re-validate before any universe broaden."
},
"platform_verdict": "ALIVE as breadth-book tilt candidate among momentum winners only \u2014 still needs a book-level experiment; not a production wire-in."
}
File diff suppressed because it is too large Load Diff
+42
View File
@@ -10,8 +10,13 @@ Pipeline
3. Fetch ~5y daily bars from Alpaca for symbols missing (or short) in the copy. 3. Fetch ~5y daily bars from Alpaca for symbols missing (or short) in the copy.
4. Insert new tickers + OHLCV; mark them in side table ``research_rank_only`` 4. Insert new tickers + OHLCV; mark them in side table ``research_rank_only``
so the harness can feed signal IC without GTL/candidate replay. so the harness can feed signal IC without GTL/candidate replay.
5. Write a **completion manifest** (``<output>.manifest.json``) with ticker /
OHLCV / rank_only counts and finished-at. Breadth runners refuse to start
without a matching complete manifest same class of guard as calendar
truncation (see 2026-07-18 21:14 race: orphaned +0.0575 on a partial pool).
Resume-friendly: re-running skips symbols that already have ``--min-bars``. Resume-friendly: re-running skips symbols that already have ``--min-bars``.
A ``--limit`` smoke run writes ``complete: false`` so breadth mode still refuses.
Example Example
------- -------
@@ -197,12 +202,21 @@ async def _fetch_symbol_bars(
async def _main() -> None: async def _main() -> None:
# ROOT is already on sys.path; keep the helper import path-local.
from research_snapshot_manifest import ( # type: ignore[import-not-found]
clear_manifest,
write_completion_manifest,
)
args = _parse_args() args = _parse_args()
source = Path(args.source) source = Path(args.source)
output = Path(args.output) output = Path(args.output)
if not source.exists(): if not source.exists():
raise SystemExit(f"Source snapshot not found: {source}") raise SystemExit(f"Source snapshot not found: {source}")
# Any rebuild/update invalidates prior completion until we finish cleanly.
clear_manifest(output)
if args.force_copy or not output.exists(): if args.force_copy or not output.exists():
output.parent.mkdir(parents=True, exist_ok=True) output.parent.mkdir(parents=True, exist_ok=True)
if output.exists(): if output.exists():
@@ -375,12 +389,40 @@ async def _main() -> None:
text("SELECT COUNT(*) FROM ohlcv_records") text("SELECT COUNT(*) FROM ohlcv_records")
).scalar_one() ).scalar_one()
# Full planned work only when --limit is unset. Smoke runs stay incomplete
# so breadth mode cannot mythologize a 50-symbol toy pool.
is_complete = args.limit is None
manifest_path = write_completion_manifest(
output,
complete=is_complete,
sources=sources,
history_days=int(args.history_days),
min_bars=int(args.min_bars),
fetch_ok=ok,
fetch_fail=fail,
limit=args.limit,
extra={
"prod_symbols_at_start": len(prod_symbols),
"pool_size": len(pool),
"to_fetch": len(to_fetch),
},
)
print("Done.") print("Done.")
print(f" output: {output}") print(f" output: {output}")
print(f" tickers: {ticker_n}") print(f" tickers: {ticker_n}")
print(f" ohlcv rows: {ohlcv_n}") print(f" ohlcv rows: {ohlcv_n}")
print(f" research_rank_only: {rank_only_n}") print(f" research_rank_only: {rank_only_n}")
print(f" fetched ok/fail: {ok}/{fail}") print(f" fetched ok/fail: {ok}/{fail}")
print(
f" completion manifest: {manifest_path} "
f"(complete={is_complete})"
)
if not is_complete:
print(
" NOTE: --limit set → complete=false; breadth runners will refuse "
"this snapshot until a full extend finishes."
)
if __name__ == "__main__": if __name__ == "__main__":
+172
View File
@@ -0,0 +1,172 @@
"""Completion manifest for research.sqlite — cheap race guard.
The 2026-07-18 21:14 breadth run fired while ``extend_snapshot_universe`` was
still (or had just been) building the snapshot. Harness and shared-filter
recomputes agree on *complete* data, so the orphaned +0.0575 was incomplete
universe, not a code path bug.
Same class of protection as calendar-truncation assertions in the research
matrix: refuse to read results from a half-built artifact.
Layout
------
Sidecar path: ``<snapshot>.manifest.json`` next to the sqlite file
(e.g. ``backtest_snapshots/research.sqlite.manifest.json``).
"""
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from sqlalchemy import create_engine, text
MANIFEST_SCHEMA_VERSION = 1
def manifest_path_for(snapshot: Path) -> Path:
"""Sidecar path for a research snapshot."""
return Path(str(snapshot) + ".manifest.json")
def _count_snapshot(snapshot: Path) -> dict[str, int]:
engine = create_engine(
f"sqlite:///{snapshot.resolve().as_posix()}",
future=True,
)
try:
with engine.connect() as conn:
ticker_n = int(conn.execute(text("SELECT COUNT(*) FROM tickers")).scalar_one())
ohlcv_n = int(
conn.execute(text("SELECT COUNT(*) FROM ohlcv_records")).scalar_one()
)
try:
rank_only_n = int(
conn.execute(text("SELECT COUNT(*) FROM research_rank_only")).scalar_one()
)
except Exception:
rank_only_n = 0
finally:
engine.dispose()
return {
"ticker_count": ticker_n,
"ohlcv_row_count": ohlcv_n,
"rank_only_count": rank_only_n,
}
def write_completion_manifest(
snapshot: Path,
*,
complete: bool,
sources: dict[str, str] | None = None,
history_days: int | None = None,
min_bars: int | None = None,
fetch_ok: int | None = None,
fetch_fail: int | None = None,
limit: int | None = None,
extra: dict[str, Any] | None = None,
) -> Path:
"""Write (or overwrite) the sidecar completion manifest for *snapshot*."""
snapshot = Path(snapshot)
counts = _count_snapshot(snapshot) if snapshot.exists() else {
"ticker_count": 0,
"ohlcv_row_count": 0,
"rank_only_count": 0,
}
payload: dict[str, Any] = {
"schema_version": MANIFEST_SCHEMA_VERSION,
"snapshot": snapshot.name,
"snapshot_resolved": str(snapshot.resolve()) if snapshot.exists() else str(snapshot),
"complete": bool(complete),
"finished_at": datetime.now(timezone.utc).isoformat(),
**counts,
"sources": sources or {},
"history_days": history_days,
"min_bars": min_bars,
"fetch_ok": fetch_ok,
"fetch_fail": fetch_fail,
"limit": limit,
}
if extra:
payload["extra"] = extra
path = manifest_path_for(snapshot)
path.write_text(json.dumps(payload, indent=2, default=str) + "\n", encoding="utf-8")
return path
def clear_manifest(snapshot: Path) -> None:
"""Remove any existing completion manifest (start of a rebuild)."""
path = manifest_path_for(Path(snapshot))
if path.exists():
path.unlink()
def load_manifest(snapshot: Path) -> dict[str, Any] | None:
path = manifest_path_for(Path(snapshot))
if not path.exists():
return None
return json.loads(path.read_text(encoding="utf-8"))
def assert_research_snapshot_complete(snapshot: Path) -> dict[str, Any]:
"""Refuse breadth-mode work unless the extender finished cleanly.
Raises ``SystemExit`` with a clear message on any failure (missing
manifest, incomplete flag, or live counts that no longer match the
recorded totals e.g. a mid-run overwrite of the sqlite file).
"""
snapshot = Path(snapshot)
if not snapshot.exists():
raise SystemExit(
f"Research snapshot missing: {snapshot}\n"
"Build it with: python scripts/extend_snapshot_universe.py"
)
path = manifest_path_for(snapshot)
if not path.exists():
raise SystemExit(
f"Research snapshot completion manifest missing: {path}\n"
"Refusing breadth run — this is the guard that would have caught "
"the 2026-07-18 21:14 race against a half-built research.sqlite.\n"
"Re-run extend_snapshot_universe.py to completion (no --limit), "
"or for a trusted existing full snapshot:\n"
" python -c \"from pathlib import Path; "
"from scripts.research_snapshot_manifest import write_completion_manifest; "
f"write_completion_manifest(Path(r'{snapshot}'), complete=True)\""
)
try:
manifest = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise SystemExit(f"Corrupt research snapshot manifest {path}: {exc}") from exc
if not manifest.get("complete"):
raise SystemExit(
f"Research snapshot marked incomplete in {path}\n"
f"(finished_at={manifest.get('finished_at')}, limit={manifest.get('limit')}).\n"
"Re-run extend_snapshot_universe.py without --limit until Done."
)
live = _count_snapshot(snapshot)
mismatches: list[str] = []
for key in ("ticker_count", "ohlcv_row_count", "rank_only_count"):
recorded = manifest.get(key)
if recorded is None:
mismatches.append(f"{key}: missing in manifest")
continue
if int(recorded) != int(live[key]):
mismatches.append(
f"{key}: manifest={recorded} live={live[key]}"
)
if mismatches:
raise SystemExit(
"Research snapshot does not match its completion manifest "
f"({path}). Likely a partial rewrite or concurrent extend:\n - "
+ "\n - ".join(mismatches)
+ "\nRe-run extend_snapshot_universe.py to completion."
)
return {**manifest, "live_counts": live}
+60 -38
View File
@@ -3,9 +3,9 @@
Uses the same collection + ``_filter_liquid_breadth_week_rich`` as Uses the same collection + ``_filter_liquid_breadth_week_rich`` as
``run_backtest`` signal_eval. No parallel mask implementation. ``run_backtest`` signal_eval. No parallel mask implementation.
Reconciles the harness +0.0575 vs prior dual-path 0.017 disagreement by Single-sourced liquid-breadth fip diagnostics through harness mask helpers.
deleting the second mask, dumping membership/pre-post stats, and re-running Re-runs unconditional / tier / prod-subset / mom-conditional ICs and context
mom-conditional IC through the surviving path only. signals. Requires a complete research.sqlite completion manifest.
Research branch only. Example: Research branch only. Example:
@@ -49,7 +49,12 @@ def _parse_args() -> argparse.Namespace:
p.add_argument("--min-price", type=float, default=5.0) p.add_argument("--min-price", type=float, default=5.0)
p.add_argument("--workers", type=int, default=max(1, (mp.cpu_count() or 4) - 1)) p.add_argument("--workers", type=int, default=max(1, (mp.cpu_count() or 4) - 1))
p.add_argument("--allow-spawn", action="store_true") p.add_argument("--allow-spawn", action="store_true")
p.add_argument("--dump-weeks", type=int, default=5, help="How many weeks to dump membership for") p.add_argument(
"--dump-weeks",
type=int,
default=0,
help="Weeks of liquid membership symbol lists to embed (default 0 — keep reports compact)",
)
p.add_argument("--out", default=None) p.add_argument("--out", default=None)
p.add_argument("--quiet", action="store_true") p.add_argument("--quiet", action="store_true")
return p.parse_args() return p.parse_args()
@@ -173,8 +178,23 @@ def main() -> None:
args = _parse_args() args = _parse_args()
research = Path(args.research_snapshot) research = Path(args.research_snapshot)
prod = Path(args.prod_snapshot) prod = Path(args.prod_snapshot)
if not research.exists():
raise SystemExit(f"Missing {research}") # Refuse half-built research.sqlite (2026-07-18 21:14 race).
scripts_dir = Path(__file__).resolve().parent
if str(scripts_dir) not in sys.path:
sys.path.insert(0, str(scripts_dir))
from research_snapshot_manifest import ( # type: ignore[import-not-found]
assert_research_snapshot_complete,
)
manifest = assert_research_snapshot_complete(research)
if not args.quiet:
print(
f"Manifest ok: tickers={manifest.get('ticker_count')} "
f"ohlcv={manifest.get('ohlcv_row_count')} "
f"finished_at={manifest.get('finished_at')}",
flush=True,
)
# Force harness liquid-mode collection (same env as breadth run). # Force harness liquid-mode collection (same env as breadth run).
os.environ["BACKTEST_LIQUID_BREADTH"] = str(int(args.top_n)) os.environ["BACKTEST_LIQUID_BREADTH"] = str(int(args.top_n))
@@ -503,9 +523,9 @@ def main() -> None:
), ),
"mom_conditional_negative_and_reliable": mom_alive, "mom_conditional_negative_and_reliable": mom_alive,
"orphan_plus_five_sigma": ( "orphan_plus_five_sigma": (
"Prior report fip-breadth-20260718-211440-breadth.json listed " "Orphaned 21:14 row (+0.0575 / t +5.12) raced a partial "
"fip IC +0.0575 / t +5.12. This single-sourced recompute is the " "research.sqlite and was removed from reports/ (Git history only). "
"authoritative number; if it disagrees, the +0.0575 row is orphaned." "Harness path and shared filter agree on complete data."
), ),
"compositional_story": ( "compositional_story": (
"fip_id pools continuous winners (neg IC) vs continuous bleeders " "fip_id pools continuous winners (neg IC) vs continuous bleeders "
@@ -513,19 +533,36 @@ def main() -> None:
"liquid is less negative / positive — composition, not jumpiness premium." "liquid is less negative / positive — composition, not jumpiness premium."
), ),
"vol_tilt_warning": ( "vol_tilt_warning": (
"High-vol names underperform on breadth relative to S&P-like books. " "Authoritative liquid vol_6m IC ≈ 0.048 / t ≈ 1.36 — directional "
"Re-validate production 80/20 high-vol tilt before any universe broaden." "hypothesis only, not significant. Do not cite the orphaned 0.16 / "
"t 6.1. Re-validate production 80/20 high-vol tilt before any "
"universe broaden; it is not a settled finding on this pool."
),
"breadth_momentum_thesis": (
"Residual mom on liquid-1500 is +0.029 / t +1.33 vs fingerprint "
"0.055 / t 1.98 on 505 names — more breadth did not strengthen the "
"momentum t-stat on this pool. Clean mom edge lives in the large-cap "
"universe already traded. A fip tilt presupposes a breadth mom book "
"worth tilting; that baseline must be proven first."
), ),
}, },
"platform_verdict": ( "platform_verdict": (
"Mom-conditional fip ALIVE as book-tilt candidate (needs book sim) — " "Mom-conditional fip ALIVE as book-tilt candidate only — requires a "
"not production wire-in. Unconditional fip not green." "pre-registered two-arm breadth book (baseline liquid-1500 mom vs +fip "
"tilt) before any gate talk. Unconditional fip not green. Production: none."
if mom_alive if mom_alive
else ( else (
"fip CLOSED for production: mom-conditional does not clear iron rule " "fip CLOSED for production: mom-conditional does not clear iron rule "
"on single-sourced path. Display card is the resting place." "on single-sourced path. Display card is the resting place."
) )
), ),
"research_snapshot_manifest": {
"finished_at": manifest.get("finished_at"),
"ticker_count": manifest.get("ticker_count"),
"ohlcv_row_count": manifest.get("ohlcv_row_count"),
"rank_only_count": manifest.get("rank_only_count"),
"complete": manifest.get("complete"),
},
} }
stamp = datetime.now().strftime("%Y%m%d-%H%M%S") stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
@@ -533,8 +570,9 @@ def main() -> None:
out.parent.mkdir(parents=True, exist_ok=True) out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(results, indent=2, default=str), encoding="utf-8") out.write_text(json.dumps(results, indent=2, default=str), encoding="utf-8")
# Update research log # Append a machine reconciliation stub next to the JSON only — never clobber
_update_md(Path("docs/research/fip-breadth-ic.md"), results, out) # the curated research log at docs/research/fip-breadth-ic.md.
_update_md(out.with_suffix(".md"), results, out)
if not args.quiet: if not args.quiet:
print("=== Harness fip_id (authoritative) ===") print("=== Harness fip_id (authoritative) ===")
@@ -560,18 +598,10 @@ def _update_md(path: Path, results: dict, artifact: Path) -> None:
"", "",
"### Problem", "### Problem",
"", "",
"Two implementations of the liquid-1500 fip IC disagreed on **sign**:", "Machine stub only — curated narrative lives in `docs/research/fip-breadth-ic.md`.",
"",
"- Harness report `fip-breadth-20260718-211440-breadth.json`: **+0.0575 / t +5.12**",
"- Dual-path diagnostics (since deleted): **0.017 / t 1.9**",
"",
"A static read cannot decide which is right without single-sourcing the mask.",
"",
"### Resolution",
"", "",
f"- **Single source:** {results.get('single_source')}", f"- **Single source:** {results.get('single_source')}",
f"- **avg_cross_section semantics:** {results.get('avg_cross_section_semantics')}", f"- Harness vs shared-filter agree: "
f"- Harness `_signal_evaluation` vs shared-filter recompute agree: "
f"**{interp.get('harness_and_shared_filter_agree')}**", f"**{interp.get('harness_and_shared_filter_agree')}**",
"", "",
"### Authoritative unconditional fip (liquid top-N, post-mask)", "### Authoritative unconditional fip (liquid top-N, post-mask)",
@@ -587,10 +617,6 @@ def _update_md(path: Path, results: dict, artifact: Path) -> None:
f"| mask_binds_pct | {h.get('mask_binds_pct')} |", f"| mask_binds_pct | {h.get('mask_binds_pct')} |",
f"| reliable | {h.get('reliable')} |", f"| reliable | {h.get('reliable')} |",
"", "",
"The **+0.0575 / +5.12** row is **orphaned** if the authoritative recompute "
"disagrees; do not cite it. Iron-rule unconditional green still requires "
"negative sign and |IC| ≳ 0.03 on this row.",
"",
"### Checks (single-sourced)", "### Checks (single-sourced)",
"", "",
"| check | mean_ic | t | weeks | avg N | reliable |", "| check | mean_ic | t | weeks | avg N | reliable |",
@@ -626,21 +652,17 @@ def _update_md(path: Path, results: dict, artifact: Path) -> None:
"", "",
results.get("platform_verdict", ""), results.get("platform_verdict", ""),
"", "",
"### Vol-tilt warning", "### Vol-tilt / breadth-momentum notes",
"", "",
interp.get("vol_tilt_warning", ""), interp.get("vol_tilt_warning", ""),
"", "",
interp.get("breadth_momentum_thesis", ""),
"",
f"Artifact: `{artifact.as_posix()}`", f"Artifact: `{artifact.as_posix()}`",
"", "",
]) ])
existing = path.read_text(encoding="utf-8") if path.exists() else "" # Always overwrite the machine stub (never the curated research log).
marker = "## Reconciliation" path.write_text("\n".join(lines).lstrip() + "\n", encoding="utf-8")
if marker in existing:
existing = existing.split(marker)[0].rstrip() + "\n"
# Also strip old dual-path diagnostics section if present after reconciliation
if "## Follow-up diagnostics" in existing and marker not in path.read_text(encoding="utf-8") if path.exists() else "":
pass
path.write_text(existing.rstrip() + "\n" + "\n".join(lines), encoding="utf-8")
if __name__ == "__main__": if __name__ == "__main__":
+25 -9
View File
@@ -1,8 +1,9 @@
"""Phase B: fip_id IC on liquid-breadth cross-section (local research only). """Phase B: fip_id IC on liquid-breadth cross-section (local research only).
1. Fingerprint check on the unextended prod snapshot (must IC 0.045 / t 2.9). 1. Fingerprint check on the unextended prod snapshot (must IC 0.045 / t 2.9).
2. Run signal_eval on research.sqlite with BACKTEST_LIQUID_BREADTH=1500 PIT mask. 2. Assert research.sqlite has a matching **completion manifest** (race guard).
3. Write a research report under docs/research/ and reports/. 3. Run signal_eval on research.sqlite with BACKTEST_LIQUID_BREADTH=1500 PIT mask.
4. Write a research report under docs/research/ and reports/.
Does not modify production DB, gate, scanner, or schedule. Does not modify production DB, gate, scanner, or schedule.
@@ -209,7 +210,9 @@ async def _main() -> None:
stamp = datetime.now().strftime("%Y%m%d-%H%M%S") stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
out_json = Path(args.out) if args.out else Path("reports") / f"fip-breadth-{stamp}.json" out_json = Path(args.out) if args.out else Path("reports") / f"fip-breadth-{stamp}.json"
out_json.parent.mkdir(parents=True, exist_ok=True) out_json.parent.mkdir(parents=True, exist_ok=True)
out_md = Path("docs/research") / "fip-breadth-ic.md" # Never clobber the curated research log (docs/research/fip-breadth-ic.md).
# Machine summary goes next to the JSON report only.
out_md = out_json.with_suffix(".md")
payload: dict = { payload: dict = {
"generated_at": datetime.now().isoformat(), "generated_at": datetime.now().isoformat(),
@@ -257,17 +260,30 @@ async def _main() -> None:
# --- 2) Breadth --- # --- 2) Breadth ---
if not args.skip_research: if not args.skip_research:
if not research.exists(): # Refuse half-built research.sqlite (2026-07-18 21:14 race).
raise SystemExit( scripts_dir = Path(__file__).resolve().parent
f"Research snapshot missing: {research}\n" if str(scripts_dir) not in sys.path:
"Build it with: python scripts/extend_snapshot_universe.py" sys.path.insert(0, str(scripts_dir))
) from research_snapshot_manifest import ( # type: ignore[import-not-found]
assert_research_snapshot_complete,
)
manifest = assert_research_snapshot_complete(research)
payload["research_snapshot_manifest"] = {
"finished_at": manifest.get("finished_at"),
"ticker_count": manifest.get("ticker_count"),
"ohlcv_row_count": manifest.get("ohlcv_row_count"),
"rank_only_count": manifest.get("rank_only_count"),
"complete": manifest.get("complete"),
}
os.environ["BACKTEST_LIQUID_BREADTH"] = str(int(args.liquid_breadth)) os.environ["BACKTEST_LIQUID_BREADTH"] = str(int(args.liquid_breadth))
os.environ["BACKTEST_LIQUID_MIN_PRICE"] = str(float(args.min_price)) os.environ["BACKTEST_LIQUID_MIN_PRICE"] = str(float(args.min_price))
if not args.quiet: if not args.quiet:
print( print(
f"Breadth run on {research} " f"Breadth run on {research} "
f"(top {args.liquid_breadth}, min_price={args.min_price})…" f"(top {args.liquid_breadth}, min_price={args.min_price}; "
f"manifest ok tickers={manifest.get('ticker_count')} "
f"finished_at={manifest.get('finished_at')})…"
) )
br_report = await _run_signal_eval( br_report = await _run_signal_eval(
research, workers=args.workers, quiet=args.quiet research, workers=args.workers, quiet=args.quiet
@@ -0,0 +1,133 @@
"""Completion-manifest guard for research.sqlite breadth runs."""
from __future__ import annotations
import json
import sys
from pathlib import Path
import pytest
from sqlalchemy import create_engine, text
ROOT = Path(__file__).resolve().parents[2]
SCRIPTS = ROOT / "scripts"
if str(SCRIPTS) not in sys.path:
sys.path.insert(0, str(SCRIPTS))
from research_snapshot_manifest import ( # noqa: E402
assert_research_snapshot_complete,
clear_manifest,
load_manifest,
manifest_path_for,
write_completion_manifest,
)
def _tiny_research_db(path: Path, *, tickers: int = 3, bars_each: int = 5) -> None:
engine = create_engine(f"sqlite:///{path.resolve().as_posix()}", future=True)
with engine.begin() as conn:
conn.execute(
text(
"CREATE TABLE tickers ("
"id INTEGER PRIMARY KEY, symbol TEXT NOT NULL UNIQUE, "
"name TEXT, created_at TEXT)"
)
)
conn.execute(
text(
"CREATE TABLE ohlcv_records ("
"id INTEGER PRIMARY KEY, ticker_id INTEGER, date TEXT, "
"open REAL, high REAL, low REAL, close REAL, volume INTEGER, "
"created_at TEXT)"
)
)
conn.execute(
text(
"CREATE TABLE research_rank_only ("
"ticker_id INTEGER PRIMARY KEY, symbol TEXT NOT NULL UNIQUE)"
)
)
for i in range(tickers):
sym = f"T{i}"
conn.execute(
text(
"INSERT INTO tickers (id, symbol, name, created_at) "
"VALUES (:id, :sym, NULL, '2026-01-01')"
),
{"id": i + 1, "sym": sym},
)
if i > 0:
conn.execute(
text(
"INSERT INTO research_rank_only (ticker_id, symbol) "
"VALUES (:id, :sym)"
),
{"id": i + 1, "sym": sym},
)
for d in range(bars_each):
conn.execute(
text(
"INSERT INTO ohlcv_records "
"(ticker_id, date, open, high, low, close, volume, created_at) "
"VALUES (:tid, :date, 1,1,1,1,100, '2026-01-01')"
),
{"tid": i + 1, "date": f"2026-01-{d+1:02d}"},
)
engine.dispose()
def test_write_and_assert_complete(tmp_path: Path) -> None:
snap = tmp_path / "research.sqlite"
_tiny_research_db(snap)
path = write_completion_manifest(snap, complete=True, sources={"t": "unit"})
assert path == manifest_path_for(snap)
assert path.exists()
m = assert_research_snapshot_complete(snap)
assert m["complete"] is True
assert m["ticker_count"] == 3
assert m["ohlcv_row_count"] == 15
assert m["rank_only_count"] == 2
assert m["live_counts"]["ticker_count"] == 3
def test_refuse_missing_manifest(tmp_path: Path) -> None:
snap = tmp_path / "research.sqlite"
_tiny_research_db(snap)
with pytest.raises(SystemExit, match="manifest missing"):
assert_research_snapshot_complete(snap)
def test_refuse_incomplete_flag(tmp_path: Path) -> None:
snap = tmp_path / "research.sqlite"
_tiny_research_db(snap)
write_completion_manifest(snap, complete=False, limit=50)
with pytest.raises(SystemExit, match="marked incomplete"):
assert_research_snapshot_complete(snap)
def test_refuse_count_mismatch(tmp_path: Path) -> None:
snap = tmp_path / "research.sqlite"
_tiny_research_db(snap)
write_completion_manifest(snap, complete=True)
# Tamper: change live DB after manifest written
engine = create_engine(f"sqlite:///{snap.resolve().as_posix()}", future=True)
with engine.begin() as conn:
conn.execute(
text(
"INSERT INTO tickers (id, symbol, name, created_at) "
"VALUES (99, 'EXTRA', NULL, '2026-01-01')"
)
)
engine.dispose()
with pytest.raises(SystemExit, match="does not match"):
assert_research_snapshot_complete(snap)
def test_clear_manifest(tmp_path: Path) -> None:
snap = tmp_path / "research.sqlite"
_tiny_research_db(snap)
write_completion_manifest(snap, complete=True)
assert load_manifest(snap) is not None
clear_manifest(snap)
assert load_manifest(snap) is None