research: survivorship sensitivity and total-return momentum investigation

Three questions answered against snapshot copies, with no production change.

Step 1 — how sensitive is the backtest to the ranking universe? Restoring 85
S&P 500 constituents removed between 2021-06 and 2026-07 moves the same window
from 2.09 Sharpe / 51.6% CAGR to 1.57 / 38.1%. The failed banks never reached
either book (identical best/worst trade R), so this measures universe
recomposition, not survivorship bias. The removals list is curated by hand with
per-symbol sources; only three explicitly confirmed terminal events get marks,
and four unresolved endings are reported as caveats rather than inferred.

Step 2 — does a total-return basis change the ranking? Ordered lists differ on
1009 of 1009 evaluated dates, 4.56 replacements per date. The shift correlates
with trailing implied yield (mean Spearman +0.79), which is strong evidence
consistent with a dividend effect but does not isolate own-yield, since both
the stock and benchmark legs change.

Step 3a — does that reach P&L? A paired replay reusing one shared candidate set
(control reproduces the production row exactly) changes ~30% of entries.
Treatment is -3.7pp CAGR and -1.9pp drawdown, with the entire return gap in
2025. A paired moving-block bootstrap (seed 20260907, 5000 replicates, 21/63
blocks, both reported) puts every interval across zero.

Decision: production stays unchanged. No schema, no ingestion, no wider sweep.
The observed window does not resolve which basis is better, and choosing a
threshold now — with -3.7pp on the table — would be fitting the rule to the
answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAH7vVtn7qbcn1EVheecHQ
This commit is contained in:
dennisthiessenandClaude Opus 5 committed 2026-09-07 19:29:34 +02:00
1 parent 8453b87290
commit ebe9047403
16 files changed
+442522

No files matched your search

+10
View File
@@ -145,6 +145,8 @@ knobs.
| **`fip_id` / liquid breadth** | Fingerprint −0.045 / t −2.91; liquid unconditional **−0.017 / t −1.85** (not green); mom-conditional **−0.088 / t −4.58** | **Parked.** Orphan +0.0575 died (snapshot race). Breadth did not strengthen resid-mom t-stat. Optional reopen = pre-registered two-arm liquid-1500 book first. See [fip-breadth-ic.md](fip-breadth-ic.md) |
| **Broader universe** | Composition changes factor signs (fip tug-of-war); vol-tilt on breadth is only a **directional hypothesis** (auth. −0.048 / t −1.36) | Any prod broaden must re-validate 80/20 tilt; offline research only; research.sqlite requires completion manifest |
| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
| **Total-return momentum basis** | Ranks change every day (screen: **1,009 of 1,009** dates, ~4.6 names replaced per date, shift correlates **+0.79** with trailing yield) **and it reaches trades** — paired replay differs on ~30% of entries (51 control-only, 63 treatment-only, 271 shared). Which basis is better is unresolved: paired block bootstrap (5,000 reps, 21- and 63-session blocks, seed fixed beforehand) gives ΔCAGR −3.6/−3.8pp with 95% intervals spanning zero ([−14.8, +6.4] and [−14.7, +5.4]), ΔSharpe ≈ −0.11, ΔMaxDD −1.0/−1.8pp in treatment's favour; ~75% of replicates on the point-estimate side. All of the CAGR shortfall sits in 2025 | **Production unchanged.** Needs a threshold pre-registered before it is applied, plus out-of-sample evidence; a capacity/date/cadence matrix would be sensitivity checks, not independent samples. Schema/ingestion deferred. [Screen](tr-rank-agreement.md) · [Paired replay](tr-total-return-ab.md) · [plan](../data-integrity-plan.md) |
| **Universe sensitivity / point-in-time membership** | Restoring 85 removed constituents (2021-06→2026-07, prod snapshot) moves the production row from **Sharpe 2.09 / CAGR 51.6%** to **1.57 / 38.1%** — all of it universe recomposition; no failed-bank position reached either book (trade extremes identical, and a control shows the extra replay signals changed no portfolio decision). Says the ranks are highly sensitive to the universe, not that bias is 13.5pp | Not a bias estimate: addition lookahead is still unremoved and arm B is not point-in-time. A real PIT audit is separate work. [Survivorship sensitivity](survivorship-audit.md) |
| **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
| **Minimum effective-risk floor** | ⛔ CLOSED NEGATIVE, not run. The floor lifts EV/trade (+0.032) and PF (+0.073) *by deleting trades* — 11.4 fewer per path, never one more — and costs **−0.753pp CAGR**, −0.047 Sharpe, −0.051 Calmar | Do not run the A/B; its EV-based pass rule would have shipped it. [Withdrawn specification](effective-risk-floor-ab.md) / [findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens) |
@@ -176,6 +178,14 @@ knobs.
`research.sqlite`. Extender now writes a completion manifest; breadth mode
refuses without a match. Same class of protection as calendar-truncation
asserts — do not re-mythologize numbers computed on half a universe.
8. **A big delta is not the mechanism you assumed.** Restoring 85 removed index
constituents cost 13.5pp of CAGR — and none of it came from the failed banks
the experiment was built around. The driver was the universe the ranks are
computed over. Measure which part of a change did the work before naming the
effect, and state what the control actually isolates: ours compared replay
*bounds* on one snapshot, so it measured the extra signals, not the synthetic
bars those signals resolve against.
[Survivorship sensitivity](survivorship-audit.md)
---