# Total-return momentum — paired portfolio replay (Step 3a)

**Run date:** 2026-09-07 · **Status:** measured; **no rollout decision taken**
**Report:** `reports/tr-total-return-ab.json` · **Script:** `scripts/run_total_return_ab.py`
**Inputs:** `backtest_snapshots/survivorship-a.sqlite` (bars),
`backtest_snapshots/tr-rank-cache.sqlite` (both adjusted series),
`backtest_snapshots/tr-ab-candidates.pkl` (202,765 shared candidates)

## Question

[The ranking screen](tr-rank-agreement.md) showed the momentum basis changes the
admitted list on every evaluated date. Does that reach actual admissions and P&L?

## What was held identical

Setup candidates were generated **once** (202,765 over 505 symbols, matching the
count in the standalone backtest report) and reused by both books. Only momentum
and its dependents were recomputed per arm: residual and activation percentiles,
`qualified`, and the 80/20 blend used for ordering.

| Held fixed | Varied |
|---|---|
| Setup geometry — entry, stop, target, R:R | Momentum series: SPLIT stock vs SPLIT SPY (control), ALL vs ALL SPY (treatment) |
| Execution on the snapshot's nominal bars | The percentiles, `qualified` and ordering that follow from it |
| `vol_6m` volatility leg, from candidate generation | |
| Non-momentum gates, `max_positions`, risk per trade, ATR-trail exit, gate-reset re-entry | |

Both arms read the same cache, so a refreshed-vs-snapshot difference cannot be
mistaken for a basis difference. Each book then keeps its own holdings, so a
divergent admission changes later slot occupancy — that path dependence is part
of what is measured.

**Harness parity.** The control arm reproduces the production row of
`reports/backtest-20260907-survivorship-a.json` exactly — 322 trades, 424.6%
total return, 51.6% CAGR, 21.4% drawdown, 2.09 Sharpe. Any difference below is
therefore attributable to the basis, not to a re-implemented book.

## Result

| Metric | Control (price return) | Treatment (total return) | Δ |
|---|---:|---:|---:|
| Trades | 322 | 334 | +12 |
| Total return % | 424.6 | 376.5 | −48.1 |
| CAGR % | 51.6 | 47.9 | **−3.7** |
| Max drawdown % | 21.4 | 19.5 | **−1.9** (better) |
| Sharpe | 2.09 | 1.98 | −0.11 |
| Win rate % | 37.0 | 36.8 | −0.2 |
| Profit factor | 2.48 | 2.33 | −0.15 |

**The books really do differ.** 271 entries are shared; **51 are taken only by
control and 63 only by treatment** — about 30% of the union of entries. The
composition follows the Step 2 mechanism: control takes MAR, AAPL, BIIB, LVS;
treatment takes BG, OKE, DE, ARES. Some differences are timing rather than
selection — ABBV is entered on 2022-09-15 by control and 2022-09-22 by treatment.

## Where the difference sits

**One year carries all of it.** Per-year returns:

| Year | Control % | Treatment % | Δpp |
|---|---:|---:|---:|
| 2022 | 8.1 | 9.9 | +1.8 |
| 2023 | 42.0 | 41.9 | −0.1 |
| 2024 | 36.4 | 38.4 | +2.0 |
| 2025 | 80.2 | 58.1 | **−22.1** |
| 2026 | 39.2 | 39.6 | +0.4 |

Treatment wins three years and loses two, and four of the five differ by under
2.1pp. The entire −3.7pp CAGR gap is 2025. That is a description of *where* the
shortfall sits — a useful thing to know when reading it — and nothing more. It is
not grounds for calling the shortfall noise or the estimate unstable.

**Uncertainty is not yet quantified.** The per-arm `sharpe_se` values (0.497 and
0.498) are the standard errors of each Sharpe *individually*. They say nothing
about the uncertainty of the **difference**: the two books share dates and most
of their positions, so the arms are strongly correlated and the standard error of
their difference depends on that covariance, not on the two marginal errors. An
earlier version of this document compared the 0.11 Sharpe gap to a 0.50 marginal
SE and concluded the books were "statistically indistinguishable" — that
inference does not follow, and the same applies to the drawdown difference, for
which no error estimate was reported at all. Both claims are withdrawn.

## Paired block bootstrap on the difference

Pre-specified before running — seed `20260907`, **5,000** replicates, block
lengths **21 and 63** sessions, both reported rather than whichever reads better.
The two arms' daily return streams are date-aligned (the script aborts if they
are not), and **the same block indices are drawn for both arms**, so the pairing
and whatever cross-arm correlation exists on those days are preserved. Moving
blocks retain some of the serial dependence an i.i.d. resample would destroy.

Treatment − control, 95% interval:

| Block | Metric | Mean | 2.5% | 97.5% | Share of replicates < 0 |
|---|---|---:|---:|---:|---:|
| 21 | ΔCAGR pp | −3.61 | −14.76 | +6.45 | 0.75 |
| 21 | ΔSharpe | −0.109 | −0.433 | +0.198 | 0.75 |
| 21 | ΔMaxDD pp | −1.00 | −7.50 | +4.34 | 0.63 |
| 63 | ΔCAGR pp | −3.78 | −14.67 | +5.40 | 0.76 |
| 63 | ΔSharpe | −0.113 | −0.428 | +0.182 | 0.76 |
| 63 | ΔMaxDD pp | −1.79 | −8.22 | +2.68 | 0.78 |

The two block lengths agree closely, which is reassuring about the resampling
itself. Every interval spans zero, and roughly three-quarters of replicates fall
below zero for CAGR and Sharpe (treatment worse) and for drawdown (treatment
better, since a negative ΔMaxDD is a shallower drawdown). So the observed window
leans toward control on return and treatment on drawdown without resolving either
sign confidently.

**What this diagnostic is not.** It resamples the *realised* daily return streams;
it does not re-run the strategy, so it cannot capture how a reordered history
would have changed admissions or holdings. It is an exploratory uncertainty
estimate from the observed window — not out-of-sample validation, and not a
rollout gate.

## Verdict

**Decision: production stays unchanged.** No schema, no ingestion, no wider
parameter sweep.

- **Established:** the basis is production-relevant. It changes roughly 30% of
  entries, in the yield-driven direction the screen predicted.
- **Not established:** which basis performs better. Point estimates favour control
  on return and Sharpe and treatment on drawdown; every bootstrap interval spans
  zero, with about three-quarters of the mass on the point-estimate side. The
  observed window does not resolve the sign.
- No acceptable return sacrifice is being chosen from these numbers, and no
  threshold is being fitted to them after the fact.

What a decision would still need:

1. **A pre-registered numeric rule, set before it is applied.** The plan commits
   to non-inferiority in principle — total return is the basis 12-1 momentum is
   defined on, so an in-sample shortfall should not automatically lock in the
   basis we believe is measured wrong — but carries no threshold. An earlier draft
   used ΔCAGR ≥ −0.50pp at ΔMaxDD ≤ +1.50pp. Choosing a threshold now, with
   −3.7pp on the table, would be fitting the rule to the answer.
2. **Out-of-sample evidence.** One five-year snapshot, in-sample throughout.
3. Note what would *not* help: a matrix over capacity, start date or cadence
   produces **sensitivity checks, not independent samples**. Fifty-three
   overlapping paths from one price history are not fifty-three pieces of
   evidence, and treating them that way would overstate what a sweep buys.

## Reproduce

```powershell
python scripts/run_total_return_ab.py --build-candidates   # ~9 min, cached
python scripts/run_total_return_ab.py                      # both arms + bootstrap, ~40 s
```

The bootstrap parameters are constants in the script (`BOOTSTRAP_SEED`,
`BOOTSTRAP_REPLICATES`, `BOOTSTRAP_BLOCKS`) rather than flags, so a run cannot
quietly pick a different seed or block length.

Guards in `tests/unit/test_total_return_ab.py`: identical input series must
produce identical books (so an observed difference is the basis, not replay
nondeterminism); a controlled momentum change must move a name across the cutoff
(so the harness can see what it is looking for); the volatility leg must be
untouched between arms; and a name with no computable momentum in an arm must not
qualify on a value carried over from candidate generation.