fix(backtest): address the second review of the breakeven and book-config work

- The breakeven study grades at the shadow book's capacity AND risk per trade, and labels each with where it came from. A missing setting row now reads "shadow_book default" instead of crediting the setting; the committed report said shadow_book_capacity though the snapshot has no such row. - _production_book_config reads strategies through the same _portfolio_monitor_strategies() hook as the monitor. - breakeven_gate_reset is rejected when there is no post-stop episode or cooldown for it to start, and the study skips its diagnostic arm for a book without one, so a production config change cannot abort the run. - identical_to_control is renamed ties_control_on_graded_metrics and needs real values: be_2r ties on every graded metric but still changed 2 trades. - _BookConfig drops an unused field and hashes by identity; the min-R:R sweep swaps its gate in with dataclasses.replace instead of passing it. - New tests cover the parity guard's success path, the no-config fallback and the gate-reset check. The breakeven report is regenerated: every row, diagnostic row and the parity check are identical; only the labels, the risk fields and the renamed flag changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
research: volatility-standardized residual momentum screen, failed
2026-10-10 21:41:44 +02:00 · 2026-10-10 18:45:24 +02:00 · 2026-10-10 12:17:27 +02:00 · 2026-10-10 09:42:47 +02:00
14 changed files with 147783 additions and 160 deletions
@@ -630,6 +630,7 @@ Research-only flags, all off by default (the default report is byte-identical to
 | `BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD` | Adds a `holdout` section: train (entries before) vs test (entries on/after), as disjoint books |
 | `BACKTEST_MIN_RR_SWEEP=1` | Sweeps the activation R:R floor against portfolio Sharpe. Combine with `BACKTEST_HOLDOUT_SPLIT` to sweep out-of-sample |
 | `BACKTEST_RESEARCH_EXITS=1` | Adds the rejected take-profit exit rows to the exit comparison |
 | `BACKTEST_BREAKEVEN_STUDY=1` | Adds a `breakeven_stop_study` section: the production book with a breakeven stop at +1R / +2R vs control, graded on a fixed train/test split ([protocol](docs/research/breakeven-stop.md)) |
 | `BACKTEST_ATR_TARGET_FALLBACK=k` | Synthesizes a k×ATR target where S/R offers none |
 | `BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1` | Restricts that fallback to setups with genuinely no structure ahead |
@@ -30,6 +30,7 @@ Environment variables (see also run_backtest_snapshot.py):
    BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1
    BACKTEST_RESEARCH_EXITS=1
    BACKTEST_MIN_RR_SWEEP=1
    BACKTEST_BREAKEVEN_STUDY=1          # docs/research/breakeven-stop.md
  Broad-universe signal research (local snapshots only; inert when unset):
    BACKTEST_LIQUID_BREADTH=1500   # PIT top-N by 63d median $vol, price floor
@@ -50,7 +51,7 @@ import statistics
 from collections import defaultdict
 from collections.abc import Callable
 from concurrent.futures import ProcessPoolExecutor
-from dataclasses import dataclass, field
+from dataclasses import dataclass, field, replace
 from datetime import date, datetime, timezone
 from types import SimpleNamespace
 from typing import Any
@@ -2298,6 +2299,8 @@ def _simulate_portfolio(
    corr_lookback: int = 120,
    corr_action: str = "skip",
    corr_min_overlap: int = 60,
    breakeven_r: float | None = None,
    breakeven_gate_reset: bool = False,
 ) -> dict | None:
    """Replay the qualified setups as ONE capital-constrained book and report
    portfolio economics from the daily equity curve (return, CAGR, drawdown,
@@ -2330,6 +2333,14 @@ def _simulate_portfolio(
    caps skip or half-size candidates whose max pairwise 120d return correlation
    with open holdings exceeds ``corr_max``.
    ``breakeven_r`` (research only, ATR-trail policies): once the highest close
    reaches entry + ``breakeven_r`` x initial risk, raise the stop to the entry
    price. Like the trail, it is active from the next bar and only moves up.
    A stop it set exits as "breakeven_stop". Live gate locks key on initial-stop
    exits only, so by default that exit, like a trailing stop, neither starts
    the cooldown nor the post-stop episode; ``breakeven_gate_reset`` makes it
    do both, as an initial stop would.
    Returns None when there is nothing to trade. ``cost_per_side`` is charged on
    entry and exit and therefore changes both cash availability and subsequent
    position sizing.
@@ -2347,6 +2358,17 @@ def _simulate_portfolio(
        raise ValueError("corr_action must be 'skip' or 'half_size'")
    if vol_target is not None and vol_target <= 0:
        raise ValueError("vol_target must be positive when set")
    if breakeven_r is not None:
        if not math.isfinite(breakeven_r) or breakeven_r <= 0:
            raise ValueError("breakeven_r must be positive and finite when set")
        if exit_policy not in ("atr_trail3", "atr_trail3_target"):
            raise ValueError("breakeven_r only applies to the ATR-trail exit policies")
    if breakeven_gate_reset and breakeven_r is None:
        raise ValueError("breakeven_gate_reset needs breakeven_r")
    if breakeven_gate_reset and post_stop_reentry_fn is None and reentry_cooldown_sessions <= 0:
        raise ValueError(
            "breakeven_gate_reset needs post_stop_reentry_fn or reentry_cooldown_sessions"
        )
    clamp_lo, clamp_hi = float(vol_clamp[0]), float(vol_clamp[1])
    if clamp_lo <= 0 or clamp_hi < clamp_lo:
        raise ValueError("vol_clamp must satisfy 0 < lo <= hi")
@@ -2478,6 +2500,23 @@ def _simulate_portfolio(
            atr_cache[key] = None
        return atr_cache[key]
    def _ratchet_stop(pos: dict, sym: str, idx: int, close: float) -> None:
        """ATR-trail policies: raise the stop from this bar's close — the trail
        off the highest close, then the breakeven floor. Effective from the next
        bar; the stop never moves down."""
        pos["highest_close"] = max(pos["highest_close"], close)
        atr = _atr(sym, idx)
        if atr is not None:
            next_stop = pos["highest_close"] - atr_trail_multiplier * atr
            if next_stop < close and next_stop > pos["stop"]:
                pos["stop"] = next_stop
                pos["stop_source"] = "trail"
        if breakeven_r is not None and pos["entry"] > pos["stop"]:
            risk = pos["entry"] - pos["initial_stop"]
            if risk > 0 and pos["highest_close"] >= pos["entry"] + breakeven_r * risk:
                pos["stop"] = pos["entry"]
                pos["stop_source"] = "breakeven"
    def _close_trade(sym: str, fill: float, reason: str) -> dict:
        nonlocal cash
        pos = positions.pop(sym)
@@ -2520,11 +2559,12 @@ def _simulate_portfolio(
            if bar.low <= pos["stop"]:
                # Same-bar stop+target resolves as the loss (conservative, like
                # the evaluator); gap through the stop fills at the open.
-                reason = (
+                if pos["stop_source"] == "breakeven":
-                    "trailing_stop"
+                    reason = "breakeven_stop"
-                    if pos["stop"] > pos["initial_stop"] + 1e-9
+                elif pos["stop"] > pos["initial_stop"] + 1e-9:
-                    else "stop"
+                    reason = "trailing_stop"
-                )
+                else:
                    reason = "stop"
                survived_refresh = False
                if reason == "stop" and initial_stop_refresh_fn is not None:
                    stop_refresh_attempts += 1
@@ -2545,9 +2585,12 @@ def _simulate_portfolio(
                if not survived_refresh:
                    fill = min(pos["stop"], bar.open)
                    closed_pos = _close_trade(sym, fill, reason)
-                    if reason == "stop" and cooldown_sessions:
+                    resets_gate = reason == "stop" or (
                        reason == "breakeven_stop" and breakeven_gate_reset
                    )
                    if resets_gate and cooldown_sessions:
                        cooldown_until_index[sym] = calendar_index + cooldown_sessions
-                    if reason == "stop" and post_stop_reentry_fn is not None:
+                    if resets_gate and post_stop_reentry_fn is not None:
                        post_stop_events += 1
                        post_stop_states[sym] = {
                            "stop_ord": o,
@@ -2584,12 +2627,7 @@ def _simulate_portfolio(
                _close_trade(sym, bar.close, "time")
                continue
            if exit_policy in ("atr_trail3", "atr_trail3_target"):
-                pos["highest_close"] = max(pos["highest_close"], bar.close)
+                _ratchet_stop(pos, sym, bar.idx, bar.close)
                atr = _atr(sym, bar.idx)
                if atr is not None:
                    next_stop = pos["highest_close"] - atr_trail_multiplier * atr
                    if next_stop < bar.close:
                        pos["stop"] = max(pos["stop"], next_stop)
        # 2) entries — close-fill at signal close, or next-open fills of prior signals
        equity = _marked_equity()
@@ -2725,6 +2763,7 @@ def _simulate_portfolio(
                "entry_ord": entry_ord,
                "initial_stop": stop,
                "stop": stop,
                "stop_source": "initial",
                "target": float(c["target"]) if c.get("target") else None,
                "entry_cost": entry_cost,
                "bars_held": 0,
@@ -2744,7 +2783,6 @@ def _simulate_portfolio(
            # bars_held stays 0 on the fill day (matches close-fill cadence).
            if fill_mode == FILL_MODE_NEXT_OPEN and fill_bar is not None:
                positions[sym]["last_close"] = fill_bar.close
                positions[sym]["highest_close"] = max(entry, fill_bar.close)
                if fill_bar.low <= stop:
                    fill = min(stop, fill_bar.open)
                    closed = _close_trade(sym, fill, "stop")
@@ -2765,16 +2803,7 @@ def _simulate_portfolio(
                            "gate_went_unqualified": False,
                        }
                elif exit_policy in ("atr_trail3", "atr_trail3_target"):
-                    atr = _atr(sym, fill_bar.idx)
+                    _ratchet_stop(positions[sym], sym, fill_bar.idx, fill_bar.close)
                    if atr is not None:
                        next_stop = (
                            positions[sym]["highest_close"]
                            - atr_trail_multiplier * atr
                        )
                        if next_stop < fill_bar.close:
                            positions[sym]["stop"] = max(
                                positions[sym]["stop"], next_stop
                            )
            equity = _marked_equity()
        for c in fill_candidates:
@@ -3059,6 +3088,17 @@ def _simulate_portfolio(
    if max_entry_gap_pct is not None:
        result["max_entry_gap_pct"] = max_entry_gap_pct
        result["skipped_gap_cap"] = skipped_gap_cap
    if breakeven_r is not None:
        result["breakeven_r"] = breakeven_r
        result["breakeven_gate_reset"] = breakeven_gate_reset
        breakeven_exit_keys = {
            (t["symbol"], t["exit_ord"]) for t in trades if t["reason"] == "breakeven_stop"
        }
        result["breakeven_exits"] = len(breakeven_exit_keys)
        # Bought back in the session it was stopped out of breakeven.
        result["breakeven_same_session_reentries"] = sum(
            1 for t in trades if (t["symbol"], t["entry_ord"]) in breakeven_exit_keys
        )
    if curve_payload is not None:
        result["equity_curve"] = curve_payload
    if benchmark_payload is not None:
@@ -3499,6 +3539,94 @@ def _entry_variant_config(variant: str) -> dict | None:
    return next((cfg for cfg in STRATEGY_VARIANTS if cfg["variant"] == variant), None)
 # eq=False keeps identity hashing: the fields hold dicts and callables.
@dataclass(frozen=True, eq=False)
 class _BookConfig:
    """How one ``PORTFOLIO_MONITOR_STRATEGIES`` entry is replayed. The monitor and
    every production-book study resolve it here, so they cannot drift apart."""
    strategy: dict
    exit_policy: str
    live_exit_mode: str | None
    hold_days: int
    trail_multiplier: float
    reentry_policy: str
    qualified_fn: Callable[[dict], bool] | None
    ranking_key: str
    max_positions: int
    risk_per_trade: float
    def post_stop_reentry_fn(
        self,
        candidates: list[dict],
        prices: dict[str, tuple],
        cadence: str,
    ) -> Callable[[str, int, dict, Any], dict | None] | None:
        """The gate-reset callback on this book's own gate, or None for immediate
        re-entry. A sweep that changes the gate replaces ``qualified_fn`` first."""
        if self.reentry_policy != "gate_reset":
            return None
        return _make_gate_reset_reentry_fn(
            candidates,
            prices,
            cadence=cadence,
            qualified_fn=self.qualified_fn,
            ranking_key=self.ranking_key,
        )
 def _strategy_book_config(
    strategy: dict, hold_days: int, live_exit_policy: dict | None
 ) -> _BookConfig | None:
    entry_cfg = _entry_variant_config(str(strategy["entry_variant"]))
    if entry_cfg is None:
        return None
    # Live-config rows replay the runtime qualification flag and Admin exit
    # policy; research rows replay their frozen variant gate and exit.
    use_live = bool(strategy.get("use_live_config"))
    exit_policy = str(strategy["exit_policy"])
    row_hold_days = hold_days
    trail_multiplier = ATR_TRAIL_MULTIPLIER
    live_exit_mode: str | None = None
    if use_live and live_exit_policy is not None:
        live_exit_mode = str(live_exit_policy.get("mode", "atr_trailing"))
        exit_policy = LIVE_EXIT_MODE_TO_SIM.get(live_exit_mode, "atr_trail3")
        row_hold_days = int(live_exit_policy.get("hold_days", hold_days))
        trail_multiplier = float(
            live_exit_policy.get("atr_multiplier", ATR_TRAIL_MULTIPLIER)
        )
    return _BookConfig(
        strategy=strategy,
        exit_policy=exit_policy,
        live_exit_mode=live_exit_mode,
        hold_days=row_hold_days,
        trail_multiplier=trail_multiplier,
        reentry_policy=str(strategy.get("reentry_policy", "immediate")),
        qualified_fn=(
            None if use_live
            else lambda c, config=entry_cfg: _qualifies_strategy_variant(c, config)
        ),
        ranking_key=str(
            strategy.get("ranking_key")
            or entry_cfg.get("ranking_key")
            or entry_cfg["percentile_key"]
        ),
        max_positions=int(entry_cfg["max_positions"]),
        risk_per_trade=float(entry_cfg["risk_per_trade"]),
    )
 def _production_book_config(
    hold_days: int, live_exit_policy: dict | None
 ) -> _BookConfig | None:
    strategy = next(
        (s for s in _portfolio_monitor_strategies() if s.get("is_production")), None
    )
    if strategy is None:
        return None
    return _strategy_book_config(strategy, hold_days, live_exit_policy)
 def _exit_policy_sims(
    candidates: list[dict],
    prices: dict[str, tuple],
@@ -3596,23 +3724,9 @@ def _min_rr_sweep(
    Graded on Sharpe/CAGR/DD under the real exit — not per-setup expectancy —
    because that is the metric every other promotion decision used.
    """
-    strategy = next((s for s in PORTFOLIO_MONITOR_STRATEGIES if s.get("is_production")), None)
+    book = _production_book_config(hold_days, live_exit_policy)
-    if strategy is None:
+    if book is None:
        return {}
    entry_cfg = _entry_variant_config(str(strategy["entry_variant"]))
    if entry_cfg is None:
        return {}
    exit_policy = str(strategy["exit_policy"])
    row_hold_days = hold_days
    trail_multiplier = ATR_TRAIL_MULTIPLIER
    reentry_policy = str(strategy.get("reentry_policy", "immediate"))
    if strategy.get("use_live_config") and live_exit_policy is not None:
        exit_policy = LIVE_EXIT_MODE_TO_SIM.get(
            str(live_exit_policy.get("mode", "atr_trailing")), "atr_trail3"
        )
        row_hold_days = int(live_exit_policy.get("hold_days", hold_days))
        trail_multiplier = float(live_exit_policy.get("atr_multiplier", ATR_TRAIL_MULTIPLIER))
    live_min_rr = float(activation.get("min_rr", 0.0))
    live_qualified = sum(1 for c in candidates if c.get("qualified"))
@@ -3639,26 +3753,16 @@ def _min_rr_sweep(
            candidates,
            prices,
            _spy_closes,
-            exit_policy,
+            book.exit_policy,
-            row_hold_days,
+            book.hold_days,
            qualified_fn=qualified_fn,
-            ranking_key=str(entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]),
+            ranking_key=book.ranking_key,
-            max_positions=int(entry_cfg["max_positions"]),
+            max_positions=book.max_positions,
-            risk_per_trade=float(entry_cfg["risk_per_trade"]),
+            risk_per_trade=book.risk_per_trade,
-            atr_trail_multiplier=trail_multiplier,
+            atr_trail_multiplier=book.trail_multiplier,
-            post_stop_reentry_fn=(
+            post_stop_reentry_fn=replace(
-                _make_gate_reset_reentry_fn(
+                book, qualified_fn=qualified_fn
-                    candidates,
+            ).post_stop_reentry_fn(candidates, prices, cadence),
                    prices,
                    cadence=cadence,
                    qualified_fn=qualified_fn,
                    ranking_key=str(
                        entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]
                    ),
                )
                if reentry_policy == "gate_reset"
                else None
            ),
            start_date=sweep_start,
        )
        if sim is None:
@@ -3682,8 +3786,8 @@ def _min_rr_sweep(
        "live_min_rr": live_min_rr,
        "live_qualified_setups": live_qualified,
        "reproduces_production_gate": reproduces,
-        "exit_policy": exit_policy,
+        "exit_policy": book.exit_policy,
-        "reentry_policy": reentry_policy,
+        "reentry_policy": book.reentry_policy,
        "entries_from": sweep_start.isoformat() if sweep_start else None,
        "window": "out-of-sample (test)" if sweep_start else "full history (in-sample)",
        "rows": rows,
@@ -3727,46 +3831,10 @@ def _holdout_evaluation(
    chosen on. This does the real thing: the test window is disjoint from the
    train window, so a rule settled on train has never seen it.
    """
-    strategy = next(
+    book = _production_book_config(hold_days, live_exit_policy)
-        (s for s in PORTFOLIO_MONITOR_STRATEGIES if s.get("is_production")),
+    if book is None:
        None,
    )
    if strategy is None:
        return {}
-    entry_cfg = _entry_variant_config(str(strategy["entry_variant"]))
+    post_stop_reentry_fn = book.post_stop_reentry_fn(candidates, prices, cadence)
    if entry_cfg is None:
        return {}
    exit_policy = str(strategy["exit_policy"])
    row_hold_days = hold_days
    trail_multiplier = ATR_TRAIL_MULTIPLIER
    reentry_policy = str(strategy.get("reentry_policy", "immediate"))
    if strategy.get("use_live_config") and live_exit_policy is not None:
        exit_policy = LIVE_EXIT_MODE_TO_SIM.get(
            str(live_exit_policy.get("mode", "atr_trailing")), "atr_trail3"
        )
        row_hold_days = int(live_exit_policy.get("hold_days", hold_days))
        trail_multiplier = float(
            live_exit_policy.get("atr_multiplier", ATR_TRAIL_MULTIPLIER)
        )
    qualified_fn = (
        None if strategy.get("use_live_config")
        else lambda c, config=entry_cfg: _qualifies_strategy_variant(c, config)
    )
    ranking_key = str(
        entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]
    )
    post_stop_reentry_fn = (
        _make_gate_reset_reentry_fn(
            candidates,
            prices,
            cadence=cadence,
            qualified_fn=qualified_fn,
            ranking_key=ranking_key,
        )
        if reentry_policy == "gate_reset"
        else None
    )
    rows: list[dict] = []
    for window, start, end in (
@@ -3777,13 +3845,13 @@ def _holdout_evaluation(
            candidates,
            prices,
            _spy_closes,
-            exit_policy,
+            book.exit_policy,
-            row_hold_days,
+            book.hold_days,
-            qualified_fn=qualified_fn,
+            qualified_fn=book.qualified_fn,
-            ranking_key=ranking_key,
+            ranking_key=book.ranking_key,
-            max_positions=int(entry_cfg["max_positions"]),
+            max_positions=book.max_positions,
-            risk_per_trade=float(entry_cfg["risk_per_trade"]),
+            risk_per_trade=book.risk_per_trade,
-            atr_trail_multiplier=trail_multiplier,
+            atr_trail_multiplier=book.trail_multiplier,
            post_stop_reentry_fn=post_stop_reentry_fn,
            start_date=start,
            end_date=end,
@@ -3791,12 +3859,12 @@ def _holdout_evaluation(
        )
        if sim is None:
            continue
-        rows.append({"window": window, "exit_policy": exit_policy, **sim})
+        rows.append({"window": window, "exit_policy": book.exit_policy, **sim})
    return {
        "split_date": split.isoformat(),
-        "strategy": strategy["strategy"],
+        "strategy": book.strategy["strategy"],
-        "reentry_policy": reentry_policy,
+        "reentry_policy": book.reentry_policy,
        "rows": rows,
        "note": (
            "Train = entries before the split; test = entries on/after it. The two "
@@ -3806,6 +3874,215 @@ def _holdout_evaluation(
    }
 # Pre-registered 2026-10-09 — docs/research/breakeven-stop.md. Two trigger levels,
 # no sweep; graded on both halves of a fixed split at the live book's capacity and risk.
 BREAKEVEN_STUDY_ARMS: tuple[tuple[str, float | None], ...] = (
    ("control", None),
    ("be_1r", 1.0),
    ("be_2r", 2.0),
 )
 # Added 2026-10-10 after the first run, so reported but never graded: be_1r with its
 # breakeven exits starting the post-stop gate reset, as an initial stop does. The
 # graded arms follow live, where only initial-stop exits lock re-entry; the gap
 # between the two shows how much of be_1r is same-session re-buys.
 BREAKEVEN_STUDY_DIAGNOSTIC_ARMS: tuple[tuple[str, float], ...] = (("be_1r_gated", 1.0),)
 BREAKEVEN_STUDY_SPLIT = date(2024, 7, 1)
 _BREAKEVEN_PARITY_KEYS = ("sharpe", "cagr_pct", "max_drawdown_pct", "trades")
 _BREAKEVEN_GRADED_KEYS = ("sharpe", "cagr_pct", "max_drawdown_pct")
 def _breakeven_study_enabled() -> bool:
    return os.getenv("BACKTEST_BREAKEVEN_STUDY", "").strip().lower() in {"1", "true", "yes", "on"}
 def _breakeven_checks(arm: dict, control: dict) -> dict[str, bool]:
    """The frozen pass rule for one window: no worse on Sharpe, CAGR or drawdown.
    A missing metric fails — an arm cannot pass on numbers it did not produce."""
    def ok(key: str, better) -> bool:
        a, c = arm.get(key), control.get(key)
        return a is not None and c is not None and better(a, c)
    return {
        "sharpe": ok("sharpe", lambda a, c: a >= c),
        "cagr": ok("cagr_pct", lambda a, c: a >= c),
        "max_drawdown": ok("max_drawdown_pct", lambda a, c: a <= c),
    }
 def _breakeven_exit_study(
    candidates: list[dict],
    prices: dict[str, tuple],
    _spy_closes: dict[date, float] | None,
    hold_days: int,
    live_exit_policy: dict | None = None,
    cadence: str = DEFAULT_BACKTEST_CADENCE,
    monitor_report: dict | None = None,
    live_book: dict | None = None,
 ) -> dict:
    """Production book with a breakeven stop added to the ATR trail, vs control.
    Mirrors ``_holdout_evaluation``'s production configuration and changes only
    ``breakeven_r`` and, for every graded row, the book's size: capacity and risk
    per trade from ``live_book`` (the shadow book's config, with where each value
    came from) when the caller could read it, else ``SIM_MAX_POSITIONS`` and the
    entry variant's risk. A control at the monitor's capacity and risk on full
    history must reproduce the monitor's production row, or the study is not
    grading the production book.
    """
    book = _production_book_config(hold_days, live_exit_policy)
    if book is None:
        return {}
    if book.exit_policy != "atr_trail3":
        return {
            "error": f"live exit maps to {book.exit_policy!r}; the study needs atr_trail3"
        }
    post_stop_reentry_fn = book.post_stop_reentry_fn(candidates, prices, cadence)
    def run(
        breakeven_r: float | None,
        max_positions: int,
        risk_per_trade: float,
        start: date | None = None,
        end: date | None = None,
        gate_reset: bool = False,
    ) -> dict | None:
        sim = _simulate_portfolio(
            candidates,
            prices,
            _spy_closes,
            book.exit_policy,
            book.hold_days,
            qualified_fn=book.qualified_fn,
            ranking_key=book.ranking_key,
            max_positions=max_positions,
            risk_per_trade=risk_per_trade,
            atr_trail_multiplier=book.trail_multiplier,
            post_stop_reentry_fn=post_stop_reentry_fn,
            start_date=start,
            end_date=end,
            include_trades=True,
            breakeven_r=breakeven_r,
            breakeven_gate_reset=gate_reset,
        )
        if sim is None:
            return None
        rs = [t["r"] for t in sim.pop("trade_details", [])]
        sim.pop("reentry_events", None)
        sim["avg_r"] = round(sum(rs) / len(rs), 3) if rs else None
        sim["trades_5r_plus"] = sum(1 for r in rs if r >= 5.0)
        return sim
    if live_book is not None:
        capacity, risk = int(live_book["capacity"]), float(live_book["risk_per_trade"])
        capacity_source, risk_source = live_book["capacity_source"], live_book["risk_source"]
    else:
        capacity, risk = SIM_MAX_POSITIONS, book.risk_per_trade
        capacity_source, risk_source = "SIM_MAX_POSITIONS", "entry variant"
    split = BREAKEVEN_STUDY_SPLIT
    windows = (("full", None, None), ("train", None, split), ("test", split, None))
    rows: list[dict] = []
    for arm, breakeven_r in BREAKEVEN_STUDY_ARMS:
        for window, start, end in windows:
            sim = run(breakeven_r, capacity, risk, start, end)
            if sim is not None:
                rows.append({"arm": arm, "window": window, **sim})
    diagnostic_rows: list[dict] = []
    # Gating needs a post-stop episode to start; a book without one has nothing
    # for the diagnostic to measure.
    diagnostic_arms = BREAKEVEN_STUDY_DIAGNOSTIC_ARMS if post_stop_reentry_fn else ()
    for arm, breakeven_r in diagnostic_arms:
        for window, start, end in windows:
            sim = run(breakeven_r, capacity, risk, start, end, gate_reset=True)
            if sim is not None:
                diagnostic_rows.append({"arm": arm, "window": window, **sim})
    parity_row = run(None, book.max_positions, book.risk_per_trade)
    monitor_row = next(
        (
            r for r in (monitor_report or {}).get("runs") or []
            if r.get("is_production") and r.get("lookback") == "all"
        ),
        None,
    )
    parity_ok = (
        parity_row is not None
        and monitor_row is not None
        and all(parity_row.get(k) == monitor_row.get(k) for k in _BREAKEVEN_PARITY_KEYS)
    )
    def row(arm: str, window: str) -> dict | None:
        return next((r for r in rows if r["arm"] == arm and r["window"] == window), None)
    def ties_control(arm: str, window: str) -> bool:
        treated, control = row(arm, window), row("control", window)
        return (
            treated is not None
            and control is not None
            and all(
                treated.get(k) is not None and treated.get(k) == control.get(k)
                for k in _BREAKEVEN_GRADED_KEYS
            )
        )
    verdicts: dict[str, dict] = {}
    for arm, _ in BREAKEVEN_STUDY_ARMS[1:]:
        verdict: dict[str, Any] = {}
        for window in ("train", "test"):
            treated, control = row(arm, window), row("control", window)
            verdict[window] = (
                _breakeven_checks(treated, control)
                if treated is not None and control is not None
                else {"sharpe": False, "cagr": False, "max_drawdown": False}
            )
        verdict["passes"] = parity_ok and all(
            all(verdict[w].values()) for w in ("train", "test")
        )
        # The frozen rule passes ties. An arm that matches control on every
        # graded metric in both windows has found nothing, even if a few of its
        # trades differ.
        verdict["ties_control_on_graded_metrics"] = all(
            ties_control(arm, w) for w in ("train", "test")
        )
        verdicts[arm] = verdict
    return {
        "protocol": "docs/research/breakeven-stop.md",
        "split_date": split.isoformat(),
        "capacity": capacity,
        "capacity_source": capacity_source,
        "risk_per_trade": risk,
        "risk_source": risk_source,
        "cadence": cadence,
        "exit_policy": book.exit_policy,
        "atr_trail_multiplier": book.trail_multiplier,
        "hold_days": book.hold_days,
        "reentry_policy": book.reentry_policy,
        "parity": {
            "ok": parity_ok,
            "capacity": book.max_positions,
            "risk_per_trade": book.risk_per_trade,
            "study_control": (
                {k: parity_row.get(k) for k in _BREAKEVEN_PARITY_KEYS} if parity_row else None
            ),
            "monitor_production_all": (
                {k: monitor_row.get(k) for k in _BREAKEVEN_PARITY_KEYS} if monitor_row else None
            ),
        },
        "rows": rows,
        "verdicts": verdicts,
        "diagnostic_rows": diagnostic_rows,
        "note": (
            "Graded on train and test only: an arm passes when Sharpe and CAGR are "
            "no lower and max drawdown no higher than control in both windows; "
            "ties_control_on_graded_metrics marks a pass that is only ties. Win rate and avg R "
            "are reported, not graded. diagnostic_rows were added after the first run "
            "and are never graded. A pass earns forward paper validation, not a "
            "production change."
        ),
    }
 def _portfolio_monitor(
    candidates: list[dict],
    prices: dict[str, tuple],
@@ -3818,58 +4095,23 @@ def _portfolio_monitor(
    rows: list[dict] = []
    strategies = _portfolio_monitor_strategies()
    for strategy in strategies:
-        entry_cfg = _entry_variant_config(str(strategy["entry_variant"]))
+        book = _strategy_book_config(strategy, hold_days, live_exit_policy)
-        if entry_cfg is None:
+        if book is None:
            continue
-        ranking_key = str(
+        post_stop_reentry_fn = book.post_stop_reentry_fn(candidates, prices, cadence)
            strategy.get("ranking_key")
            or entry_cfg.get("ranking_key")
            or entry_cfg["percentile_key"]
        )
        # Live-config rows replay the runtime qualification flag and Admin exit
        # policy. The overlay opts into this deliberately so only ordering
        # changes relative to the production row.
        use_live = bool(strategy.get("use_live_config"))
        reentry_policy = str(strategy.get("reentry_policy", "immediate"))
        exit_policy = str(strategy["exit_policy"])
        row_hold_days = hold_days
        trail_multiplier = ATR_TRAIL_MULTIPLIER
        live_exit_mode: str | None = None
        if use_live and live_exit_policy is not None:
            live_exit_mode = str(live_exit_policy.get("mode", "atr_trailing"))
            exit_policy = LIVE_EXIT_MODE_TO_SIM.get(live_exit_mode, "atr_trail3")
            row_hold_days = int(live_exit_policy.get("hold_days", hold_days))
            trail_multiplier = float(
                live_exit_policy.get("atr_multiplier", ATR_TRAIL_MULTIPLIER)
            )
        qualified_fn = (
            None if use_live
            else lambda c, config=entry_cfg: _qualifies_strategy_variant(c, config)
        )
        post_stop_reentry_fn = (
            _make_gate_reset_reentry_fn(
                candidates,
                prices,
                cadence=cadence,
                qualified_fn=qualified_fn,
                ranking_key=ranking_key,
            )
            if reentry_policy == "gate_reset"
            else None
        )
        for lookback in PORTFOLIO_MONITOR_LOOKBACKS:
            start = _lookback_start(latest_ord, lookback["days"])
            sim = _simulate_portfolio(
                candidates,
                prices,
                _spy_closes,
-                exit_policy,
+                book.exit_policy,
-                row_hold_days,
+                book.hold_days,
-                qualified_fn=qualified_fn,
+                qualified_fn=book.qualified_fn,
-                ranking_key=ranking_key,
+                ranking_key=book.ranking_key,
-                max_positions=int(entry_cfg["max_positions"]),
+                max_positions=book.max_positions,
-                risk_per_trade=float(entry_cfg["risk_per_trade"]),
+                risk_per_trade=book.risk_per_trade,
-                atr_trail_multiplier=trail_multiplier,
+                atr_trail_multiplier=book.trail_multiplier,
                post_stop_reentry_fn=post_stop_reentry_fn,
                start_date=start,
                include_curve=True,
@@ -3883,10 +4125,10 @@ def _portfolio_monitor(
                "is_production": bool(strategy.get("is_production")),
                "comparison_arm": strategy.get("comparison_arm"),
                "entry_variant": strategy["entry_variant"],
-                "ranking_key": ranking_key,
+                "ranking_key": book.ranking_key,
-                "exit_policy": exit_policy,
+                "exit_policy": book.exit_policy,
-                "live_exit_mode": live_exit_mode,
+                "live_exit_mode": book.live_exit_mode,
-                "reentry_policy": reentry_policy,
+                "reentry_policy": book.reentry_policy,
                "lookback": lookback["lookback"],
                "lookback_label": lookback["label"],
                **sim,
@@ -4508,6 +4750,7 @@ async def run_backtest(
    portfolio_monitor_report: dict | None = None
    holdout_report: dict | None = None
    min_rr_sweep_report: dict | None = None
    breakeven_study_report: dict | None = None
    if not _signal_eval_only():
        try:
            qual_symbols = sorted({
@@ -4572,6 +4815,44 @@ async def run_backtest(
                    candidates, price_columns, spy_closes, activation, current_min_pct,
                    hold_horizon, live_exit_policy=live_exit_policy, cadence=cadence,
                )
            if _breakeven_study_enabled():
                live_book: dict | None = None
                try:
                    from app.services import shadow_book_service as shadow_book
                    shadow_config = await shadow_book.get_config(db)
                    stored = await settings_store.get_map(
                        db, [shadow_book.KEY_CAPACITY, shadow_book.KEY_RISK_PCT]
                    )
                    def _source(key: str) -> str:
                        # get_config falls back to its default for a missing or
                        # unreadable row; say so rather than credit the setting.
                        try:
                            float(stored[key])
                        except (KeyError, TypeError, ValueError):
                            return "shadow_book default"
                        return key
                    live_book = {
                        "capacity": int(shadow_config["capacity"]),
                        "risk_per_trade": float(shadow_config["risk_pct"]) / 100.0,
                        "capacity_source": _source(shadow_book.KEY_CAPACITY),
                        "risk_source": _source(shadow_book.KEY_RISK_PCT),
                    }
                except Exception:
                    logger.exception(
                        "Shadow book config load failed; breakeven study uses "
                        "SIM_MAX_POSITIONS and the entry variant's risk"
                    )
                    await _rollback_quietly(db, "shadow book config load")
                breakeven_study_report = _breakeven_exit_study(
                    candidates, price_columns, spy_closes, hold_horizon,
                    live_exit_policy=live_exit_policy,
                    cadence=cadence,
                    monitor_report=portfolio_monitor_report,
                    live_book=live_book,
                )
        except Exception:
            logger.exception("Portfolio simulation failed")
            # Catches the price_columns fetch loop, which has no handler of its
@@ -4703,6 +4984,9 @@ async def run_backtest(
            "~6 months ≈ one market regime — treat as directional, not gospel."
        ),
    }
    if breakeven_study_report is not None:
        # Added only when opted in, so the default report keeps its exact shape.
        report["breakeven_stop_study"] = breakeven_study_report
    report["recommendation"] = _build_recommendation(report)
    report["research_recommendation"] = _build_research_recommendation(report)
    return report
@@ -5,6 +5,13 @@ stop re-litigating settled questions: **if a row says "rejected", the experiment
 was run and the data said no.** Detail lives in the linked docs and in
 `reports/*.json` (all committed).
 [Strategy research review (2026-09-08)](strategy-review-2026-09-08.md) compares
 the existing evidence with published research and specifies a proposed
 volatility-standardized residual-momentum experiment. The subsequent
 [normalization screen failed](residual-normalization-findings.md): IC 0.0128
 versus 0.0145 for matched control, below the frozen 0.03 floor. No portfolio
 replay or production change followed.
 **The one-line summary of the whole platform:** it is a **long-only
 cross-sectional momentum book** — buy the top quintile by beta-adjusted 12-1
 momentum, tilt toward higher volatility, hold ≤ 10 names, cut at 1.5× ATR, then
@@ -48,6 +55,8 @@ score, Structural S/R, the Gate Target Ladder, sentiment, fundamentals) is
 | 11 | **FIP path-smoothness** as tie-breaker/filter | Non-monotonic within the qualified set; thinning the entry stream costs more compounding than the tilt returns | **Rejected as a filter** — but see §4, it's the strongest raw signal we've measured | — |
 | 12 | **Fixed take-profit sweep** (R-multiples) | No interior optimum ever found — the best TP is "no TP" | **Rejected.** Momentum's edge lives in the right tail | `backtest_service.py:450` |
 | 13 | **Sector-residual 12-1** (`mom_12_1_sector_resid` / sector demean) as replacement for market residual | Short-window IC/A/B looked knife-edge green; deep repaired + **liquid-1500** retest: weeks 83, mild +IC **0.027** / t 1.69, **below iron bar 0.03** (FAIL). Demean already weaker | **Rejected / closed.** Keep production market residual. Do not resurrect without a new pre-registered protocol | [sector-residual-momentum.md](sector-residual-momentum.md) · `sector-resid-deep-20260719-113319.json` · history-depth supersession note |
 | 14 | **Volatility-standardized market-residual 12-1** | 73 non-overlapping windows: IC **0.01284** vs matched control **0.01446**, below frozen 0.03 floor. Paired delta -0.00162, 95% interval [-0.00496, +0.00156] | **Closed negative at signal screen.** No demonstrated improvement; difference is inconclusive. Conditional portfolio stage not run | [Findings](residual-normalization-findings.md) · [Protocol](residual-normalization.md) · `residual-normalization-20260908.json` |
 | 15 | **Breakeven stop** (stop to entry once the trade is +1R / +2R, on top of the 3× ATR trail) | `be_1r` fails the frozen rule: train worse on Sharpe, CAGR and drawdown, test better on return but worse on drawdown. `be_2r` changed 2 of 480 trades, because the trail already reaches entry near +2R. A post-hoc arm that gates re-entry after breakeven exits does better but still fails on drawdown | **Rejected / closed for this snapshot.** Keep the production exit. Reopen only with a new pre-registered test on unseen data | [breakeven-stop.md](breakeven-stop.md) · `backtest-20261009-breakeven-stop.json` |
 ---
@@ -0,0 +1,214 @@
 # Breakeven stop — pre-registered exit test
 **Date:** 2026-10-09
 **Question:** a trade ran roughly +10%, the 3× ATR trail still sat near entry,
 and the pullback closed it at a small loss. Would moving the stop to the entry
 price once a trade is up by k R turn trades like that into wins *without*
 costing the book its right tail?
 Every "lock in gains sooner" exit tested so far lost money: a trail of 2× ATR or
 tighter, the target take-profit (Sharpe 2.04 → 1.47), and every fixed
 take-profit level ([research log](README.md) #1, #12; method rules 3 and 5). The
 breakeven stop is the one member of that family never run. The expectation is a
 fail — just after it triggers it acts like a tighter trail — but it is an
 untested arm, not a settled one.
 ## Protocol (frozen before the run)
 **Data and book.** `backtest_snapshots/prod.sqlite`, daily entry cadence (the
 live scan is daily, and a rule that frees slots earlier needs daily entries to
 refill them). The production book exactly as the `portfolio_monitor` production
 row replays it — live activation gate, 80/20 residual/high-vol rank, post-stop
 gate reset, 1% risk, 20% notional cap, close fill, 0.1% cost per side — at
 **capacity 15**, the live setting (the monitor row still uses 10).
 **Arms.** Only the exit changes.
 | arm | rule |
 |---|---|
 | `control` | production: 1.5× ATR initial stop, 3× ATR trail on the highest close, 30-session max hold |
 | `be_1r` | control, plus: once the highest close reaches entry + 1R, the stop rises to the entry price |
 | `be_2r` | same at entry + 2R |
 R = entry − initial stop. The trigger reads closes, like the trail, and a raised
 stop is active from the next bar; a gap through it fills at the open. The stop
 only ever moves up — the trail keeps working above entry. No other trigger
 levels are run.
 **Windows.** Train = entries before 2024-07-01, test = entries on/after it, as
 disjoint books (`_simulate_portfolio` `end_date`/`start_date`). Full history is
 reported but not graded.
 **Pass rule.** An arm passes only if, in **both** train and test, all three hold
 versus `control` at report precision (Sharpe 2 dp, CAGR and drawdown 1 dp):
 - Sharpe ≥ control
 - CAGR ≥ control
 - max drawdown ≤ control
 Win rate and average R per trade are reported but **not** graded: a breakeven
 stop raises both mechanically, and it changes trade count, so EV per trade is
 the wrong lens ([capacity findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens)).
 **Parity guard.** The study also runs `control` at the monitor's capacity 10 on
 full history. It must equal the `portfolio_monitor` production `all` row
 (Sharpe, CAGR, drawdown, trades) or the study is measuring a different book and
 no row is valid.
 **What a pass earns.** The post-2024 window has informed earlier research, so it
 is not a pristine holdout. A pass earns a forward paper trial only — no
 production change. A fail closes the question for this snapshot.
 ## Reproducing
 `prod.sqlite` predates later ticker/trade columns, so `select(Ticker)` fails on
 it (see [survivorship-audit.md](survivorship-audit.md#caveats)). Run on a copy
 synced with `prep_survivorship_snapshot.sync_schema`, which only adds nullable
 columns and changes no data:
 ```bash
 cp backtest_snapshots/prod.sqlite /tmp/prod-synced.sqlite
 python -c "import sys; sys.path[:0] = ['.', 'scripts']; from pathlib import Path; \
 from prep_survivorship_snapshot import sync_schema; sync_schema(Path('/tmp/prod-synced.sqlite'))"
 BACKTEST_BREAKEVEN_STUDY=1 python scripts/run_backtest_snapshot.py \
    /tmp/prod-synced.sqlite --cadence daily --workers 7 --allow-spawn \
    --out reports/backtest-20261009-breakeven-stop.json
 ```
 The graded capacity and risk per trade come from the shadow book's config in the
 database the run reads (`shadow_book_capacity`, `shadow_book_risk_pct`).
 `prod.sqlite` has neither row, so the shadow book's defaults apply: 15
 positions and 1% risk. The report records each value with its source
 (`capacity_source`, `risk_source`), which reads `shadow_book default` here.
 ## Result — no change; `be_1r` fails, `be_2r` passes only by being the production exit
 Report: `reports/backtest-20261009-breakeven-stop.json` (daily cadence, 1,011,248
 candidates, 5,449 qualified; 505 of 506 symbols evaluated — RHM has no stored
 bars, nothing failed). **Parity held exactly:** the capacity-10 control reproduces
 the monitor's production row (Sharpe 1.84, CAGR 49.8%, DD 18.3%, 465 trades).
 The report was regenerated on 2026-10-10 after a code review
 ([corrections below](#code-review-corrections-2026-10-10)); every graded number
 is unchanged.
 Capacity 15, production book:
 | arm | window | Sharpe (SE) | CAGR | max DD | trades | win % | avg R | ≥5R trades | breakeven exits |
 |---|---|---|---|---|---|---|---|---|---|
 | control | train | 1.71 (0.68) | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | — |
 | control | test | 1.46 (0.71) | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | — |
 | control | full | 1.69 (0.50) | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | — |
 | `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 35.0 | 0.595 | 22 | 36 |
 | `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 34.4 | 0.455 | 10 | 46 |
 | `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 34.9 | 0.539 | 32 | 82 |
 | `be_2r` | train | 1.71 | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | 1 |
 | `be_2r` | test | 1.46 | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | 1 |
 | `be_2r` | full | 1.69 | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | 2 |
 **`be_1r`: FAIL.** Train is worse on all three graded metrics (Sharpe 1.71 → 1.67,
 CAGR 47.3% → 45.3%, DD 17.8% → 18.5%). Test is much better on return (Sharpe
 1.46 → 1.71, CAGR +8.8pp) but drawdown is still worse (16.9% → 18.1%). The
 halves pointing opposite ways is the same signature as the rejected max-hold
 arms: a regime interaction, not an edge. The verdict rests on the frozen rule;
 no significance test was run on the gaps. The per-arm Sharpe SEs (~0.7) do not
 bound them either: the arms share most of their trades, so the gap between two
 arms is far less noisy than either Sharpe on its own.
 82 of 502 trades ended at the breakeven stop, and the book took 22 more trades
 than control. Those were not freed slots: the count cap never bound in any
 capacity-15 row (0 refusals). An earlier exit releases the ticker itself (one
 position per ticker) and the cash it held. The simulator does not count entries
 dropped or shrunk for lack of cash, so the split between the two is not
 measured. Only 4 of the 82 exits were bought back in the same session.
 **`be_2r`: passes the rule on ties, and that pass means nothing.** It changed 2
 of 480 trades. The reason is arithmetic: the initial stop is 1.5× ATR = 1R, and
 the trail sits 3× ATR = 2R below the highest close. By the time a trade is +2R
 up, the trail has already reached entry, unless ATR has widened since entry.
 `be_2r` is therefore the production exit, with nothing to paper-trade.
 Win rate *fell* under `be_1r` (36.0 → 34.9): an exit at entry is a small loss
 after the 0.1%/side cost, not a win. Average R rose only because trade count and
 mix changed — the reason it was not graded.
 **Decision: keep the production exit. The breakeven question is closed for this
 snapshot.** The trade that prompted it (a roughly 10% run that ended below
 entry) was not identified, so its R is unknown. With ATR unchanged the trail
 only reaches entry at +2R; that trade either peaked below +2R of its own risk or
 saw ATR widen during the run. Either way, the rule that would have saved it
 (`be_1r`, if it reached +1R) costs the book about as much elsewhere: no reliable
 improvement in either direction.
 **Side observation, not graded (worth its own look):** on this single daily
 path, the capacity-15 control (Sharpe 1.69, CAGR 44.3%) is *below* the
 capacity-10 production row (1.84, 49.8%) at the same 18.3% drawdown. At capacity
 10 the count cap refused **510** entries; at 15 it refused none. Those two
 counts do not measure the same thing: at 15, cash plus the 20% notional cap is
 the binding limit, and the entries it drops or shrinks are not counted. What the
 path does show is that, here, the tighter 10-slot book did better. The 2026-08-05 capacity decision
 was made across 175 start-date paths (51 better, 2 worse), so one full-history
 path disagreeing does not overturn it, but a −5.5pp gap is large enough to
 deserve its own look.
 ## Code-review corrections (2026-10-10)
 A review of the study code found several problems. Fixing them changed no graded
 number. One of them prompted a diagnostic arm.
 **Re-entry after a breakeven exit.** The first run labeled a breakeven exit
 `trailing_stop`, because the stop sat above the initial stop. It now has its own
 label, `breakeven_stop`, and the count is exact rather than inferred from the
 stop price (82 / 36 / 46, as before). Either label means the same thing for
 re-entry: the live lock (`trade_policy.get_reentry_gate_locks`) applies only
 after an initial-stop exit. So under `be_1r` a name stopped at breakeven can be
 bought again without a gate reset. That is how the rule would behave live, so
 the graded arms keep it. A diagnostic arm, `be_1r_gated`, sends breakeven exits
 through the post-stop gate reset as an initial stop would. It was added after
 the first run, so it is reported and never graded.
 | arm | window | Sharpe | CAGR | max DD | trades | breakeven exits | same-session re-buys |
 |---|---|---|---|---|---|---|---|
 | control | train | 1.71 | 47.3% | 17.8% | 243 | — | — |
 | control | test | 1.46 | 34.3% | 16.9% | 249 | — | — |
 | control | full | 1.69 | 44.3% | 18.3% | 480 | — | — |
 | `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 36 | 1 |
 | `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 46 | 3 |
 | `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 82 | 4 |
 | `be_1r_gated` | train | 1.72 | 47.1% | 18.0% | 253 | 38 | 0 |
 | `be_1r_gated` | test | 1.78 | 45.0% | 18.1% | 249 | 46 | 0 |
 | `be_1r_gated` | full | 1.79 | 47.7% | 18.0% | 500 | 84 | 0 |
 The review suspected part of `be_1r`'s test-window lift came from same-session
 re-buys. It did not: there were only 4. Gating re-entry after breakeven exits
 did help `be_1r`, though: it improves Sharpe and CAGR in every window and
 drawdown in two of three. The gate changes *which* re-entries happen, not just
 whether a name is re-bought, and this is one post-hoc comparison on one path.
 The result is consistent with the post-stop re-entry finding, where a gate
 reset beat immediate re-entry ([post-stop-reentry.md](post-stop-reentry.md)),
 but it does not establish the mechanism.
 **The decision stands.** `be_1r_gated` would also fail the frozen rule:
 drawdown is worse than control in both windows (17.8% → 18.0% train,
 16.9% → 18.1% test), and train CAGR is 47.1% against 47.3%. It was also chosen
 after seeing the data, so it earns nothing here. If the breakeven question is
 reopened, this is the arm to pre-register. It needs data the study has not
 seen, and the live gate lock extended to breakeven exits, so that the backtest
 and live still agree.
 **Other fixes. None moves a graded number.**
 - A breakeven stop now also applies on a `next_open` fill bar. Before, it fired
  one bar late there. The trail and breakeven rules now share one function. The
  study uses close fills, so its rows are unaffected.
 - `breakeven_r` must be positive and finite, and is rejected for exit policies
  without an ATR trail. Before, a non-positive value moved the stop to entry on
  the first bar, and other policies ignored the setting silently.
 - Each verdict now records `ties_control_on_graded_metrics`, which is true for
  `be_2r`. A pass that is only ties can no longer read as a finding. The flag
  compares the graded metrics only: `be_2r` still changed 2 trades.
 - The graded capacity and risk per trade come from the shadow book's config
  instead of the `SIM_MAX_POSITIONS` constant and the entry variant's fixed 1%.
  On a snapshot that is the snapshot's copy of the settings, not the current
  live values, and a missing row is labeled as the default it fell back to
  ([Reproducing](#reproducing)). In this run both are defaults, equal to the
  values the first run used.
@@ -0,0 +1,80 @@
 # Residual normalization — findings
 **2026-09-08: closed negative at the signal screen. Keep production unchanged.**
 The proposed normalization did not meet the rule frozen in
 [the experiment protocol](residual-normalization.md), and its point estimates
 did not improve on the current momentum signal. No portfolio replay was run:
 the protocol requires a passing signal screen before that stage.
 The candidate used the production SPY-adjusted 12-1 return sum divided by sample
 standard deviation of the same 231 daily adjusted returns. All comparisons used
 identical as-of dates and 30-SPY-session forward-return endpoints. No new factor,
 lookback, volatility weight, sizing rule, or exit was tested.
 | Era | Non-overlapping windows | Current signal, matched coverage | Normalized signal | Difference |
 |---|---:|---:|---:|---:|
 | Full period | 73 | 0.01446 | 0.01284 | -0.00162 |
 | Before 2021 | 30 | 0.00776 | 0.00496 | -0.00280 |
 | From 2021 | 42 | 0.02377 | 0.02259 | -0.00118 |
 Values are mean **Spearman information coefficients (IC)**: how well signal
 ordering agrees with subsequent stock returns. They are neither Sharpe ratios
 nor trading returns. Higher is better for this positive-momentum hypothesis.
 The candidate needed full-period IC >= 0.03, at least 12 windows, and positive
 mean IC in both eras. It passed the count and era-sign conditions but failed the
 0.03 threshold. The rule was evaluated before rounding.
 The paired normalized-minus-current IC difference has a 95% moving-block interval
 of **[-0.00496, +0.00156]**. This spans zero: the study does not establish that
 normalization is worse, and it supplies no convincing improvement evidence.
 The mean top-minus-bottom-quintile 30-session return spread also fell, from
 **0.967% to 0.751%**. That is an unconditional diagnostic, not the P&L of the
 long-only, gated, capital-constrained production book.
 **Coverage and checks**
 - The snapshot completion manifest matches 4,650 registry tickers, 5,081,073
  OHLCV rows, and 4,144 research-only names. The experiment used the remaining
  506 production-universe registry names, averaging 485.4 paired stocks per
  retained window.
 - Actual retained as-of dates run from **2017-01-06 to 2026-05-01**; the final
  forward endpoint is **2026-06-15**. Later dates cannot necessarily add another
  complete non-overlapping 30-session window. SPY history starts in January
  2016, so a full one-year signal cannot start in July 2016.
 - One retained window crosses the 2021 boundary and belongs to the full-period
  result only, explaining why the era counts sum to 72 rather than 73.
 - The unrestricted production control had **35,441** retained stock/date cells;
  paired control and treatment each had **35,433**. Its mean IC was 0.01447,
  essentially the same as paired control. Coverage restriction did not account
  for the observed result.
 - Numerator parity was checked against the production implementation on each
  eligible evaluated cell. The snapshot and frozen source hashes were unchanged
  across the run. SQLite was opened read-only.
 - Twelve targeted tests passed, including numerical invariance, production
  parity, lookahead exclusion, zero-variance handling, missing-data pairing,
  calendar overlap, and an end-to-end screen check. Ruff passed for the runner
  and its tests. The existing Python runtime was usable outside the filesystem
  sandbox; no environment replacement was needed.
 - A 2026-10-10 code review added four tests (missing prices, file release,
  paired-coverage guard, source ids) with the runner fixes they cover. The
  report was regenerated with the fixed runner; its screen results are
  byte-identical to the original run.
 **Interpretation and decision**
 My proposed mechanism was plausible but unsupported by this experiment. Do not
 add normalization to production or search denominator variants on this history.
 This ends this specific experiment, not all possible residual-momentum research.
 The current signal also falls below 0.03 on this particular long-history screen.
 That cautions against treating its predictive evidence as universally strong;
 it does not by itself measure or invalidate the complete production book. The
 unconditional universe, true session spacing, dates, and return objective differ
 from earlier setup/portfolio studies. Historical constituents are still projected
 backward, this history has already been inspected, and the bootstrap intervals
 remain exploratory. No new CAGR, drawdown, cost-stress, or live result is claimed.
 **Artifacts:** [machine-readable results](../../reports/residual-normalization-20260908.json),
 [generated tables](../../reports/residual-normalization-20260908.md),
 [runner](../../scripts/run_residual_normalization.py), and
 [tests](../../tests/unit/test_residual_normalization.py).
@@ -0,0 +1,87 @@
 # Residual normalization experiment
 Protocol frozen 2026-09-08 before computing either signal's results. Implements
 the first candidate in [the strategy review](strategy-review-2026-09-08.md).
 This is offline research; it does not change production selection or settings.
 **Question:** does dividing the current SPY-adjusted 12-1 score by its own daily
 residual standard deviation improve predictive ordering and portfolio economics?
 **Signal:** use the existing 252-bar lookback, skip 21 bars, estimate SPY beta
 with centered covariance/variance, and retain the fitted intercept's contribution
 exactly as production does. The treatment divides the production sum by sample
 standard deviation (`ddof=1`) over the same 231 adjusted daily returns. No
 denominator clipping, alternate window, factor addition, or weight sweep.
 An exactly market-tracking series can leave machine-rounding residuals; sample
 standard deviation <= 64 machine epsilons times `max(1, max(abs(stock_returns)))`
 is treated as numerical zero, not divided into the signal.
 **Data:** the completed `backtest_snapshots/research.sqlite`, restricted to
 tickers absent from `research_rank_only` (506 production-universe registry names).
 Nominal as-of start 2016-07-01; end bounded by actual SPY history and availability
 of a 30-SPY-session forward endpoint. This snapshot starts SPY in January 2016,
 so a fully aligned one-year signal cannot start in July 2016. Report actual
 signal and forward-return dates. All history is previously inspected and
 survivorship-biased; none is labeled fresh out-of-sample.
 Require exactly 231 return pairs over consecutive benchmark sessions in the
 formation window, positive finite prices, and nonzero residual variance for
 both paired arms. A separate unrestricted production control retains production's
 minimum-100-pair behavior. Report the removed population. Forward returns use
 the same exact stock endpoint dates 30 SPY sessions apart, without filling absent
 prices, without subtracting SPY, and without changing the stored return basis.
 **Stage 1 — signal screen:** consider the last SPY session of each ISO week.
 Keep a date with at least 20 common eligible stocks only if its as-of session is
 at or after the preceding retained forward endpoint. Thus successive intervals
 share at most an endpoint, not a return, including around holidays and year ends.
 Use those dates for all three arms. Report per-window Spearman IC, top-minus-bottom
 quintile forward-return spread, cross-sectional coverage, and paired IC deltas.
 Era summaries use windows ending before 2021-01-01 and windows starting on/after
 2021-01-01; boundary-crossing windows remain in full-period results only.
 The screen passes only if the treatment has full-period mean IC >= 0.03, at
 least 12 non-overlapping full-period windows, and strictly positive mean IC in
 both eras. Mean IC and signs are checked before rounding. No additional paired
 delta threshold is silently added. Report uncertainty whether the screen passes
 or fails. For IC intervals use paired moving blocks of four retained windows,
 5,000 replicates, NumPy PCG64 seed 20260908. These are exploratory intervals;
 non-overlap alone does not establish statistical independence.
 **Stage 2 — conditional portfolio comparison:** run only if stage 1 passes.
 Replay daily, with unrestricted control, common-coverage control, and normalized
 treatment; rank the full available production universe once per ticker/date.
 Use unchanged 80/20 momentum/6-month-volatility ordering, top-20% momentum gate,
 GTL qualification, capacity 15, 1% fixed-fractional risk, 20% notional cap,
 cash constraints, 1.5 ATR initial stop, 3 ATR trail, 30-session hold, no
 take-profit, and independent gate-reset histories. Keep the volatility rank
 universe and its values identical between the paired arms. Use the snapshot's
 non-momentum configuration and assert it agrees with the documented production
 floors before running. Keep all non-qualified scans for gate-reset observations.
 Use close fills as the historical execution scenario and costs 10 and 20 bps per
 side. On a common calendar report paired returns, trade overlap, exposure,
 turnover, risk per fill, and winning/losing contributions. Advancement requires
 positive paired Sharpe improvement, no lower point-estimate CAGR, no deeper
 point-estimate max drawdown, and a positive lower 95% bootstrap bound for Sharpe
 improvement at both costs and both 21-/63-session block lengths. Use 5,000
 paired replicates and seed 20260908. Bootstrap the realized return streams;
 this does not model counterfactual changes in decisions along resampled prices.
 A failed screen ends this experiment. A portfolio pass would justify forward
 research, not automatic promotion. No historical variation or rule relaxation
 will be chosen after seeing this result. Point-in-time membership and actual
 execution evidence remain separate requirements for deployment.
 **Reproduce stage 1:**
 ```powershell
 .venv\Scripts\python.exe scripts/run_residual_normalization.py
 .venv\Scripts\python.exe -m pytest tests/unit/test_residual_normalization.py -q
 ```
 The runner reads SQLite in read-only mode, validates the manifest, records
 snapshot/protocol/code hashes and package versions, checks production-numerator
 parity on the evaluated cells, and verifies the snapshot hash again before
 writing completed results. Errors must not become skipped tickers or partial
 successful reports.
@@ -0,0 +1,80 @@
 # Strategy research review — 2026-09-08
 **Subsequent experiment:** the first candidate below was implemented and
 [failed the frozen signal screen](residual-normalization-findings.md) on
 2026-09-08. Its IC was 0.0128 versus 0.0145 for matched control, below the 0.03
 floor. No portfolio replay or production change followed. The original research
 rationale below is preserved as the proposal made before seeing that result.
 **Recommendation:** make volatility-standardized market-residual momentum the next small research experiment. Keep the current strategy as the control. Before treating any winner as deployable, establish its result on historical membership and a forward record with observable execution. A separate cross-asset trend strategy is the more substantial diversification candidate.
 **Status:** research synthesis and proposed experiment, not a new backtest or a promotion. Objective assumed: better net returns relative to drawdown. Historical figures below were checked against the saved JSON reports; production code and settings were not changed. No replacement has been demonstrated superior by this review.
 **What the existing evidence actually supports**
 The current code defaults are a long-only, top-quintile market-residual 12-1 momentum book, ordered by 80% momentum percentile and 20% six-month volatility percentile. Capacity is 15, with 1% account risk per entry subject to cash and notional limits, a 1.5 ATR initial stop, a 3 ATR trail, a 30-session maximum hold, and gate-reset re-entry. Capacity and near-close scheduling have already shipped; some introductory research text still describes capacity 10 or scheduling as open work. These are code defaults and documented implementation, not an inspection of live Admin settings. See [momentum implementation](../../app/services/momentum_service.py), [capacity findings](portfolio-capacity-bracket-findings.md), and [execution decision](execution-recovery.md).
 | Existing experiment | Recorded result | Implication |
 |---|---|---|
 | Production universe, entries from July 2022 | Sharpe 1.32; CAGR 31.8%; max drawdown 18.9% | Positive on that particular replay. |
 | Same package, production plus liquid breadth | Sharpe 0.14; CAGR -1.8%; max drawdown 55.3% | Expanding the tradable universe under the existing rules already failed. |
 | Production universe, entries from July 2016 | Sharpe 0.88; CAGR 16.7%; max drawdown 24.4% | Longer-history evidence is appreciably weaker. |
 | Deep history plus liquid breadth | Sharpe -0.06; CAGR -7.0%; max drawdown 73.9% | More history did not rescue breadth. |
 | Restore 85 removed constituents, September sensitivity study | Sharpe 2.09 → 1.57; CAGR 51.6% → 38.1% | Cross-sectional universe composition materially changes results. |
 | Close versus next-open execution, July study | Sharpe 1.77 → 1.20; CAGR 48.3% → 30.0% | Execution assumptions can dominate a signal improvement. |
 The first four rows come from one [universe/horizon matrix](../../reports/prod-book-universe-horizon-20260719-140737.json), using the then-current capacity of 10. They are not new measurements of today's capacity-15 book. The last two rows come from separate [survivorship](survivorship-audit.md) and [execution](../../reports/execution_recovery_matrix.json) studies. Do not pool these as one baseline: cadence, history, universe, configuration, and end dates differ. Today's constituents projected backward remain a limitation of the deep-history study. Restoring removals is also not point-in-time membership, and the 13.5 percentage-point CAGR difference is not an estimate of survivorship bias. The sensitivity study attributes the portfolio change to universe recomposition, not positions in the failed banks.
 **First candidate: standardize the existing residual signal**
 The code estimates a stock's beta to SPY over the 12-1 formation window and sums daily stock returns minus beta times SPY returns. It deliberately does not subtract a fitted intercept. It also does not divide this score by residual volatility. That is a legitimate practical approximation; the name alone does not make it a replication of the academic strategy.
 Blitz, Huij and Martens estimate market, size, and value exposures using 36 monthly observations, then standardize the 12-1 residual return by its volatility. Their main portfolios are monthly long-short decile portfolios. They explicitly omit the estimated alpha from the momentum calculation. Their reported improvement therefore does not establish an improvement for this concentrated long-only ATR strategy. [Residual Momentum, accepted manuscript, pp. 10–12](https://repub.eur.nl/pub/22252/ResidualMomentum-2011.pdf)
 My proposed first experiment isolates standardization using the existing SPY model:
 ```text
 u[d] = stock_return[d] - beta * SPY_return[d]
 control_score = sum(u[d])
 candidate_score = control_score / sample_standard_deviation(u[d])
 ```
 Use the same 231 daily return pairs from the existing 252-session lookback with the final 21 sessions skipped. Require a complete, aligned window and positive residual standard deviation in both arms for the paired signal comparison. Retain a separate unrestricted control to quantify any coverage effect. Do not add a fitted-intercept subtraction, new lookback, or new volatility weight.
 **Hypothesis, not finding:** measuring a stock-specific rise relative to its own noise may improve ordering and reduce sensitivity to unusually volatile names. This changes the signal itself; it is distinct from the previously rejected inverse-volatility position sizing or standalone low-volatility ranking. Nevertheless, those negative results and the strategy's dependence on a few large winners lower confidence: standardization could also remove valuable winners. The research search found no recorded direct test of this exact normalization in the current scripts and research documents.
 Keep the existing 80/20 blend in the first portfolio comparison, replacing only its momentum input and the momentum gate input. Removing the high-volatility tilt simultaneously would obscure the mechanism. If this narrow test fails, record the failure without searching over denominator windows, powers, or clipping rules on the same history.
 **Second candidate: a fuller factor-residual model**
 A later, separately specified candidate could remove market, size, and value exposures with a longer estimation window. This addresses factor contamination that single-market beta adjustment leaves behind. It is distinct from the rejected sector-residual experiment, but that rejection still argues for modest expectations. Model estimation, monthly versus daily updates, coverage, and the existing daily gate-reset behavior would all need explicit treatment.
 The factor data are available from the [Kenneth French Data Library](https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html), which states that historical returns are reconstructed during updates and can change. Freeze the source vintage and record when factors were actually available. A factor observation dated month-end is not automatically usable for a trade at that same close. Current downloadable history alone does not establish historical publication availability. A monthly paper replication and a daily deployable adaptation must be labeled separately.
 Prioritize the simpler normalization experiment because it isolates one mechanism without adding a factor-data dependency. Neither candidate fixes historical universe membership or corporate-action errors. The existing [total-return A/B](tr-total-return-ab.md) remains unresolved; silently changing dividend treatment in the same comparison would introduce another cause for any difference.
 **Larger alternative: an independent cross-asset trend allocation**
 Research a separate portfolio of liquid equity-index, bond, commodity, and currency exposures, taking positions according to each market's own past return. The purpose is to diversify the source of portfolio returns beyond stock selection. This would be a new strategy and data pipeline, with its own capital allocation and execution model.
 Hurst, Ooi and Pedersen study time-series momentum across futures and forwards and extend the historical examination back to 1880. Their results motivate investigating the strategy family; they do not supply an expected return or optimal allocation for this platform. [A Century of Evidence on Trend-Following Investing](https://www.aqr.com/insights/research/journal-article/a-century-of-evidence-on-trend-following-investing)
 A long-only ETF/cash version may be easier to operate, but cannot reproduce the short positions, leverage, or historical instrument coverage of the published strategy. Start with one fixed specification and evaluate both its standalone economics and its contribution to a fixed combined portfolio after costs. Diversification must be measured, especially during equity losses; shared use of the word momentum does not determine correlation. This is a larger effort than the first candidate and remains untested here.
 **What to leave closed**
 The local record does not support another round of take-profit exits, SPY moving-average entry bans, gap-up caps, high-volatility breadth expansion, earnings-surprise gates, fundamental score weights, or ATR/hold-period sweeps on the same snapshots. See the [experiment index](README.md), [earnings findings](earnings-gap-and-sue.md), and [fundamental-weight findings](fundamentals-weight-backtest.md). Repeating them would add selection opportunities rather than independent evidence.
 Crash-management papers also need careful translation. Daniel, Jagannathan and Kim identify a short-side mechanism in long-short momentum crashes. That is not direct evidence for adding an equity-regime veto to this long-only strategy. [Tail Risk in Momentum Strategy Returns](https://www.nber.org/papers/w18169)
 **Proposed evaluation contract for the first candidate**
 1. **Freeze the inputs and trial first.** Record snapshot and code hashes, symbol/date eligibility, cash treatment, costs, signal formula, and report boundaries. Verify the research snapshot completion manifest against actual counts. Treat the previously inspected 2016–2026 history as exploratory evidence, including periods historically labeled validation or test.
 2. **Screen the signal on identical cells.** Compare the candidate and current score against the same 30-session forward returns. Select actual non-overlapping session intervals; report window count, mean Spearman IC, uncertainty, quintile spreads, and pre-/post-2021 signs. The existing IC >= 0.03, consistent-sign, >= 12-window rule is a screening requirement. The harness's `reliable` flag counts windows; it is not a statistical significance result. Measure paired IC differences as well as each signal's absolute IC.
 3. **Replay the whole policy if the screen passes.** Use daily scans, capacity 15, unchanged cash/notional limits, stops, trail, hold, costs, and 80/20 weights. Recompute qualifications and gate-reset histories separately for each arm. Reusing the incumbent's qualified candidates or lock histories would miss decisions caused by the treatment.
 4. **Grade portfolio economics.** For the default objective, propose a conservative advancement rule: positive paired Sharpe improvement, no lower point-estimate CAGR, and no deeper point-estimate maximum drawdown, with a paired 95% interval for Sharpe improvement above zero. Report 21- and 63-session block-bootstrap intervals, with seed and replicate count frozen before execution. Require the conclusion to survive both block lengths and a doubled-cost stress from 10 to 20 bps per side. These are proposed research criteria, not previously passed tests. Failure or inadequate precision means no promotion; do not relax the rule after seeing results.
 5. **Check why the book changed.** Report turnover, exposure, entry overlap, risk per fill, best/worst contributors, and foregone winners. Pair daily portfolio returns on a common calendar. Overlapping start-date paths are sensitivity checks, not independent observations. Resampling realized returns does not simulate how different price histories would change future decisions.
 6. **Establish deployability separately.** Use dated constituent membership, symbol identity and delisting treatment, and a documented price/return basis. For execution, retain the actual scan inputs, order timestamps, decision prices, and fills. The current shadow book records near-close setup prices; those are simulated entries, not broker execution evidence. Daily bars cannot reconstruct the exact partial bar that a near-close scan saw. Historical close/stale-close results are scenarios, not a guaranteed bracket for future live Sharpe.
 7. **Reserve prospective evidence.** Choose a fixed forward evaluation schedule before collecting the comparison, with separate paper books. Require enough calendar history to assess uncertainty; a small trade count or an early favorable month is not a promotion criterion. Maintain the full experiment ledger. Deflated Sharpe can address multiple selection and non-normality when the relevant trial information is available, but cannot repair lookahead or invent untouched data. [Bailey and López de Prado, The Deflated Sharpe Ratio](https://www.davidhbailey.com/dhbpapers/deflated-sharpe.pdf)
 **Deliverable and limits:** this review supplies a prioritized candidate and a concrete evaluation design. It verifies existing evidence rather than reporting a newly discovered edge. No new replay was executed; the checked-in virtual environment points to an unavailable Python interpreter in this session. The research recommendation does not depend on treating that missing run as a result.
@@ -1,5 +1,20 @@
 # Backtest report index
 The 2026-09-08 residual-normalization signal experiment is preserved in
 `residual-normalization-20260908.json` and its generated Markdown table.
 The JSON includes every retained window, coverage, paired uncertainty, the
 mechanical verdict, the snapshot hash, and the git blob ids of the source it
 ran. The screen failed its frozen IC floor, so no portfolio replay was run. Interpretation:
 [`residual-normalization-findings.md`](../docs/research/residual-normalization-findings.md).
 The 2026-10-09 breakeven-stop exit test is preserved in
 `backtest-20261009-breakeven-stop.json` (regenerated 2026-10-10 after a code
 review, with no graded number changed). Its `breakeven_stop_study` section holds
 the parity guard, the graded rows and verdicts for `control`, `be_1r` and
 `be_2r`, and the ungraded `be_1r_gated` diagnostic rows. No arm passed with a
 real difference from control, so the production exit is unchanged.
 Interpretation: [`breakeven-stop.md`](../docs/research/breakeven-stop.md).
 Reports dated 2026-07-11 or earlier are the historical production research
 record and remain untouched.
@@ -0,0 +1,35 @@
 # Residual normalization — signal screen
 Generated: 2026-10-10T07:53:06.165467+00:00
 **Verdict: FAIL**
 Previously inspected historical data; no production change or new out-of-sample claim.
 | Era | Arm | Windows | Avg stocks | Mean IC | IC 95% interval | Quintile spread |
 |---|---|---:|---:|---:|---|---:|
 | full | unrestricted_control | 73 | 485.5 | 0.0145 | [-0.0257, 0.0450] | 0.97% |
 | full | paired_control | 73 | 485.4 | 0.0145 | [-0.0257, 0.0450] | 0.97% |
 | full | normalized | 73 | 485.4 | 0.0128 | [-0.0271, 0.0442] | 0.75% |
 | pre_2021 | unrestricted_control | 30 | 472.0 | 0.0078 | [-0.0338, 0.0682] | 0.76% |
 | pre_2021 | paired_control | 30 | 471.7 | 0.0078 | [-0.0338, 0.0683] | 0.75% |
 | pre_2021 | normalized | 30 | 471.7 | 0.0050 | [-0.0395, 0.0661] | 0.46% |
 | post_2021 | unrestricted_control | 42 | 495.2 | 0.0238 | [-0.0125, 0.0630] | 1.26% |
 | post_2021 | paired_control | 42 | 495.2 | 0.0238 | [-0.0125, 0.0630] | 1.26% |
 | post_2021 | normalized | 42 | 495.2 | 0.0226 | [-0.0147, 0.0631] | 1.06% |
 Paired normalized-minus-control IC differences:
 - full: -0.0016; 95% interval [-0.004960466538844113, 0.0015584142071432597].
 - pre_2021: -0.0028; 95% interval [-0.008151700038659202, 0.001515881215178447].
 - post_2021: -0.0012; 95% interval [-0.005965195533675296, 0.003076462018338155].
 Screen checks: `{"mean_ic_ge_0_03": false, "at_least_12_windows": true, "pre_2021_positive": true, "post_2021_positive": true}`
 Portfolio stage: **not_run_failed_signal_screen**.
 Signal IC and quintile spreads are predictive diagnostics, not simulated strategy returns.
 Intervals resample paired blocks of four non-overlapping forward windows; they remain exploratory.
 The unrestricted control shows the effect of requiring complete formation history.
 [Frozen protocol](../docs/research/residual-normalization.md)
@@ -0,0 +1,360 @@
 """Frozen, paired residual-volatility normalization screen. Offline only.
 See docs/research/residual-normalization.md. A failed screen terminates the
 experiment; a passing screen allows the separately specified daily book replay.
 """
 from __future__ import annotations
 import argparse
 import contextlib
 import hashlib
 import importlib.metadata
 import json
 import math
 import sqlite3
 import statistics
 import sys
 from collections import Counter
 from datetime import date, datetime, timezone
 from pathlib import Path
 import numpy as np
 ROOT = Path(__file__).resolve().parents[1]
 if str(ROOT) not in sys.path:
    sys.path.insert(0, str(ROOT))
 from app.services.backtest_service import _quintile_spread, _spearman  # noqa: E402
 from app.services.momentum_service import compute_residual_12_1_momentum  # noqa: E402
 START = date(2016, 7, 1)
 ERA_SPLIT = date(2021, 1, 1)
 LOOKBACK = 252
 SKIP = 21
 PAIRS = LOOKBACK - SKIP
 HORIZON = 30
 MIN_CROSS = 20
 MIN_WINDOWS = 12
 IC_FLOOR = 0.03
 SEED = 20260908
 REPLICATES = 5000
 IC_BLOCK = 4
 ARMS = ("unrestricted_control", "paired_control", "normalized")
 PROTOCOL = ROOT / "docs/research/residual-normalization.md"
 def sha256(path: Path) -> str:
    with path.open("rb") as source:
        return hashlib.file_digest(source, "sha256").hexdigest()
 def git_blob_id(path: Path) -> str:
    """The id ``git hash-object`` gives this file. Line endings are normalized
    first, as git does on commit, so a CRLF checkout and an LF checkout of the
    same source record the same id — one that can be looked up in history."""
    data = path.read_bytes().replace(b"\r\n", b"\n")
    return hashlib.sha1(b"blob %d\0" % len(data) + data).hexdigest()
 def read_snapshot(path: Path) -> tuple[dict, dict, dict]:
    """Fail closed on incomplete data; never open a live application database."""
    path = path.resolve(strict=True)
    manifest = json.loads(Path(str(path) + ".manifest.json").read_text(encoding="utf-8"))
    if manifest.get("complete") is not True:
        raise ValueError("Snapshot completion manifest is not complete")
    if any(Path(str(path) + suffix).exists() for suffix in ("-wal", "-journal")):
        raise ValueError("Snapshot has a journal; use a completed, quiescent copy")
    # sqlite3's own context manager only ends the transaction; closing releases
    # the file handle, which on Windows otherwise pins the snapshot for the run.
    with contextlib.closing(sqlite3.connect(path.as_uri() + "?mode=ro", uri=True)) as con:
        con.execute("PRAGMA query_only=ON")
        counts = {
            key: con.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]
            for key, table in (
                ("ticker_count", "tickers"),
                ("ohlcv_row_count", "ohlcv_records"),
                ("rank_only_count", "research_rank_only"),
            )
        }
        if any(manifest.get(k) != v for k, v in counts.items()):
            raise ValueError(f"Snapshot completion counts do not match: {counts}")
        tickers = con.execute(
            "SELECT id, symbol FROM tickers WHERE symbol NOT IN "
            "(SELECT symbol FROM research_rank_only) ORDER BY symbol"
        ).fetchall()
        benchmark = {
            date.fromisoformat(d[:10]): float(c)
            for d, c in con.execute(
                "SELECT date, close FROM benchmark_prices WHERE symbol='SPY' ORDER BY date"
            )
        }
        if not benchmark or any(not math.isfinite(c) or c <= 0 for c in benchmark.values()):
            raise ValueError("Missing or invalid SPY history")
        prices = {}
        for ticker_id, symbol in tickers:
            rows = con.execute(
                "SELECT date, open, high, low, close, volume FROM ohlcv_records "
                "WHERE ticker_id=? ORDER BY date", (ticker_id,),
            ).fetchall()
            dates = [date.fromisoformat(r[0][:10]) for r in rows]
            if len(dates) != len(set(dates)):
                raise ValueError(f"Duplicate stock dates: {symbol}")
            # A missing price must stop the run: read as 0 it would put a -100%
            # return into the unrestricted control while the paired arms drop it.
            if any(r[k] is None for r in rows for k in range(1, 5)):
                raise ValueError(f"Missing OHLC price: {symbol}")
            columns = (
                ([d.toordinal() for d in dates],)
                + tuple([float(r[k]) for r in rows] for k in range(1, 5))
                + ([float(r[5] or 0) for r in rows],)
            )
            prices[symbol] = columns
    return prices, benchmark, {**manifest, "verified_counts": counts}
 def signal_at(
    dates: list[date], closes: list[float], i: int,
    benchmark: dict[date, float], benchmark_index: dict[date, int],
 ) -> dict:
    """Production numerator, paired with a strictly aligned sample denominator."""
    empty = {"unrestricted_control": None, "paired_control": None,
             "normalized": None, "residual_sd": None, "reason": "short_history"}
    if i < LOOKBACK:
        return empty
    window_dates = dates[i - LOOKBACK:i + 1]
    window_closes = closes[i - LOOKBACK:i + 1]
    raw = compute_residual_12_1_momentum(window_dates, window_closes, benchmark)
    if raw is not None and not math.isfinite(raw):
        raise ValueError("Non-finite production signal")
    empty["unrestricted_control"] = raw
    formation_dates = window_dates[:PAIRS + 1]
    formation_closes = window_closes[:PAIRS + 1]
    indices = [benchmark_index.get(d) for d in formation_dates]
    if any(x is None for x in indices) or any(
        b != a + 1 for a, b in zip(indices, indices[1:])
    ):
        return {**empty, "reason": "unaligned_formation"}
    if any(not math.isfinite(c) or c <= 0 for c in formation_closes):
        return {**empty, "reason": "invalid_formation_price"}
    market = [benchmark[d] for d in formation_dates]
    stock_returns = [b / a - 1 for a, b in zip(formation_closes, formation_closes[1:])]
    market_returns = [b / a - 1 for a, b in zip(market, market[1:])]
    mean_s, mean_m = statistics.fmean(stock_returns), statistics.fmean(market_returns)
    var_m = sum((r - mean_m) ** 2 for r in market_returns)
    if var_m <= 0 or raw is None:
        return {**empty, "reason": "unavailable_beta"}
    beta = sum((s - mean_s) * (m - mean_m)
               for s, m in zip(stock_returns, market_returns)) / var_m
    residuals = [s - beta * m for s, m in zip(stock_returns, market_returns)]
    # Independent arithmetic verifies that the denominator uses the numerator's
    # exact return window. It must never silently redefine production momentum.
    if not math.isclose(sum(residuals), raw, rel_tol=1e-10, abs_tol=1e-12):
        raise AssertionError("Production numerator parity failed")
    sd = statistics.stdev(residuals)
    # Treat machine-precision residuals of a perfect market tracker as zero,
    # not as infinite signal strength. This is a numerical degeneracy check,
    # not a tunable volatility floor.
    numerical_zero = 64 * np.finfo(float).eps * max(1.0, max(map(abs, stock_returns)))
    if sd <= numerical_zero:
        return {**empty, "reason": "zero_residual_variance"}
    normalized = raw / sd
    if not math.isfinite(normalized):
        raise ValueError("Non-finite normalized signal")
    return {"unrestricted_control": raw, "paired_control": raw,
            "normalized": normalized, "residual_sd": sd, "reason": "eligible"}
 def weekly_indices(calendar: list[date]) -> list[int]:
    last = {}
    for i, day in enumerate(calendar):
        last[day.isocalendar()[:2]] = i
    return sorted(i for i in last.values() if calendar[i] >= START and i + HORIZON < len(calendar))
 def nonoverlapping_indices(indices: list[int], horizon: int = HORIZON) -> list[int]:
    kept = []
    for i in sorted(indices):
        if not kept or i >= kept[-1] + horizon:
            kept.append(i)
    return kept
 def interval(values) -> list[float] | None:
    if len(values) == 0:
        return None
    return [float(v) for v in np.quantile(values, [0.025, 0.975])]
 def bootstrap_means(matrix: np.ndarray) -> dict:
    """Shared moving-block indices preserve both the dates and arm pairing."""
    n = len(matrix)
    if n < IC_BLOCK:
        return {"reason": "fewer_windows_than_block", "arms": {}, "paired_delta_ci95": None}
    rng = np.random.default_rng(SEED)
    starts = rng.integers(0, n - IC_BLOCK + 1, (REPLICATES, math.ceil(n / IC_BLOCK)))
    indices = (starts[:, :, None] + np.arange(IC_BLOCK)).reshape(REPLICATES, -1)[:, :n]
    means = matrix[indices].mean(axis=1)
    return {
        "arms": {arm: interval(means[:, i]) for i, arm in enumerate(ARMS)},
        "paired_delta_ci95": interval(means[:, 2] - means[:, 1]),
    }
 def summarize(windows: list[dict]) -> dict:
    if not windows:
        return {"windows": 0, "arms": {}, "paired_mean_ic_delta": None}
    matrix = np.array([[w["arms"][arm]["ic"] for arm in ARMS] for w in windows])
    uncertainty = bootstrap_means(matrix)
    arms = {}
    for i, arm in enumerate(ARMS):
        values = matrix[:, i]
        se = float(values.std(ddof=1) / math.sqrt(len(values))) if len(values) > 1 else None
        arms[arm] = {
            "mean_ic": float(values.mean()),
            "ic_t_stat_descriptive": float(values.mean() / se) if se and se > 0 else None,
            "ic_ci95": uncertainty["arms"].get(arm),
            "positive_window_fraction": float((values > 0).mean()),
            "avg_cross_section": statistics.fmean(w["arms"][arm]["n"] for w in windows),
            "mean_quintile_spread": statistics.fmean(
                w["arms"][arm]["quintile_spread"] for w in windows
            ),
            "reliable_window_count": len(windows) >= MIN_WINDOWS,
        }
    return {"windows": len(windows), "first_asof": windows[0]["asof"],
            "last_asof": windows[-1]["asof"], "last_forward_end": windows[-1]["forward_end"],
            "arms": arms, "paired_mean_ic_delta": float((matrix[:, 2] - matrix[:, 1]).mean()),
            "paired_ic_delta_ci95": uncertainty["paired_delta_ci95"]}
 def screen_pass(summaries: dict) -> dict:
    full = summaries["full"]
    means = {era: summary.get("arms", {}).get("normalized", {}).get("mean_ic")
             for era, summary in summaries.items()}
    checks = {
        "mean_ic_ge_0_03": means["full"] is not None and means["full"] >= IC_FLOOR,
        "at_least_12_windows": full["windows"] >= MIN_WINDOWS,
        "pre_2021_positive": means["pre_2021"] is not None and means["pre_2021"] > 0,
        "post_2021_positive": means["post_2021"] is not None and means["post_2021"] > 0,
    }
    return {"passed": all(checks.values()), "checks": checks,
            "failed_checks": [k for k, v in checks.items() if not v]}
 def run_screen(prices: dict, benchmark: dict) -> dict:
    calendar = sorted(benchmark)
    bench_index = {d: i for i, d in enumerate(calendar)}
    weekly = weekly_indices(calendar)
    observations = {i: [] for i in weekly}
    reasons = Counter()
    for count, (symbol, columns) in enumerate(sorted(prices.items()), 1):
        dates = [date.fromordinal(int(o)) for o in columns[0]]
        closes = columns[4]
        index = {d: i for i, d in enumerate(dates)}
        for asof in weekly:
            i, forward = index.get(calendar[asof]), index.get(calendar[asof + HORIZON])
            if i is None or forward is None:
                reasons["missing_forward_or_asof"] += 1
                continue
            if any(not math.isfinite(closes[j]) or closes[j] <= 0 for j in (i, forward)):
                reasons["invalid_forward_or_asof"] += 1
                continue
            signal = signal_at(dates, closes, i, benchmark, bench_index)
            reasons[signal["reason"]] += 1
            observations[asof].append({"symbol": symbol, **signal,
                                       "forward": closes[forward] / closes[i] - 1})
        if count % 50 == 0 or count == len(prices):
            print(f"signal screen: {count}/{len(prices)} symbols", flush=True)
    usable = [i for i in weekly if sum(o["normalized"] is not None for o in observations[i]) >= MIN_CROSS]
    kept = nonoverlapping_indices(usable)
    windows = []
    for i in kept:
        rows = observations[i]
        arms = {}
        for arm in ARMS:
            pairs = [(o[arm], o["forward"]) for o in rows if o[arm] is not None]
            ic = _spearman([p[0] for p in pairs], [p[1] for p in pairs])
            if ic is None:
                raise ValueError(f"Degenerate {arm} cross-section on {calendar[i]}")
            arms[arm] = {"n": len(pairs), "ic": ic, "quintile_spread": _quintile_spread(pairs)}
        # Raised, not asserted: python -O must not strip the pairing guarantee.
        if arms["paired_control"]["n"] != arms["normalized"]["n"]:
            raise AssertionError(f"Paired arms scored different stocks on {calendar[i]}")
        windows.append({"asof": calendar[i].isoformat(),
                        "forward_end": calendar[i + HORIZON].isoformat(), "arms": arms})
    summaries = {
        "full": summarize(windows),
        "pre_2021": summarize([w for w in windows if w["forward_end"] < ERA_SPLIT.isoformat()]),
        "post_2021": summarize([w for w in windows if w["asof"] >= ERA_SPLIT.isoformat()]),
    }
    return {"summaries": summaries, "verdict": screen_pass(summaries), "windows": windows,
            "coverage": {"registry_symbols": len(prices), "weekly_asofs_considered": len(weekly),
                         "usable_weekly_asofs": len(usable), "all_weekly_cell_reasons": dict(reasons),
                         "retained_unrestricted_cells": sum(w["arms"][ARMS[0]]["n"] for w in windows),
                         "retained_paired_cells": sum(w["arms"][ARMS[1]]["n"] for w in windows)},
            "bootstrap": {"seed": SEED, "replicates": REPLICATES, "window_block": IC_BLOCK,
                          "generator": "NumPy PCG64", "interval": "percentile 95%"}}
 def render_report(report: dict) -> str:
    screen = report["screen"]
    lines = ["# Residual normalization — signal screen", "",
             f"Generated: {report['generated_at']}", "",
             f"**Verdict: {'PASS' if screen['verdict']['passed'] else 'FAIL'}**", "",
             "Previously inspected historical data; no production change or new out-of-sample claim.", "",
             "| Era | Arm | Windows | Avg stocks | Mean IC | IC 95% interval | Quintile spread |",
             "|---|---|---:|---:|---:|---|---:|"]
    for era, summary in screen["summaries"].items():
        for arm, row in summary["arms"].items():
            ci = row["ic_ci95"]
            ci_text = f"[{ci[0]:.4f}, {ci[1]:.4f}]" if ci else "unavailable"
            lines.append(f"| {era} | {arm} | {summary['windows']} | {row['avg_cross_section']:.1f} "
                         f"| {row['mean_ic']:.4f} | {ci_text} | {100 * row['mean_quintile_spread']:.2f}% |")
    lines.extend(["", "Paired normalized-minus-control IC differences:", ""])
    for era, summary in screen["summaries"].items():
        if summary["windows"]:
            lines.append(f"- {era}: {summary['paired_mean_ic_delta']:.4f}; "
                         f"95% interval {summary['paired_ic_delta_ci95']}.")
    lines.extend(["", f"Screen checks: `{json.dumps(screen['verdict']['checks'])}`", "",
                  f"Portfolio stage: **{report['portfolio_stage']}**.", "",
                  "Signal IC and quintile spreads are predictive diagnostics, not simulated strategy returns.",
                  "Intervals resample paired blocks of four non-overlapping forward windows; they remain exploratory.",
                  "The unrestricted control shows the effect of requiring complete formation history.", "",
                  "[Frozen protocol](../docs/research/residual-normalization.md)", ""])
    return "\n".join(lines)
 def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--snapshot", type=Path, default=ROOT / "backtest_snapshots/research.sqlite")
    parser.add_argument("--out", type=Path, default=ROOT / "reports/residual-normalization-20260908.json")
    args = parser.parse_args()
    snapshot = args.snapshot.resolve(strict=True)
    before = sha256(snapshot)
    code_paths = [Path(__file__).resolve(), PROTOCOL, ROOT / "app/services/momentum_service.py",
                  ROOT / "app/services/backtest_service.py"]
    hashes = {p.relative_to(ROOT).as_posix(): git_blob_id(p) for p in code_paths}
    prices, benchmark, manifest = read_snapshot(snapshot)
    print(f"manifest verified: {len(prices)} production-universe names; snapshot {before[:16]}", flush=True)
    screen = run_screen(prices, benchmark)
    if sha256(snapshot) != before or any(git_blob_id(ROOT / p) != h for p, h in hashes.items()):
        raise RuntimeError("Snapshot, protocol, or implementation changed during the run")
    report = {
        "schema_version": 1, "generated_at": datetime.now(timezone.utc).isoformat(),
        "status": "complete_signal_screen", "snapshot": str(snapshot), "snapshot_sha256": before,
        "source_git_blobs": hashes, "manifest": manifest, "python": sys.version,
        "packages": {p: importlib.metadata.version(p) for p in ("numpy", "sqlalchemy")},
        "screen": screen,
        "portfolio_stage": "eligible_pending_replay" if screen["verdict"]["passed"] else "not_run_failed_signal_screen",
        "production_changed": False,
    }
    args.out.parent.mkdir(parents=True, exist_ok=True)
    args.out.write_text(json.dumps(report, indent=2, allow_nan=False) + "\n", encoding="utf-8")
    args.out.with_suffix(".md").write_text(render_report(report), encoding="utf-8")
    print(json.dumps({"verdict": screen["verdict"], "summaries": screen["summaries"],
                      "report": str(args.out)}, indent=2), flush=True)
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -920,6 +920,190 @@ class TestSimulatePortfolio:
        assert sim["exit_reasons"] == {"low20": 1}
        assert sim["worst_trade_r"] == pytest.approx(-0.25)
    def test_breakeven_stop_lifts_stop_to_entry_after_k_r(self):
        # Entry 100, stop 95 → 1R = 5. The 106 close clears +1R, so from the next
        # bar the stop sits at 100; the 101 close (low 100) takes it out flat.
        # Too few bars for an ATR, so the trail never moves — only breakeven does.
        closes = [100.0, 103.0, 106.0, 104.0, 101.0, 99.0, 97.0, 96.0]
        prices = {"AAA": _sim_prices(self.ORD, closes)}
        cand = _sim_cand("AAA", self.ORD, entry=100.0, stop=95.0, target=130.0)
        control = bt._simulate_portfolio([cand], prices, None, "atr_trail3", 30)
        be1 = bt._simulate_portfolio(
            [cand], prices, None, "atr_trail3", 30, include_trades=True, breakeven_r=1.0
        )
        be2 = bt._simulate_portfolio([cand], prices, None, "atr_trail3", 30, breakeven_r=2.0)
        assert "breakeven_exits" not in control
        assert control["exit_reasons"] == {"stop": 1}
        assert be1["exit_reasons"] == {"breakeven_stop": 1}
        assert be1["breakeven_exits"] == 1
        assert be1["trade_details"][0]["fill"] == pytest.approx(100.0)
        assert be1["trade_details"][0]["exit_date"] == date.fromordinal(self.ORD + 4).isoformat()
        # +2R needs a 110 close, never reached: identical to control.
        assert be2["breakeven_exits"] == 0
        assert be2["final_equity"] == control["final_equity"]
    def test_breakeven_exit_locks_reentry_only_when_gated(self):
        # Same path; AAA qualifies again on the breakeven exit day. Live only locks
        # re-entry after an initial stop, so by default it is bought back that
        # session. Gated, the exit opens a post-stop episode that never resets.
        closes = [100.0, 103.0, 106.0, 104.0, 101.0, 99.0, 97.0, 96.0]
        prices = {"AAA": _sim_prices(self.ORD, closes)}
        cands = [
            _sim_cand("AAA", self.ORD, entry=100.0, stop=95.0, target=130.0),
            _sim_cand("AAA", self.ORD + 4, entry=101.0, stop=96.0, target=130.0),
        ]
        def run(gate_reset: bool) -> dict:
            return bt._simulate_portfolio(
                cands, prices, None, "atr_trail3", 30,
                post_stop_reentry_fn=lambda *_: None,
                breakeven_r=1.0,
                breakeven_gate_reset=gate_reset,
            )
        live, gated = run(False), run(True)
        # Live: the re-buy at 101 later hits its own initial stop.
        assert live["exit_reasons"] == {"breakeven_stop": 1, "stop": 1}
        assert live["breakeven_same_session_reentries"] == 1
        assert gated["exit_reasons"] == {"breakeven_stop": 1}
        assert gated["breakeven_same_session_reentries"] == 0
        assert gated["post_stop_events"] == 1
    def test_breakeven_applies_on_the_next_open_fill_bar(self):
        # Flat warm-up gives ATR 4: fill at the 100 open, stop 94, 1R = 6. The fill
        # bar closes at 107 (> +1R), so the next bar's 99.5 low takes the trade out
        # at entry — not a session later, after the next bar's own ratchet.
        n = 42
        opens, highs, lows, closes = [100.0] * n, [102.0] * n, [98.0] * n, [100.0] * n
        opens[40], highs[40], lows[40], closes[40] = 100.0, 108.0, 99.0, 107.0
        opens[41], highs[41], lows[41], closes[41] = 104.0, 105.0, 99.5, 103.0
        ords = list(range(self.ORD, self.ORD + n))
        prices = {"AAA": (ords, opens, highs, lows, closes, [1_000_000] * n)}
        cand = _sim_cand("AAA", self.ORD + 39, entry=100.0, stop=95.0, target=130.0)
        sim = bt._simulate_portfolio(
            [cand], prices, None, "atr_trail3", 30,
            fill_mode=bt.FILL_MODE_NEXT_OPEN,
            cost_per_side=0.0,
            include_trades=True,
            breakeven_r=1.0,
        )
        trade = sim["trade_details"][0]
        assert trade["initial_stop"] == pytest.approx(94.0)
        assert trade["reason"] == "breakeven_stop"
        assert trade["fill"] == pytest.approx(100.0)
        assert trade["exit_date"] == date.fromordinal(self.ORD + 41).isoformat()
    @pytest.mark.parametrize(
        ("policy", "kwargs"),
        [
            ("atr_trail3", {"breakeven_r": 0.0}),
            ("atr_trail3", {"breakeven_r": -1.0}),
            ("atr_trail3", {"breakeven_r": float("nan")}),
            ("hold", {"breakeven_r": 1.0}),
            ("atr_trail3", {"breakeven_gate_reset": True}),
            # Gated, but nothing to gate: no post-stop episode and no cooldown.
            ("atr_trail3", {"breakeven_r": 1.0, "breakeven_gate_reset": True}),
        ],
    )
    def test_breakeven_rejects_settings_it_cannot_honor(self, policy, kwargs):
        prices = {"AAA": _sim_prices(self.ORD, [100.0, 101.0])}
        cand = _sim_cand("AAA", self.ORD, entry=100.0, stop=95.0, target=130.0)
        with pytest.raises(ValueError):
            bt._simulate_portfolio([cand], prices, None, policy, 30, **kwargs)
    LIVE_EXIT = {"mode": "atr_trailing", "hold_days": 30, "atr_multiplier": 3.0}
    def _split_book(self, monkeypatch, per_window: int = 1) -> tuple[list[dict], dict]:
        """``per_window`` trades on each side of the split, one symbol each, 50
        days apart. be_2r never triggers (+2R needs a 110 close), so it ties
        control in both windows. Three per window spans the >0.25y CAGR needs."""
        closes = [100.0, 103.0, 106.0, 104.0, 101.0, 99.0, 97.0, 96.0]
        split = self.ORD + 50 * per_window
        prices, cands = {}, []
        for side, first in (("A", self.ORD), ("B", split)):
            for k in range(per_window):
                sym, day = f"{side}{k}", first + 50 * k
                prices[sym] = _sim_prices(day, closes)
                cands.append(_sim_cand(sym, day, entry=100.0, stop=95.0, target=130.0))
        monkeypatch.setattr(bt, "BREAKEVEN_STUDY_SPLIT", date.fromordinal(split))
        return cands, prices
    def test_breakeven_study_grades_the_live_book_and_flags_ties(self, monkeypatch):
        cands, prices = self._split_book(monkeypatch, per_window=3)
        sizes: list[tuple[int, float]] = []
        real_simulate = bt._simulate_portfolio
        def spy(*args, **kwargs):
            sizes.append((kwargs["max_positions"], kwargs["risk_per_trade"]))
            return real_simulate(*args, **kwargs)
        monkeypatch.setattr(bt, "_simulate_portfolio", spy)
        report = bt._breakeven_exit_study(
            cands, prices, None, 30,
            live_exit_policy=self.LIVE_EXIT,
            live_book={
                "capacity": 3, "risk_per_trade": 0.02,
                "capacity_source": "shadow_book_capacity",
                "risk_source": "shadow_book default",
            },
        )
        assert (report["capacity"], report["risk_per_trade"]) == (3, 0.02)
        assert report["capacity_source"] == "shadow_book_capacity"
        assert report["risk_source"] == "shadow_book default"
        assert report["verdicts"]["be_2r"]["ties_control_on_graded_metrics"] is True
        assert report["verdicts"]["be_1r"]["ties_control_on_graded_metrics"] is False
        # No monitor row to match, so nothing can pass.
        assert report["parity"]["ok"] is False
        assert not any(v["passes"] for v in report["verdicts"].values())
        assert {r["arm"] for r in report["diagnostic_rows"]} == {"be_1r_gated"}
        assert all(r["breakeven_gate_reset"] for r in report["diagnostic_rows"])
        # 3 graded arms + 1 diagnostic arm x 3 windows on the live book, then the
        # parity row on the monitor's.
        assert sizes[:12] == [(3, 0.02)] * 12
        assert sizes[12:] == [
            (report["parity"]["capacity"], report["parity"]["risk_per_trade"])
        ]
    def test_breakeven_study_without_live_config_says_so(self, monkeypatch):
        cands, prices = self._split_book(monkeypatch)
        report = bt._breakeven_exit_study(cands, prices, None, 30, live_exit_policy=self.LIVE_EXIT)
        assert (report["capacity"], report["capacity_source"]) == (
            bt.SIM_MAX_POSITIONS, "SIM_MAX_POSITIONS",
        )
        assert report["risk_source"] == "entry variant"
    def test_breakeven_parity_finds_the_monitor_production_row(self, monkeypatch):
        # The guard's success path: the study's control at the monitor's capacity
        # must match the monitor's production "all" row, and then a tie can pass.
        cands, prices = self._split_book(monkeypatch, per_window=3)
        monitor = bt._portfolio_monitor(cands, prices, None, 30, live_exit_policy=self.LIVE_EXIT)
        report = bt._breakeven_exit_study(
            cands, prices, None, 30, live_exit_policy=self.LIVE_EXIT, monitor_report=monitor,
        )
        parity = report["parity"]
        assert parity["monitor_production_all"] is not None
        assert parity["study_control"] == parity["monitor_production_all"]
        assert parity["ok"] is True
        assert report["verdicts"]["be_2r"]["passes"] is True
        assert report["verdicts"]["be_2r"]["ties_control_on_graded_metrics"] is True
    def test_breakeven_checks_fail_on_missing_metrics(self):
        control = {"sharpe": 1.5, "cagr_pct": 40.0, "max_drawdown_pct": 20.0}
        assert bt._breakeven_checks(dict(control), control) == {
            "sharpe": True, "cagr": True, "max_drawdown": True,
        }
        worse_dd = {**control, "max_drawdown_pct": 20.1}
        assert bt._breakeven_checks(worse_dd, control)["max_drawdown"] is False
        assert bt._breakeven_checks({**control, "cagr_pct": None}, control)["cagr"] is False
    def test_nothing_qualified_returns_none(self):
        assert bt._simulate_portfolio([], {}, None, "hold", 30) is None
@@ -0,0 +1,238 @@
 """Economic invariants and calendar/coverage guards for normalization research."""
 import json
 import sqlite3
 from datetime import date, timedelta
 from pathlib import Path
 import numpy as np
 import pytest
 from app.services.momentum_service import compute_residual_12_1_momentum
 from scripts import run_residual_normalization as study
 def sessions(n=400):
    days, day = [], date(2019, 1, 2)
    while len(days) < n:
        if day.weekday() < 5:
            days.append(day)
        day += timedelta(days=1)
    return days
 def series(beta=1.3, scale=1.0):
    days = sessions()
    rng = np.random.default_rng(59)
    market = rng.normal(0.0003, 0.01, len(days) - 1)
    specific = rng.normal(0.0008, 0.005, len(days) - 1)
    spy = [100.0, *list(100 * np.cumprod(1 + market))]
    stock = [100.0, *list(100 * np.cumprod(1 + beta * market + scale * specific))]
    return days, stock, dict(zip(days, spy)), {d: i for i, d in enumerate(days)}
 def test_numerator_is_production_and_denominator_uses_sample_variance():
    days, stock, spy, index = series()
    i = 300
    got = study.signal_at(days, stock, i, spy, index)
    assert got["paired_control"] == compute_residual_12_1_momentum(days[:i + 1], stock[:i + 1], spy)
    s = np.diff(np.array(stock[i - 252:i - 21 + 1])) / np.array(stock[i - 252:i - 21])
    mclose = np.array([spy[d] for d in days[i - 252:i - 21 + 1]])
    m = np.diff(mclose) / mclose[:-1]
    # Independent least-squares slope calculation, keeping alpha in the score.
    intercept, beta = np.linalg.lstsq(np.column_stack([np.ones(len(m)), m]), s, rcond=None)[0]
    assert intercept != 0
    expected_sd = (s - beta * m).std(ddof=1)
    assert got["residual_sd"] == pytest.approx(expected_sd)
    assert got["normalized"] == pytest.approx(got["paired_control"] / expected_sd)
 def test_normalization_removes_specific_return_scale_but_keeps_direction():
    days, stock, spy, index = series(scale=1)
    _, larger, _, _ = series(scale=3)
    a = study.signal_at(days, stock, 350, spy, index)
    b = study.signal_at(days, larger, 350, spy, index)
    assert abs(a["paired_control"]) > 0.01
    assert b["paired_control"] == pytest.approx(3 * a["paired_control"])
    assert b["normalized"] == pytest.approx(a["normalized"])
    assert np.sign(a["normalized"]) == np.sign(a["paired_control"])
 def test_market_beta_change_does_not_become_specific_strength():
    days, stock, spy, index = series(beta=0.5)
    _, high_beta, _, _ = series(beta=2.5)
    a = study.signal_at(days, stock, 350, spy, index)
    b = study.signal_at(days, high_beta, 350, spy, index)
    assert b["normalized"] == pytest.approx(a["normalized"])
 def test_skipped_month_and_future_are_not_signal_inputs():
    days, stock, spy, index = series()
    i = 300
    baseline = study.signal_at(days, stock, i, spy, index)
    for k in range(i - 20, len(stock)):
        stock[k] *= 1.8
        spy[days[k]] *= 0.7
    assert study.signal_at(days, stock, i, spy, index) == baseline
 def test_missing_benchmark_pair_stays_only_in_unrestricted_control():
    days, stock, spy, index = series()
    del spy[days[200]]
    index = {d: i for i, d in enumerate(sorted(spy))}
    got = study.signal_at(days, stock, 350, spy, index)
    assert got["unrestricted_control"] is not None
    assert got["paired_control"] is None
    assert got["normalized"] is None
 def test_missing_stock_session_cannot_be_called_daily_aligned():
    days, stock, spy, index = series()
    del days[200]
    del stock[200]
    got = study.signal_at(days, stock, 349, spy, index)
    assert got["unrestricted_control"] is not None
    assert got["reason"] == "unaligned_formation"
 def test_perfect_market_tracker_is_not_an_infinite_score():
    days, _, spy, index = series()
    got = study.signal_at(days, [spy[d] for d in days], 350, spy, index)
    assert got["normalized"] is None
    assert got["reason"] == "zero_residual_variance"
 def test_nonoverlap_uses_sessions_and_permits_only_a_shared_endpoint():
    # Arbitrary indices stand in for a calendar with holidays/year boundaries.
    candidates = [100, 104, 109, 114, 119, 123, 128, 133, 138, 143, 148, 153, 158, 163]
    assert study.nonoverlapping_indices(candidates) == [100, 133, 163]
 def test_weekly_endpoint_not_moved_earlier_to_obtain_forward_data():
    days = sessions(100)
    got = study.weekly_indices(days)
    last_per_week = {}
    for i, day in enumerate(days):
        last_per_week[day.isocalendar()[:2]] = i
    assert all(last_per_week[days[i].isocalendar()[:2]] == i for i in got)
    assert all(i + study.HORIZON < len(days) for i in got)
 def test_identical_columns_have_zero_paired_bootstrap_difference():
    x = np.linspace(-0.1, 0.2, 25)
    matrix = np.column_stack([x, x, x])
    got = study.bootstrap_means(matrix)
    assert got["paired_delta_ci95"] == [0.0, 0.0]
    assert got == study.bootstrap_means(matrix)
 def test_screen_uses_unrounded_floor_and_requires_both_eras():
    def summaries(value, pre=0.01, post=0.04):
        return {era: {"windows": 20, "arms": {"normalized": {"mean_ic": mean}}}
                for era, mean in (("full", value), ("pre_2021", pre), ("post_2021", post))}
    assert not study.screen_pass(summaries(0.0299999))["passed"]
    assert study.screen_pass(summaries(0.03))["passed"]
    assert not study.screen_pass(summaries(0.04, pre=-0.001))["passed"]
 def test_screen_keeps_paired_stocks_and_forward_endpoints_identical(monkeypatch):
    days = sessions()
    benchmark = {d: 100 + i for i, d in enumerate(days)}
    prices = {}
    for k in range(25):
        closes = [100 + k + i * (k + 1) / 100 for i in range(len(days))]
        prices[f"S{k:02}"] = ([d.toordinal() for d in days], [], [], [], closes, [])
    def fake_signal(dates, closes, i, spy, index):
        value = closes[0]
        # One name remains in the unrestricted control but not the paired set.
        excluded = value == 100
        return {"unrestricted_control": value, "paired_control": None if excluded else value,
                "normalized": None if excluded else value / 2,
                "reason": "unaligned_formation" if excluded else "eligible"}
    monkeypatch.setattr(study, "signal_at", fake_signal)
    report = study.run_screen(prices, benchmark)
    assert report["windows"]
    index = {d.isoformat(): i for i, d in enumerate(days)}
    previous_end = None
    for window in report["windows"]:
        assert index[window["forward_end"]] - index[window["asof"]] == 30
        if previous_end:
            assert window["asof"] >= previous_end
        previous_end = window["forward_end"]
        assert window["arms"]["unrestricted_control"]["n"] == 25
        assert window["arms"]["paired_control"]["n"] == 24
        assert window["arms"]["normalized"]["n"] == 24
        assert window["arms"]["normalized"]["ic"] == window["arms"]["paired_control"]["ic"]
 def write_snapshot(path, closes):
    """Minimal snapshot: one ticker, SPY, and a matching completion manifest."""
    days = sessions(len(closes))
    con = sqlite3.connect(path)
    con.executescript(
        "CREATE TABLE tickers (id INTEGER, symbol TEXT);"
        "CREATE TABLE research_rank_only (symbol TEXT);"
        "CREATE TABLE ohlcv_records (ticker_id INTEGER, date TEXT, open REAL, high REAL,"
        " low REAL, close REAL, volume REAL);"
        "CREATE TABLE benchmark_prices (symbol TEXT, date TEXT, close REAL);"
        "INSERT INTO tickers VALUES (1, 'AAA');"
    )
    con.executemany(
        "INSERT INTO ohlcv_records VALUES (1, ?, ?, ?, ?, ?, NULL)",
        [(d.isoformat(), c, c, c, c) for d, c in zip(days, closes)],
    )
    con.executemany(
        "INSERT INTO benchmark_prices VALUES ('SPY', ?, 100.0)", [(d.isoformat(),) for d in days]
    )
    con.commit()
    con.close()
    Path(str(path) + ".manifest.json").write_text(json.dumps({
        "complete": True, "ticker_count": 1, "ohlcv_row_count": len(closes), "rank_only_count": 0,
    }))
 def test_missing_price_stops_the_run_instead_of_reading_as_zero(tmp_path):
    path = tmp_path / "snap.sqlite"
    write_snapshot(path, [100.0, None, 102.0])
    with pytest.raises(ValueError, match="Missing OHLC price: AAA"):
        study.read_snapshot(path)
 def test_snapshot_read_keeps_missing_volume_and_releases_the_file(tmp_path):
    path = tmp_path / "snap.sqlite"
    write_snapshot(path, [100.0, 101.0, 102.0])
    prices, benchmark, _ = study.read_snapshot(path)
    assert prices["AAA"][4] == [100.0, 101.0, 102.0]
    assert prices["AAA"][5] == [0.0, 0.0, 0.0]
    assert len(benchmark) == 3
    path.unlink()  # Windows refuses while a connection still holds the file.
 def test_paired_coverage_mismatch_raises_rather_than_asserts(monkeypatch):
    days = sessions()
    benchmark = {d: 100 + i for i, d in enumerate(days)}
    prices = {
        f"S{k:02}": ([d.toordinal() for d in days], [], [], [],
                     [100 + k + i * (k + 1) / 100 for i in range(len(days))], [])
        for k in range(25)
    }
    def broken_pairing(dates, closes, i, spy, index):
        value = closes[0]
        return {"unrestricted_control": value, "paired_control": value,
                "normalized": None if value == 100 else value / 2, "reason": "eligible"}
    monkeypatch.setattr(study, "signal_at", broken_pairing)
    with pytest.raises(AssertionError, match="Paired arms scored different stocks"):
        study.run_screen(prices, benchmark)
 def test_source_ids_are_git_blob_ids_whatever_the_line_endings(tmp_path):
    lf, crlf = tmp_path / "lf.txt", tmp_path / "crlf.txt"
    lf.write_bytes(b"hello\n")
    crlf.write_bytes(b"hello\r\n")
    # `printf 'hello\n' | git hash-object --stdin`
    assert study.git_blob_id(lf) == "ce013625030ba8dba906f756967f9e9ca394464a"
    assert study.git_blob_id(crlf) == study.git_blob_id(lf)
Author	SHA1	Message	Date
dennisthiessenandClaude Opus 5.5	231372fb3a	fix(backtest): address the second review of the breakeven and book-config work Deploy / lint (push) Successful in 14s Details Deploy / test (push) Successful in 2m0s Details Deploy / deploy (push) Successful in 48s Details - The breakeven study grades at the shadow book's capacity AND risk per trade, and labels each with where it came from. A missing setting row now reads "shadow_book default" instead of crediting the setting; the committed report said shadow_book_capacity though the snapshot has no such row. - _production_book_config reads strategies through the same _portfolio_monitor_strategies() hook as the monitor. - breakeven_gate_reset is rejected when there is no post-stop episode or cooldown for it to start, and the study skips its diagnostic arm for a book without one, so a production config change cannot abort the run. - identical_to_control is renamed ties_control_on_graded_metrics and needs real values: be_2r ties on every graded metric but still changed 2 trades. - _BookConfig drops an unused field and hashes by identity; the min-R:R sweep swaps its gate in with dataclasses.replace instead of passing it. - New tests cover the parity guard's success path, the no-config fallback and the gate-reset check. The breakeven report is regenerated: every row, diagnostic row and the parity check are identical; only the labels, the risk fields and the renamed flag changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 21:41:44 +02:00
dennisthiessenandClaude Opus 5.5	7a9b3ad38b	research: volatility-standardized residual momentum screen, failed Strategy review (2026-09-08) and a frozen, paired signal screen of residual 12-1 momentum divided by its formation-window residual volatility. Across 73 non-overlapping windows the normalized signal's IC is 0.0128 against 0.0145 for the matched control, below the frozen 0.03 floor, so the conditional portfolio replay was not run and nothing in production changes. Code-review fixes to the runner, before first commit: a missing OHLC price stops the run instead of reading as 0; the snapshot connection is closed explicitly (on Windows it otherwise kept the file locked); the paired-coverage check raises instead of asserting, so python -O cannot strip it; and source files are recorded as git blob ids with line endings normalized, so the report can be checked against history. The report was regenerated with the fixed runner; its screen section is byte-identical to the original run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 18:45:24 +02:00
dennisthiessenandClaude Opus 5.5	70d17c1491	refactor(backtest): resolve the production book config in one place The portfolio monitor, holdout evaluation, min-R:R sweep and breakeven study each rebuilt the strategy's exit, hold, trail, gate, ranking and re-entry settings by hand, and the copies had drifted: only the monitor honored a strategy-level ranking_key. They now share _strategy_book_config / _production_book_config, which follow the monitor's rule. No output changes. On the prod snapshot (daily cadence, holdout split, min-R:R sweep and breakeven study all on), all four functions return byte-identical JSON to the previous commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 12:17:27 +02:00
dennisthiessenandClaude Opus 5.5	3545b483d8	research(backtest): breakeven-stop exit test, rejected Pre-registered test of raising the stop to entry once a trade is +1R or +2R, on top of the 3x ATR trail. be_1r fails the frozen rule (train worse on all three graded metrics, test drawdown worse); be_2r changes 2 of 480 trades because the trail already reaches entry near +2R. The production exit stays. Breakeven exits get their own breakeven_stop label. They follow live re-entry semantics (only initial-stop exits lock re-entry), and an ungraded be_1r_gated diagnostic arm sends them through the post-stop gate reset instead. The trail and breakeven ratchet share one helper, so breakeven also applies on a next_open fill bar. breakeven_r is validated, verdicts flag passes that are only ties, and the graded capacity comes from the shadow book config. Every graded number and the parity row are unchanged from the first run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 09:42:47 +02:00