Files
dennisthiessenandClaude Opus 5.5 231372fb3a
Deploy / lint (push) Successful in 14s
Deploy / test (push) Successful in 2m0s
Deploy / deploy (push) Successful in 48s
fix(backtest): address the second review of the breakeven and book-config work
- The breakeven study grades at the shadow book's capacity AND risk per
  trade, and labels each with where it came from. A missing setting row
  now reads "shadow_book default" instead of crediting the setting; the
  committed report said shadow_book_capacity though the snapshot has no
  such row.
- _production_book_config reads strategies through the same
  _portfolio_monitor_strategies() hook as the monitor.
- breakeven_gate_reset is rejected when there is no post-stop episode or
  cooldown for it to start, and the study skips its diagnostic arm for a
  book without one, so a production config change cannot abort the run.
- identical_to_control is renamed ties_control_on_graded_metrics and needs
  real values: be_2r ties on every graded metric but still changed 2 trades.
- _BookConfig drops an unused field and hashes by identity; the min-R:R
  sweep swaps its gate in with dataclasses.replace instead of passing it.
- New tests cover the parity guard's success path, the no-config fallback
  and the gate-reset check.

The breakeven report is regenerated: every row, diagnostic row and the
parity check are identical; only the labels, the risk fields and the
renamed flag changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:41:44 +02:00

12 KiB
Raw Permalink Blame History

Breakeven stop — pre-registered exit test

Date: 2026-10-09 Question: a trade ran roughly +10%, the 3× ATR trail still sat near entry, and the pullback closed it at a small loss. Would moving the stop to the entry price once a trade is up by k R turn trades like that into wins without costing the book its right tail?

Every "lock in gains sooner" exit tested so far lost money: a trail of 2× ATR or tighter, the target take-profit (Sharpe 2.04 → 1.47), and every fixed take-profit level (research log #1, #12; method rules 3 and 5). The breakeven stop is the one member of that family never run. The expectation is a fail — just after it triggers it acts like a tighter trail — but it is an untested arm, not a settled one.

Protocol (frozen before the run)

Data and book. backtest_snapshots/prod.sqlite, daily entry cadence (the live scan is daily, and a rule that frees slots earlier needs daily entries to refill them). The production book exactly as the portfolio_monitor production row replays it — live activation gate, 80/20 residual/high-vol rank, post-stop gate reset, 1% risk, 20% notional cap, close fill, 0.1% cost per side — at capacity 15, the live setting (the monitor row still uses 10).

Arms. Only the exit changes.

arm rule
control production: 1.5× ATR initial stop, 3× ATR trail on the highest close, 30-session max hold
be_1r control, plus: once the highest close reaches entry + 1R, the stop rises to the entry price
be_2r same at entry + 2R

R = entry − initial stop. The trigger reads closes, like the trail, and a raised stop is active from the next bar; a gap through it fills at the open. The stop only ever moves up — the trail keeps working above entry. No other trigger levels are run.

Windows. Train = entries before 2024-07-01, test = entries on/after it, as disjoint books (_simulate_portfolio end_date/start_date). Full history is reported but not graded.

Pass rule. An arm passes only if, in both train and test, all three hold versus control at report precision (Sharpe 2 dp, CAGR and drawdown 1 dp):

  • Sharpe ≥ control
  • CAGR ≥ control
  • max drawdown ≤ control

Win rate and average R per trade are reported but not graded: a breakeven stop raises both mechanically, and it changes trade count, so EV per trade is the wrong lens (capacity findings).

Parity guard. The study also runs control at the monitor's capacity 10 on full history. It must equal the portfolio_monitor production all row (Sharpe, CAGR, drawdown, trades) or the study is measuring a different book and no row is valid.

What a pass earns. The post-2024 window has informed earlier research, so it is not a pristine holdout. A pass earns a forward paper trial only — no production change. A fail closes the question for this snapshot.

Reproducing

prod.sqlite predates later ticker/trade columns, so select(Ticker) fails on it (see survivorship-audit.md). Run on a copy synced with prep_survivorship_snapshot.sync_schema, which only adds nullable columns and changes no data:

cp backtest_snapshots/prod.sqlite /tmp/prod-synced.sqlite
python -c "import sys; sys.path[:0] = ['.', 'scripts']; from pathlib import Path; \
from prep_survivorship_snapshot import sync_schema; sync_schema(Path('/tmp/prod-synced.sqlite'))"
BACKTEST_BREAKEVEN_STUDY=1 python scripts/run_backtest_snapshot.py \
    /tmp/prod-synced.sqlite --cadence daily --workers 7 --allow-spawn \
    --out reports/backtest-20261009-breakeven-stop.json

The graded capacity and risk per trade come from the shadow book's config in the database the run reads (shadow_book_capacity, shadow_book_risk_pct). prod.sqlite has neither row, so the shadow book's defaults apply: 15 positions and 1% risk. The report records each value with its source (capacity_source, risk_source), which reads shadow_book default here.

Result — no change; be_1r fails, be_2r passes only by being the production exit

Report: reports/backtest-20261009-breakeven-stop.json (daily cadence, 1,011,248 candidates, 5,449 qualified; 505 of 506 symbols evaluated — RHM has no stored bars, nothing failed). Parity held exactly: the capacity-10 control reproduces the monitor's production row (Sharpe 1.84, CAGR 49.8%, DD 18.3%, 465 trades). The report was regenerated on 2026-10-10 after a code review (corrections below); every graded number is unchanged.

Capacity 15, production book:

arm window Sharpe (SE) CAGR max DD trades win % avg R ≥5R trades breakeven exits
control train 1.71 (0.68) 47.3% 17.8% 243 36.6 0.563 20 —
control test 1.46 (0.71) 34.3% 16.9% 249 35.3 0.366 12 —
control full 1.69 (0.50) 44.3% 18.3% 480 36.0 0.477 32 —
be_1r train 1.67 45.3% 18.5% 254 35.0 0.595 22 36
be_1r test 1.71 43.1% 18.1% 250 34.4 0.455 10 46
be_1r full 1.72 45.7% 18.5% 502 34.9 0.539 32 82
be_2r train 1.71 47.3% 17.8% 243 36.6 0.563 20 1
be_2r test 1.46 34.3% 16.9% 249 35.3 0.366 12 1
be_2r full 1.69 44.3% 18.3% 480 36.0 0.477 32 2

be_1r: FAIL. Train is worse on all three graded metrics (Sharpe 1.71 → 1.67, CAGR 47.3% → 45.3%, DD 17.8% → 18.5%). Test is much better on return (Sharpe 1.46 → 1.71, CAGR +8.8pp) but drawdown is still worse (16.9% → 18.1%). The halves pointing opposite ways is the same signature as the rejected max-hold arms: a regime interaction, not an edge. The verdict rests on the frozen rule; no significance test was run on the gaps. The per-arm Sharpe SEs (~0.7) do not bound them either: the arms share most of their trades, so the gap between two arms is far less noisy than either Sharpe on its own.

82 of 502 trades ended at the breakeven stop, and the book took 22 more trades than control. Those were not freed slots: the count cap never bound in any capacity-15 row (0 refusals). An earlier exit releases the ticker itself (one position per ticker) and the cash it held. The simulator does not count entries dropped or shrunk for lack of cash, so the split between the two is not measured. Only 4 of the 82 exits were bought back in the same session.

be_2r: passes the rule on ties, and that pass means nothing. It changed 2 of 480 trades. The reason is arithmetic: the initial stop is 1.5× ATR = 1R, and the trail sits 3× ATR = 2R below the highest close. By the time a trade is +2R up, the trail has already reached entry, unless ATR has widened since entry. be_2r is therefore the production exit, with nothing to paper-trade.

Win rate fell under be_1r (36.0 → 34.9): an exit at entry is a small loss after the 0.1%/side cost, not a win. Average R rose only because trade count and mix changed — the reason it was not graded.

Decision: keep the production exit. The breakeven question is closed for this snapshot. The trade that prompted it (a roughly 10% run that ended below entry) was not identified, so its R is unknown. With ATR unchanged the trail only reaches entry at +2R; that trade either peaked below +2R of its own risk or saw ATR widen during the run. Either way, the rule that would have saved it (be_1r, if it reached +1R) costs the book about as much elsewhere: no reliable improvement in either direction.

Side observation, not graded (worth its own look): on this single daily path, the capacity-15 control (Sharpe 1.69, CAGR 44.3%) is below the capacity-10 production row (1.84, 49.8%) at the same 18.3% drawdown. At capacity 10 the count cap refused 510 entries; at 15 it refused none. Those two counts do not measure the same thing: at 15, cash plus the 20% notional cap is the binding limit, and the entries it drops or shrinks are not counted. What the path does show is that, here, the tighter 10-slot book did better. The 2026-08-05 capacity decision was made across 175 start-date paths (51 better, 2 worse), so one full-history path disagreeing does not overturn it, but a −5.5pp gap is large enough to deserve its own look.

Code-review corrections (2026-10-10)

A review of the study code found several problems. Fixing them changed no graded number. One of them prompted a diagnostic arm.

Re-entry after a breakeven exit. The first run labeled a breakeven exit trailing_stop, because the stop sat above the initial stop. It now has its own label, breakeven_stop, and the count is exact rather than inferred from the stop price (82 / 36 / 46, as before). Either label means the same thing for re-entry: the live lock (trade_policy.get_reentry_gate_locks) applies only after an initial-stop exit. So under be_1r a name stopped at breakeven can be bought again without a gate reset. That is how the rule would behave live, so the graded arms keep it. A diagnostic arm, be_1r_gated, sends breakeven exits through the post-stop gate reset as an initial stop would. It was added after the first run, so it is reported and never graded.

arm window Sharpe CAGR max DD trades breakeven exits same-session re-buys
control train 1.71 47.3% 17.8% 243 — —
control test 1.46 34.3% 16.9% 249 — —
control full 1.69 44.3% 18.3% 480 — —
be_1r train 1.67 45.3% 18.5% 254 36 1
be_1r test 1.71 43.1% 18.1% 250 46 3
be_1r full 1.72 45.7% 18.5% 502 82 4
be_1r_gated train 1.72 47.1% 18.0% 253 38 0
be_1r_gated test 1.78 45.0% 18.1% 249 46 0
be_1r_gated full 1.79 47.7% 18.0% 500 84 0

The review suspected part of be_1r's test-window lift came from same-session re-buys. It did not: there were only 4. Gating re-entry after breakeven exits did help be_1r, though: it improves Sharpe and CAGR in every window and drawdown in two of three. The gate changes which re-entries happen, not just whether a name is re-bought, and this is one post-hoc comparison on one path. The result is consistent with the post-stop re-entry finding, where a gate reset beat immediate re-entry (post-stop-reentry.md), but it does not establish the mechanism.

The decision stands. be_1r_gated would also fail the frozen rule: drawdown is worse than control in both windows (17.8% → 18.0% train, 16.9% → 18.1% test), and train CAGR is 47.1% against 47.3%. It was also chosen after seeing the data, so it earns nothing here. If the breakeven question is reopened, this is the arm to pre-register. It needs data the study has not seen, and the live gate lock extended to breakeven exits, so that the backtest and live still agree.

Other fixes. None moves a graded number.

  • A breakeven stop now also applies on a next_open fill bar. Before, it fired one bar late there. The trail and breakeven rules now share one function. The study uses close fills, so its rows are unaffected.
  • breakeven_r must be positive and finite, and is rejected for exit policies without an ATR trail. Before, a non-positive value moved the stop to entry on the first bar, and other policies ignored the setting silently.
  • Each verdict now records ties_control_on_graded_metrics, which is true for be_2r. A pass that is only ties can no longer read as a finding. The flag compares the graded metrics only: be_2r still changed 2 trades.
  • The graded capacity and risk per trade come from the shadow book's config instead of the SIM_MAX_POSITIONS constant and the entry variant's fixed 1%. On a snapshot that is the snapshot's copy of the settings, not the current live values, and a missing row is labeled as the default it fell back to (Reproducing). In this run both are defaults, equal to the values the first run used.