- The breakeven study grades at the shadow book's capacity AND risk per trade, and labels each with where it came from. A missing setting row now reads "shadow_book default" instead of crediting the setting; the committed report said shadow_book_capacity though the snapshot has no such row. - _production_book_config reads strategies through the same _portfolio_monitor_strategies() hook as the monitor. - breakeven_gate_reset is rejected when there is no post-stop episode or cooldown for it to start, and the study skips its diagnostic arm for a book without one, so a production config change cannot abort the run. - identical_to_control is renamed ties_control_on_graded_metrics and needs real values: be_2r ties on every graded metric but still changed 2 trades. - _BookConfig drops an unused field and hashes by identity; the min-R:R sweep swaps its gate in with dataclasses.replace instead of passing it. - New tests cover the parity guard's success path, the no-config fallback and the gate-reset check. The breakeven report is regenerated: every row, diagnostic row and the parity check are identical; only the labels, the risk fields and the renamed flag changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
215 lines
12 KiB
Markdown
215 lines
12 KiB
Markdown
# Breakeven stop — pre-registered exit test
|
||
|
||
**Date:** 2026-10-09
|
||
**Question:** a trade ran roughly +10%, the 3× ATR trail still sat near entry,
|
||
and the pullback closed it at a small loss. Would moving the stop to the entry
|
||
price once a trade is up by k R turn trades like that into wins *without*
|
||
costing the book its right tail?
|
||
|
||
Every "lock in gains sooner" exit tested so far lost money: a trail of 2× ATR or
|
||
tighter, the target take-profit (Sharpe 2.04 → 1.47), and every fixed
|
||
take-profit level ([research log](README.md) #1, #12; method rules 3 and 5). The
|
||
breakeven stop is the one member of that family never run. The expectation is a
|
||
fail — just after it triggers it acts like a tighter trail — but it is an
|
||
untested arm, not a settled one.
|
||
|
||
## Protocol (frozen before the run)
|
||
|
||
**Data and book.** `backtest_snapshots/prod.sqlite`, daily entry cadence (the
|
||
live scan is daily, and a rule that frees slots earlier needs daily entries to
|
||
refill them). The production book exactly as the `portfolio_monitor` production
|
||
row replays it — live activation gate, 80/20 residual/high-vol rank, post-stop
|
||
gate reset, 1% risk, 20% notional cap, close fill, 0.1% cost per side — at
|
||
**capacity 15**, the live setting (the monitor row still uses 10).
|
||
|
||
**Arms.** Only the exit changes.
|
||
|
||
| arm | rule |
|
||
|---|---|
|
||
| `control` | production: 1.5× ATR initial stop, 3× ATR trail on the highest close, 30-session max hold |
|
||
| `be_1r` | control, plus: once the highest close reaches entry + 1R, the stop rises to the entry price |
|
||
| `be_2r` | same at entry + 2R |
|
||
|
||
R = entry − initial stop. The trigger reads closes, like the trail, and a raised
|
||
stop is active from the next bar; a gap through it fills at the open. The stop
|
||
only ever moves up — the trail keeps working above entry. No other trigger
|
||
levels are run.
|
||
|
||
**Windows.** Train = entries before 2024-07-01, test = entries on/after it, as
|
||
disjoint books (`_simulate_portfolio` `end_date`/`start_date`). Full history is
|
||
reported but not graded.
|
||
|
||
**Pass rule.** An arm passes only if, in **both** train and test, all three hold
|
||
versus `control` at report precision (Sharpe 2 dp, CAGR and drawdown 1 dp):
|
||
|
||
- Sharpe ≥ control
|
||
- CAGR ≥ control
|
||
- max drawdown ≤ control
|
||
|
||
Win rate and average R per trade are reported but **not** graded: a breakeven
|
||
stop raises both mechanically, and it changes trade count, so EV per trade is
|
||
the wrong lens ([capacity findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens)).
|
||
|
||
**Parity guard.** The study also runs `control` at the monitor's capacity 10 on
|
||
full history. It must equal the `portfolio_monitor` production `all` row
|
||
(Sharpe, CAGR, drawdown, trades) or the study is measuring a different book and
|
||
no row is valid.
|
||
|
||
**What a pass earns.** The post-2024 window has informed earlier research, so it
|
||
is not a pristine holdout. A pass earns a forward paper trial only — no
|
||
production change. A fail closes the question for this snapshot.
|
||
|
||
## Reproducing
|
||
|
||
`prod.sqlite` predates later ticker/trade columns, so `select(Ticker)` fails on
|
||
it (see [survivorship-audit.md](survivorship-audit.md#caveats)). Run on a copy
|
||
synced with `prep_survivorship_snapshot.sync_schema`, which only adds nullable
|
||
columns and changes no data:
|
||
|
||
```bash
|
||
cp backtest_snapshots/prod.sqlite /tmp/prod-synced.sqlite
|
||
python -c "import sys; sys.path[:0] = ['.', 'scripts']; from pathlib import Path; \
|
||
from prep_survivorship_snapshot import sync_schema; sync_schema(Path('/tmp/prod-synced.sqlite'))"
|
||
BACKTEST_BREAKEVEN_STUDY=1 python scripts/run_backtest_snapshot.py \
|
||
/tmp/prod-synced.sqlite --cadence daily --workers 7 --allow-spawn \
|
||
--out reports/backtest-20261009-breakeven-stop.json
|
||
```
|
||
|
||
The graded capacity and risk per trade come from the shadow book's config in the
|
||
database the run reads (`shadow_book_capacity`, `shadow_book_risk_pct`).
|
||
`prod.sqlite` has neither row, so the shadow book's defaults apply: 15
|
||
positions and 1% risk. The report records each value with its source
|
||
(`capacity_source`, `risk_source`), which reads `shadow_book default` here.
|
||
|
||
## Result — no change; `be_1r` fails, `be_2r` passes only by being the production exit
|
||
|
||
Report: `reports/backtest-20261009-breakeven-stop.json` (daily cadence, 1,011,248
|
||
candidates, 5,449 qualified; 505 of 506 symbols evaluated — RHM has no stored
|
||
bars, nothing failed). **Parity held exactly:** the capacity-10 control reproduces
|
||
the monitor's production row (Sharpe 1.84, CAGR 49.8%, DD 18.3%, 465 trades).
|
||
The report was regenerated on 2026-10-10 after a code review
|
||
([corrections below](#code-review-corrections-2026-10-10)); every graded number
|
||
is unchanged.
|
||
|
||
Capacity 15, production book:
|
||
|
||
| arm | window | Sharpe (SE) | CAGR | max DD | trades | win % | avg R | ≥5R trades | breakeven exits |
|
||
|---|---|---|---|---|---|---|---|---|---|
|
||
| control | train | 1.71 (0.68) | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | — |
|
||
| control | test | 1.46 (0.71) | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | — |
|
||
| control | full | 1.69 (0.50) | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | — |
|
||
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 35.0 | 0.595 | 22 | 36 |
|
||
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 34.4 | 0.455 | 10 | 46 |
|
||
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 34.9 | 0.539 | 32 | 82 |
|
||
| `be_2r` | train | 1.71 | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | 1 |
|
||
| `be_2r` | test | 1.46 | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | 1 |
|
||
| `be_2r` | full | 1.69 | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | 2 |
|
||
|
||
**`be_1r`: FAIL.** Train is worse on all three graded metrics (Sharpe 1.71 → 1.67,
|
||
CAGR 47.3% → 45.3%, DD 17.8% → 18.5%). Test is much better on return (Sharpe
|
||
1.46 → 1.71, CAGR +8.8pp) but drawdown is still worse (16.9% → 18.1%). The
|
||
halves pointing opposite ways is the same signature as the rejected max-hold
|
||
arms: a regime interaction, not an edge. The verdict rests on the frozen rule;
|
||
no significance test was run on the gaps. The per-arm Sharpe SEs (~0.7) do not
|
||
bound them either: the arms share most of their trades, so the gap between two
|
||
arms is far less noisy than either Sharpe on its own.
|
||
|
||
82 of 502 trades ended at the breakeven stop, and the book took 22 more trades
|
||
than control. Those were not freed slots: the count cap never bound in any
|
||
capacity-15 row (0 refusals). An earlier exit releases the ticker itself (one
|
||
position per ticker) and the cash it held. The simulator does not count entries
|
||
dropped or shrunk for lack of cash, so the split between the two is not
|
||
measured. Only 4 of the 82 exits were bought back in the same session.
|
||
|
||
**`be_2r`: passes the rule on ties, and that pass means nothing.** It changed 2
|
||
of 480 trades. The reason is arithmetic: the initial stop is 1.5× ATR = 1R, and
|
||
the trail sits 3× ATR = 2R below the highest close. By the time a trade is +2R
|
||
up, the trail has already reached entry, unless ATR has widened since entry.
|
||
`be_2r` is therefore the production exit, with nothing to paper-trade.
|
||
|
||
Win rate *fell* under `be_1r` (36.0 → 34.9): an exit at entry is a small loss
|
||
after the 0.1%/side cost, not a win. Average R rose only because trade count and
|
||
mix changed — the reason it was not graded.
|
||
|
||
**Decision: keep the production exit. The breakeven question is closed for this
|
||
snapshot.** The trade that prompted it (a roughly 10% run that ended below
|
||
entry) was not identified, so its R is unknown. With ATR unchanged the trail
|
||
only reaches entry at +2R; that trade either peaked below +2R of its own risk or
|
||
saw ATR widen during the run. Either way, the rule that would have saved it
|
||
(`be_1r`, if it reached +1R) costs the book about as much elsewhere: no reliable
|
||
improvement in either direction.
|
||
|
||
**Side observation, not graded (worth its own look):** on this single daily
|
||
path, the capacity-15 control (Sharpe 1.69, CAGR 44.3%) is *below* the
|
||
capacity-10 production row (1.84, 49.8%) at the same 18.3% drawdown. At capacity
|
||
10 the count cap refused **510** entries; at 15 it refused none. Those two
|
||
counts do not measure the same thing: at 15, cash plus the 20% notional cap is
|
||
the binding limit, and the entries it drops or shrinks are not counted. What the
|
||
path does show is that, here, the tighter 10-slot book did better. The 2026-08-05 capacity decision
|
||
was made across 175 start-date paths (51 better, 2 worse), so one full-history
|
||
path disagreeing does not overturn it, but a −5.5pp gap is large enough to
|
||
deserve its own look.
|
||
|
||
## Code-review corrections (2026-10-10)
|
||
|
||
A review of the study code found several problems. Fixing them changed no graded
|
||
number. One of them prompted a diagnostic arm.
|
||
|
||
**Re-entry after a breakeven exit.** The first run labeled a breakeven exit
|
||
`trailing_stop`, because the stop sat above the initial stop. It now has its own
|
||
label, `breakeven_stop`, and the count is exact rather than inferred from the
|
||
stop price (82 / 36 / 46, as before). Either label means the same thing for
|
||
re-entry: the live lock (`trade_policy.get_reentry_gate_locks`) applies only
|
||
after an initial-stop exit. So under `be_1r` a name stopped at breakeven can be
|
||
bought again without a gate reset. That is how the rule would behave live, so
|
||
the graded arms keep it. A diagnostic arm, `be_1r_gated`, sends breakeven exits
|
||
through the post-stop gate reset as an initial stop would. It was added after
|
||
the first run, so it is reported and never graded.
|
||
|
||
| arm | window | Sharpe | CAGR | max DD | trades | breakeven exits | same-session re-buys |
|
||
|---|---|---|---|---|---|---|---|
|
||
| control | train | 1.71 | 47.3% | 17.8% | 243 | — | — |
|
||
| control | test | 1.46 | 34.3% | 16.9% | 249 | — | — |
|
||
| control | full | 1.69 | 44.3% | 18.3% | 480 | — | — |
|
||
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 36 | 1 |
|
||
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 46 | 3 |
|
||
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 82 | 4 |
|
||
| `be_1r_gated` | train | 1.72 | 47.1% | 18.0% | 253 | 38 | 0 |
|
||
| `be_1r_gated` | test | 1.78 | 45.0% | 18.1% | 249 | 46 | 0 |
|
||
| `be_1r_gated` | full | 1.79 | 47.7% | 18.0% | 500 | 84 | 0 |
|
||
|
||
The review suspected part of `be_1r`'s test-window lift came from same-session
|
||
re-buys. It did not: there were only 4. Gating re-entry after breakeven exits
|
||
did help `be_1r`, though: it improves Sharpe and CAGR in every window and
|
||
drawdown in two of three. The gate changes *which* re-entries happen, not just
|
||
whether a name is re-bought, and this is one post-hoc comparison on one path.
|
||
The result is consistent with the post-stop re-entry finding, where a gate
|
||
reset beat immediate re-entry ([post-stop-reentry.md](post-stop-reentry.md)),
|
||
but it does not establish the mechanism.
|
||
|
||
**The decision stands.** `be_1r_gated` would also fail the frozen rule:
|
||
drawdown is worse than control in both windows (17.8% → 18.0% train,
|
||
16.9% → 18.1% test), and train CAGR is 47.1% against 47.3%. It was also chosen
|
||
after seeing the data, so it earns nothing here. If the breakeven question is
|
||
reopened, this is the arm to pre-register. It needs data the study has not
|
||
seen, and the live gate lock extended to breakeven exits, so that the backtest
|
||
and live still agree.
|
||
|
||
**Other fixes. None moves a graded number.**
|
||
|
||
- A breakeven stop now also applies on a `next_open` fill bar. Before, it fired
|
||
one bar late there. The trail and breakeven rules now share one function. The
|
||
study uses close fills, so its rows are unaffected.
|
||
- `breakeven_r` must be positive and finite, and is rejected for exit policies
|
||
without an ATR trail. Before, a non-positive value moved the stop to entry on
|
||
the first bar, and other policies ignored the setting silently.
|
||
- Each verdict now records `ties_control_on_graded_metrics`, which is true for
|
||
`be_2r`. A pass that is only ties can no longer read as a finding. The flag
|
||
compares the graded metrics only: `be_2r` still changed 2 trades.
|
||
- The graded capacity and risk per trade come from the shadow book's config
|
||
instead of the `SIM_MAX_POSITIONS` constant and the entry variant's fixed 1%.
|
||
On a snapshot that is the snapshot's copy of the settings, not the current
|
||
live values, and a missing row is labeled as the default it fell back to
|
||
([Reproducing](#reproducing)). In this run both are defaults, equal to the
|
||
values the first run used.
|