Files
signal-platform/docs/research/breakeven-stop.md
T
dennisthiessenandClaude Opus 5.5 231372fb3a
Deploy / lint (push) Successful in 14s
Deploy / test (push) Successful in 2m0s
Deploy / deploy (push) Successful in 48s
fix(backtest): address the second review of the breakeven and book-config work
- The breakeven study grades at the shadow book's capacity AND risk per
  trade, and labels each with where it came from. A missing setting row
  now reads "shadow_book default" instead of crediting the setting; the
  committed report said shadow_book_capacity though the snapshot has no
  such row.
- _production_book_config reads strategies through the same
  _portfolio_monitor_strategies() hook as the monitor.
- breakeven_gate_reset is rejected when there is no post-stop episode or
  cooldown for it to start, and the study skips its diagnostic arm for a
  book without one, so a production config change cannot abort the run.
- identical_to_control is renamed ties_control_on_graded_metrics and needs
  real values: be_2r ties on every graded metric but still changed 2 trades.
- _BookConfig drops an unused field and hashes by identity; the min-R:R
  sweep swaps its gate in with dataclasses.replace instead of passing it.
- New tests cover the parity guard's success path, the no-config fallback
  and the gate-reset check.

The breakeven report is regenerated: every row, diagnostic row and the
parity check are identical; only the labels, the risk fields and the
renamed flag changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 21:41:44 +02:00

215 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Breakeven stop — pre-registered exit test
**Date:** 2026-10-09
**Question:** a trade ran roughly +10%, the 3× ATR trail still sat near entry,
and the pullback closed it at a small loss. Would moving the stop to the entry
price once a trade is up by k R turn trades like that into wins *without*
costing the book its right tail?
Every "lock in gains sooner" exit tested so far lost money: a trail of 2× ATR or
tighter, the target take-profit (Sharpe 2.04 → 1.47), and every fixed
take-profit level ([research log](README.md) #1, #12; method rules 3 and 5). The
breakeven stop is the one member of that family never run. The expectation is a
fail — just after it triggers it acts like a tighter trail — but it is an
untested arm, not a settled one.
## Protocol (frozen before the run)
**Data and book.** `backtest_snapshots/prod.sqlite`, daily entry cadence (the
live scan is daily, and a rule that frees slots earlier needs daily entries to
refill them). The production book exactly as the `portfolio_monitor` production
row replays it — live activation gate, 80/20 residual/high-vol rank, post-stop
gate reset, 1% risk, 20% notional cap, close fill, 0.1% cost per side — at
**capacity 15**, the live setting (the monitor row still uses 10).
**Arms.** Only the exit changes.
| arm | rule |
|---|---|
| `control` | production: 1.5× ATR initial stop, 3× ATR trail on the highest close, 30-session max hold |
| `be_1r` | control, plus: once the highest close reaches entry + 1R, the stop rises to the entry price |
| `be_2r` | same at entry + 2R |
R = entry − initial stop. The trigger reads closes, like the trail, and a raised
stop is active from the next bar; a gap through it fills at the open. The stop
only ever moves up — the trail keeps working above entry. No other trigger
levels are run.
**Windows.** Train = entries before 2024-07-01, test = entries on/after it, as
disjoint books (`_simulate_portfolio` `end_date`/`start_date`). Full history is
reported but not graded.
**Pass rule.** An arm passes only if, in **both** train and test, all three hold
versus `control` at report precision (Sharpe 2 dp, CAGR and drawdown 1 dp):
- Sharpe ≥ control
- CAGR ≥ control
- max drawdown ≤ control
Win rate and average R per trade are reported but **not** graded: a breakeven
stop raises both mechanically, and it changes trade count, so EV per trade is
the wrong lens ([capacity findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens)).
**Parity guard.** The study also runs `control` at the monitor's capacity 10 on
full history. It must equal the `portfolio_monitor` production `all` row
(Sharpe, CAGR, drawdown, trades) or the study is measuring a different book and
no row is valid.
**What a pass earns.** The post-2024 window has informed earlier research, so it
is not a pristine holdout. A pass earns a forward paper trial only — no
production change. A fail closes the question for this snapshot.
## Reproducing
`prod.sqlite` predates later ticker/trade columns, so `select(Ticker)` fails on
it (see [survivorship-audit.md](survivorship-audit.md#caveats)). Run on a copy
synced with `prep_survivorship_snapshot.sync_schema`, which only adds nullable
columns and changes no data:
```bash
cp backtest_snapshots/prod.sqlite /tmp/prod-synced.sqlite
python -c "import sys; sys.path[:0] = ['.', 'scripts']; from pathlib import Path; \
from prep_survivorship_snapshot import sync_schema; sync_schema(Path('/tmp/prod-synced.sqlite'))"
BACKTEST_BREAKEVEN_STUDY=1 python scripts/run_backtest_snapshot.py \
/tmp/prod-synced.sqlite --cadence daily --workers 7 --allow-spawn \
--out reports/backtest-20261009-breakeven-stop.json
```
The graded capacity and risk per trade come from the shadow book's config in the
database the run reads (`shadow_book_capacity`, `shadow_book_risk_pct`).
`prod.sqlite` has neither row, so the shadow book's defaults apply: 15
positions and 1% risk. The report records each value with its source
(`capacity_source`, `risk_source`), which reads `shadow_book default` here.
## Result — no change; `be_1r` fails, `be_2r` passes only by being the production exit
Report: `reports/backtest-20261009-breakeven-stop.json` (daily cadence, 1,011,248
candidates, 5,449 qualified; 505 of 506 symbols evaluated — RHM has no stored
bars, nothing failed). **Parity held exactly:** the capacity-10 control reproduces
the monitor's production row (Sharpe 1.84, CAGR 49.8%, DD 18.3%, 465 trades).
The report was regenerated on 2026-10-10 after a code review
([corrections below](#code-review-corrections-2026-10-10)); every graded number
is unchanged.
Capacity 15, production book:
| arm | window | Sharpe (SE) | CAGR | max DD | trades | win % | avg R | ≥5R trades | breakeven exits |
|---|---|---|---|---|---|---|---|---|---|
| control | train | 1.71 (0.68) | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | — |
| control | test | 1.46 (0.71) | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | — |
| control | full | 1.69 (0.50) | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | — |
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 35.0 | 0.595 | 22 | 36 |
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 34.4 | 0.455 | 10 | 46 |
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 34.9 | 0.539 | 32 | 82 |
| `be_2r` | train | 1.71 | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | 1 |
| `be_2r` | test | 1.46 | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | 1 |
| `be_2r` | full | 1.69 | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | 2 |
**`be_1r`: FAIL.** Train is worse on all three graded metrics (Sharpe 1.71 → 1.67,
CAGR 47.3% → 45.3%, DD 17.8% → 18.5%). Test is much better on return (Sharpe
1.46 → 1.71, CAGR +8.8pp) but drawdown is still worse (16.9% → 18.1%). The
halves pointing opposite ways is the same signature as the rejected max-hold
arms: a regime interaction, not an edge. The verdict rests on the frozen rule;
no significance test was run on the gaps. The per-arm Sharpe SEs (~0.7) do not
bound them either: the arms share most of their trades, so the gap between two
arms is far less noisy than either Sharpe on its own.
82 of 502 trades ended at the breakeven stop, and the book took 22 more trades
than control. Those were not freed slots: the count cap never bound in any
capacity-15 row (0 refusals). An earlier exit releases the ticker itself (one
position per ticker) and the cash it held. The simulator does not count entries
dropped or shrunk for lack of cash, so the split between the two is not
measured. Only 4 of the 82 exits were bought back in the same session.
**`be_2r`: passes the rule on ties, and that pass means nothing.** It changed 2
of 480 trades. The reason is arithmetic: the initial stop is 1.5× ATR = 1R, and
the trail sits 3× ATR = 2R below the highest close. By the time a trade is +2R
up, the trail has already reached entry, unless ATR has widened since entry.
`be_2r` is therefore the production exit, with nothing to paper-trade.
Win rate *fell* under `be_1r` (36.0 → 34.9): an exit at entry is a small loss
after the 0.1%/side cost, not a win. Average R rose only because trade count and
mix changed — the reason it was not graded.
**Decision: keep the production exit. The breakeven question is closed for this
snapshot.** The trade that prompted it (a roughly 10% run that ended below
entry) was not identified, so its R is unknown. With ATR unchanged the trail
only reaches entry at +2R; that trade either peaked below +2R of its own risk or
saw ATR widen during the run. Either way, the rule that would have saved it
(`be_1r`, if it reached +1R) costs the book about as much elsewhere: no reliable
improvement in either direction.
**Side observation, not graded (worth its own look):** on this single daily
path, the capacity-15 control (Sharpe 1.69, CAGR 44.3%) is *below* the
capacity-10 production row (1.84, 49.8%) at the same 18.3% drawdown. At capacity
10 the count cap refused **510** entries; at 15 it refused none. Those two
counts do not measure the same thing: at 15, cash plus the 20% notional cap is
the binding limit, and the entries it drops or shrinks are not counted. What the
path does show is that, here, the tighter 10-slot book did better. The 2026-08-05 capacity decision
was made across 175 start-date paths (51 better, 2 worse), so one full-history
path disagreeing does not overturn it, but a −5.5pp gap is large enough to
deserve its own look.
## Code-review corrections (2026-10-10)
A review of the study code found several problems. Fixing them changed no graded
number. One of them prompted a diagnostic arm.
**Re-entry after a breakeven exit.** The first run labeled a breakeven exit
`trailing_stop`, because the stop sat above the initial stop. It now has its own
label, `breakeven_stop`, and the count is exact rather than inferred from the
stop price (82 / 36 / 46, as before). Either label means the same thing for
re-entry: the live lock (`trade_policy.get_reentry_gate_locks`) applies only
after an initial-stop exit. So under `be_1r` a name stopped at breakeven can be
bought again without a gate reset. That is how the rule would behave live, so
the graded arms keep it. A diagnostic arm, `be_1r_gated`, sends breakeven exits
through the post-stop gate reset as an initial stop would. It was added after
the first run, so it is reported and never graded.
| arm | window | Sharpe | CAGR | max DD | trades | breakeven exits | same-session re-buys |
|---|---|---|---|---|---|---|---|
| control | train | 1.71 | 47.3% | 17.8% | 243 | — | — |
| control | test | 1.46 | 34.3% | 16.9% | 249 | — | — |
| control | full | 1.69 | 44.3% | 18.3% | 480 | — | — |
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 36 | 1 |
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 46 | 3 |
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 82 | 4 |
| `be_1r_gated` | train | 1.72 | 47.1% | 18.0% | 253 | 38 | 0 |
| `be_1r_gated` | test | 1.78 | 45.0% | 18.1% | 249 | 46 | 0 |
| `be_1r_gated` | full | 1.79 | 47.7% | 18.0% | 500 | 84 | 0 |
The review suspected part of `be_1r`'s test-window lift came from same-session
re-buys. It did not: there were only 4. Gating re-entry after breakeven exits
did help `be_1r`, though: it improves Sharpe and CAGR in every window and
drawdown in two of three. The gate changes *which* re-entries happen, not just
whether a name is re-bought, and this is one post-hoc comparison on one path.
The result is consistent with the post-stop re-entry finding, where a gate
reset beat immediate re-entry ([post-stop-reentry.md](post-stop-reentry.md)),
but it does not establish the mechanism.
**The decision stands.** `be_1r_gated` would also fail the frozen rule:
drawdown is worse than control in both windows (17.8% → 18.0% train,
16.9% → 18.1% test), and train CAGR is 47.1% against 47.3%. It was also chosen
after seeing the data, so it earns nothing here. If the breakeven question is
reopened, this is the arm to pre-register. It needs data the study has not
seen, and the live gate lock extended to breakeven exits, so that the backtest
and live still agree.
**Other fixes. None moves a graded number.**
- A breakeven stop now also applies on a `next_open` fill bar. Before, it fired
one bar late there. The trail and breakeven rules now share one function. The
study uses close fills, so its rows are unaffected.
- `breakeven_r` must be positive and finite, and is rejected for exit policies
without an ATR trail. Before, a non-positive value moved the stop to entry on
the first bar, and other policies ignored the setting silently.
- Each verdict now records `ties_control_on_graded_metrics`, which is true for
`be_2r`. A pass that is only ties can no longer read as a finding. The flag
compares the graded metrics only: `be_2r` still changed 2 trades.
- The graded capacity and risk per trade come from the shadow book's config
instead of the `SIM_MAX_POSITIONS` constant and the entry variant's fixed 1%.
On a snapshot that is the snapshot's copy of the settings, not the current
live values, and a missing row is labeled as the default it fell back to
([Reproducing](#reproducing)). In this run both are defaults, equal to the
values the first run used.