research(backtest): breakeven-stop exit test, rejected
Pre-registered test of raising the stop to entry once a trade is +1R or +2R, on top of the 3x ATR trail. be_1r fails the frozen rule (train worse on all three graded metrics, test drawdown worse); be_2r changes 2 of 480 trades because the trail already reaches entry near +2R. The production exit stays. Breakeven exits get their own breakeven_stop label. They follow live re-entry semantics (only initial-stop exits lock re-entry), and an ungraded be_1r_gated diagnostic arm sends them through the post-stop gate reset instead. The trail and breakeven ratchet share one helper, so breakeven also applies on a next_open fill bar. breakeven_r is validated, verdicts flag passes that are only ties, and the graded capacity comes from the shadow book config. Every graded number and the parity row are unchanged from the first run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dennisthiessenandClaude Opus 5.5 committed
1 parent
4876f9ffdd
commit
3545b483d8
7 files changed
+144938
-24
No files matched your search
@@ -48,6 +48,7 @@ score, Structural S/R, the Gate Target Ladder, sentiment, fundamentals) is
|
||||
| 11 | **FIP path-smoothness** as tie-breaker/filter | Non-monotonic within the qualified set; thinning the entry stream costs more compounding than the tilt returns | **Rejected as a filter** — but see §4, it's the strongest raw signal we've measured | — |
|
||||
| 12 | **Fixed take-profit sweep** (R-multiples) | No interior optimum ever found — the best TP is "no TP" | **Rejected.** Momentum's edge lives in the right tail | `backtest_service.py:450` |
|
||||
| 13 | **Sector-residual 12-1** (`mom_12_1_sector_resid` / sector demean) as replacement for market residual | Short-window IC/A/B looked knife-edge green; deep repaired + **liquid-1500** retest: weeks 83, mild +IC **0.027** / t 1.69, **below iron bar 0.03** (FAIL). Demean already weaker | **Rejected / closed.** Keep production market residual. Do not resurrect without a new pre-registered protocol | [sector-residual-momentum.md](sector-residual-momentum.md) · `sector-resid-deep-20260719-113319.json` · history-depth supersession note |
|
||||
| 15 | **Breakeven stop** (stop to entry once the trade is +1R / +2R, on top of the 3× ATR trail) | `be_1r` fails the frozen rule: train worse on Sharpe, CAGR and drawdown, test better on return but worse on drawdown. `be_2r` changed 2 of 480 trades, because the trail already reaches entry near +2R. A post-hoc arm that gates re-entry after breakeven exits does better but still fails on drawdown | **Rejected / closed for this snapshot.** Keep the production exit. Reopen only with a new pre-registered test on unseen data | [breakeven-stop.md](breakeven-stop.md) · `backtest-20261009-breakeven-stop.json` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
# Breakeven stop — pre-registered exit test
|
||||
|
||||
**Date:** 2026-10-09
|
||||
**Question:** a trade ran roughly +10%, the 3× ATR trail still sat near entry,
|
||||
and the pullback closed it at a small loss. Would moving the stop to the entry
|
||||
price once a trade is up by k R turn trades like that into wins *without*
|
||||
costing the book its right tail?
|
||||
|
||||
Every "lock in gains sooner" exit tested so far lost money: a trail of 2× ATR or
|
||||
tighter, the target take-profit (Sharpe 2.04 → 1.47), and every fixed
|
||||
take-profit level ([research log](README.md) #1, #12; method rules 3 and 5). The
|
||||
breakeven stop is the one member of that family never run. The expectation is a
|
||||
fail — just after it triggers it acts like a tighter trail — but it is an
|
||||
untested arm, not a settled one.
|
||||
|
||||
## Protocol (frozen before the run)
|
||||
|
||||
**Data and book.** `backtest_snapshots/prod.sqlite`, daily entry cadence (the
|
||||
live scan is daily, and a rule that frees slots earlier needs daily entries to
|
||||
refill them). The production book exactly as the `portfolio_monitor` production
|
||||
row replays it — live activation gate, 80/20 residual/high-vol rank, post-stop
|
||||
gate reset, 1% risk, 20% notional cap, close fill, 0.1% cost per side — at
|
||||
**capacity 15**, the live setting (the monitor row still uses 10).
|
||||
|
||||
**Arms.** Only the exit changes.
|
||||
|
||||
| arm | rule |
|
||||
|---|---|
|
||||
| `control` | production: 1.5× ATR initial stop, 3× ATR trail on the highest close, 30-session max hold |
|
||||
| `be_1r` | control, plus: once the highest close reaches entry + 1R, the stop rises to the entry price |
|
||||
| `be_2r` | same at entry + 2R |
|
||||
|
||||
R = entry − initial stop. The trigger reads closes, like the trail, and a raised
|
||||
stop is active from the next bar; a gap through it fills at the open. The stop
|
||||
only ever moves up — the trail keeps working above entry. No other trigger
|
||||
levels are run.
|
||||
|
||||
**Windows.** Train = entries before 2024-07-01, test = entries on/after it, as
|
||||
disjoint books (`_simulate_portfolio` `end_date`/`start_date`). Full history is
|
||||
reported but not graded.
|
||||
|
||||
**Pass rule.** An arm passes only if, in **both** train and test, all three hold
|
||||
versus `control` at report precision (Sharpe 2 dp, CAGR and drawdown 1 dp):
|
||||
|
||||
- Sharpe ≥ control
|
||||
- CAGR ≥ control
|
||||
- max drawdown ≤ control
|
||||
|
||||
Win rate and average R per trade are reported but **not** graded: a breakeven
|
||||
stop raises both mechanically, and it changes trade count, so EV per trade is
|
||||
the wrong lens ([capacity findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens)).
|
||||
|
||||
**Parity guard.** The study also runs `control` at the monitor's capacity 10 on
|
||||
full history. It must equal the `portfolio_monitor` production `all` row
|
||||
(Sharpe, CAGR, drawdown, trades) or the study is measuring a different book and
|
||||
no row is valid.
|
||||
|
||||
**What a pass earns.** The post-2024 window has informed earlier research, so it
|
||||
is not a pristine holdout. A pass earns a forward paper trial only — no
|
||||
production change. A fail closes the question for this snapshot.
|
||||
|
||||
## Reproducing
|
||||
|
||||
`prod.sqlite` predates later ticker/trade columns, so `select(Ticker)` fails on
|
||||
it (see [survivorship-audit.md](survivorship-audit.md#caveats)). Run on a copy
|
||||
synced with `prep_survivorship_snapshot.sync_schema`, which only adds nullable
|
||||
columns and changes no data:
|
||||
|
||||
```bash
|
||||
cp backtest_snapshots/prod.sqlite /tmp/prod-synced.sqlite
|
||||
python -c "import sys; sys.path[:0] = ['.', 'scripts']; from pathlib import Path; \
|
||||
from prep_survivorship_snapshot import sync_schema; sync_schema(Path('/tmp/prod-synced.sqlite'))"
|
||||
BACKTEST_BREAKEVEN_STUDY=1 python scripts/run_backtest_snapshot.py \
|
||||
/tmp/prod-synced.sqlite --cadence daily --workers 7 --allow-spawn \
|
||||
--out reports/backtest-20261009-breakeven-stop.json
|
||||
```
|
||||
|
||||
The graded capacity is read from the snapshot's `shadow_book_capacity` setting.
|
||||
`prod.sqlite` has no such row, so the shadow book's default of 15 applies; the
|
||||
report records it as `capacity` / `capacity_source`.
|
||||
|
||||
## Result — no change; `be_1r` fails, `be_2r` passes only by being the production exit
|
||||
|
||||
Report: `reports/backtest-20261009-breakeven-stop.json` (daily cadence, 1,011,248
|
||||
candidates, 5,449 qualified; 505 of 506 symbols evaluated — RHM has no stored
|
||||
bars, nothing failed). **Parity held exactly:** the capacity-10 control reproduces
|
||||
the monitor's production row (Sharpe 1.84, CAGR 49.8%, DD 18.3%, 465 trades).
|
||||
The report was regenerated on 2026-10-10 after a code review
|
||||
([corrections below](#code-review-corrections-2026-10-10)); every graded number
|
||||
is unchanged.
|
||||
|
||||
Capacity 15, production book:
|
||||
|
||||
| arm | window | Sharpe (SE) | CAGR | max DD | trades | win % | avg R | ≥5R trades | breakeven exits |
|
||||
|---|---|---|---|---|---|---|---|---|---|
|
||||
| control | train | 1.71 (0.68) | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | — |
|
||||
| control | test | 1.46 (0.71) | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | — |
|
||||
| control | full | 1.69 (0.50) | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | — |
|
||||
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 35.0 | 0.595 | 22 | 36 |
|
||||
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 34.4 | 0.455 | 10 | 46 |
|
||||
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 34.9 | 0.539 | 32 | 82 |
|
||||
| `be_2r` | train | 1.71 | 47.3% | 17.8% | 243 | 36.6 | 0.563 | 20 | 1 |
|
||||
| `be_2r` | test | 1.46 | 34.3% | 16.9% | 249 | 35.3 | 0.366 | 12 | 1 |
|
||||
| `be_2r` | full | 1.69 | 44.3% | 18.3% | 480 | 36.0 | 0.477 | 32 | 2 |
|
||||
|
||||
**`be_1r`: FAIL.** Train is worse on all three graded metrics (Sharpe 1.71 → 1.67,
|
||||
CAGR 47.3% → 45.3%, DD 17.8% → 18.5%). Test is much better on return (Sharpe
|
||||
1.46 → 1.71, CAGR +8.8pp) but drawdown is still worse (16.9% → 18.1%). The
|
||||
halves pointing opposite ways is the same signature as the rejected max-hold
|
||||
arms: a regime interaction, not an edge. The verdict rests on the frozen rule;
|
||||
no significance test was run on the gaps. The per-arm Sharpe SEs (~0.7) do not
|
||||
bound them either: the arms share most of their trades, so the gap between two
|
||||
arms is far less noisy than either Sharpe on its own.
|
||||
|
||||
82 of 502 trades ended at the breakeven stop, and the book took 22 more trades
|
||||
than control. Those were not freed slots: the count cap never bound in any
|
||||
capacity-15 row (0 refusals). An earlier exit releases the ticker itself (one
|
||||
position per ticker) and the cash it held. The simulator does not count entries
|
||||
dropped or shrunk for lack of cash, so the split between the two is not
|
||||
measured. Only 4 of the 82 exits were bought back in the same session.
|
||||
|
||||
**`be_2r`: passes the rule on ties, and that pass means nothing.** It changed 2
|
||||
of 480 trades. The reason is arithmetic: the initial stop is 1.5× ATR = 1R, and
|
||||
the trail sits 3× ATR = 2R below the highest close. By the time a trade is +2R
|
||||
up, the trail has already reached entry, unless ATR has widened since entry.
|
||||
`be_2r` is therefore the production exit, with nothing to paper-trade.
|
||||
|
||||
Win rate *fell* under `be_1r` (36.0 → 34.9): an exit at entry is a small loss
|
||||
after the 0.1%/side cost, not a win. Average R rose only because trade count and
|
||||
mix changed — the reason it was not graded.
|
||||
|
||||
**Decision: keep the production exit. The breakeven question is closed for this
|
||||
snapshot.** The trade that prompted it (a roughly 10% run that ended below
|
||||
entry) was not identified, so its R is unknown. With ATR unchanged the trail
|
||||
only reaches entry at +2R; that trade either peaked below +2R of its own risk or
|
||||
saw ATR widen during the run. Either way, the rule that would have saved it
|
||||
(`be_1r`, if it reached +1R) costs the book about as much elsewhere: no reliable
|
||||
improvement in either direction.
|
||||
|
||||
**Side observation, not graded (worth its own look):** on this single daily
|
||||
path, the capacity-15 control (Sharpe 1.69, CAGR 44.3%) is *below* the
|
||||
capacity-10 production row (1.84, 49.8%) at the same 18.3% drawdown. At capacity
|
||||
10 the count cap refused **510** entries; at 15 it refused none. Those two
|
||||
counts do not measure the same thing: at 15, cash plus the 20% notional cap is
|
||||
the binding limit, and the entries it drops or shrinks are not counted. What the
|
||||
path does show is that, here, the tighter 10-slot book did better. The 2026-08-05 capacity decision
|
||||
was made across 175 start-date paths (51 better, 2 worse), so one full-history
|
||||
path disagreeing does not overturn it, but a −5.5pp gap is large enough to
|
||||
deserve its own look.
|
||||
|
||||
## Code-review corrections (2026-10-10)
|
||||
|
||||
A review of the study code found several problems. Fixing them changed no graded
|
||||
number. One of them prompted a diagnostic arm.
|
||||
|
||||
**Re-entry after a breakeven exit.** The first run labeled a breakeven exit
|
||||
`trailing_stop`, because the stop sat above the initial stop. It now has its own
|
||||
label, `breakeven_stop`, and the count is exact rather than inferred from the
|
||||
stop price (82 / 36 / 46, as before). Either label means the same thing for
|
||||
re-entry: the live lock (`trade_policy.get_reentry_gate_locks`) applies only
|
||||
after an initial-stop exit. So under `be_1r` a name stopped at breakeven can be
|
||||
bought again without a gate reset. That is how the rule would behave live, so
|
||||
the graded arms keep it. A diagnostic arm, `be_1r_gated`, sends breakeven exits
|
||||
through the post-stop gate reset as an initial stop would. It was added after
|
||||
the first run, so it is reported and never graded.
|
||||
|
||||
| arm | window | Sharpe | CAGR | max DD | trades | breakeven exits | same-session re-buys |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| control | train | 1.71 | 47.3% | 17.8% | 243 | — | — |
|
||||
| control | test | 1.46 | 34.3% | 16.9% | 249 | — | — |
|
||||
| control | full | 1.69 | 44.3% | 18.3% | 480 | — | — |
|
||||
| `be_1r` | train | 1.67 | 45.3% | 18.5% | 254 | 36 | 1 |
|
||||
| `be_1r` | test | 1.71 | 43.1% | 18.1% | 250 | 46 | 3 |
|
||||
| `be_1r` | full | 1.72 | 45.7% | 18.5% | 502 | 82 | 4 |
|
||||
| `be_1r_gated` | train | 1.72 | 47.1% | 18.0% | 253 | 38 | 0 |
|
||||
| `be_1r_gated` | test | 1.78 | 45.0% | 18.1% | 249 | 46 | 0 |
|
||||
| `be_1r_gated` | full | 1.79 | 47.7% | 18.0% | 500 | 84 | 0 |
|
||||
|
||||
The review suspected part of `be_1r`'s test-window lift came from same-session
|
||||
re-buys. It did not: there were only 4. Gating re-entry after breakeven exits
|
||||
did help `be_1r`, though: it improves Sharpe and CAGR in every window and
|
||||
drawdown in two of three. The gate changes *which* re-entries happen, not just
|
||||
whether a name is re-bought, and this is one post-hoc comparison on one path.
|
||||
The result is consistent with the post-stop re-entry finding, where a gate
|
||||
reset beat immediate re-entry ([post-stop-reentry.md](post-stop-reentry.md)),
|
||||
but it does not establish the mechanism.
|
||||
|
||||
**The decision stands.** `be_1r_gated` would also fail the frozen rule:
|
||||
drawdown is worse than control in both windows (17.8% → 18.0% train,
|
||||
16.9% → 18.1% test), and train CAGR is 47.1% against 47.3%. It was also chosen
|
||||
after seeing the data, so it earns nothing here. If the breakeven question is
|
||||
reopened, this is the arm to pre-register. It needs data the study has not
|
||||
seen, and the live gate lock extended to breakeven exits, so that the backtest
|
||||
and live still agree.
|
||||
|
||||
**Other fixes. None moves a graded number.**
|
||||
|
||||
- A breakeven stop now also applies on a `next_open` fill bar. Before, it fired
|
||||
one bar late there. The trail and breakeven rules now share one function. The
|
||||
study uses close fills, so its rows are unaffected.
|
||||
- `breakeven_r` must be positive and finite, and is rejected for exit policies
|
||||
without an ATR trail. Before, a non-positive value moved the stop to entry on
|
||||
the first bar, and other policies ignored the setting silently.
|
||||
- Each verdict now records `identical_to_control`, which is true for `be_2r`. A
|
||||
pass that is only ties can no longer read as a finding.
|
||||
- The graded capacity comes from the shadow book's capacity config (the
|
||||
`shadow_book_capacity` setting in the database the run reads, or its default
|
||||
of 15 when the row is missing) instead of the `SIM_MAX_POSITIONS` constant. On
|
||||
a snapshot that is the snapshot's copy of the setting, not the current live
|
||||
value ([Reproducing](#reproducing)). `capacity_source` reads
|
||||
`shadow_book_capacity` in both cases.
|
||||
Reference in new issue
Block a user