Research: S/R levels, the target exit, and the entry gate
Investigated whether our support/resistance detection follows best practice and whether we actually use it that way. Three findings, all backed by runs against the prod snapshot and written up in docs/research/sr-levels-and-exits.md: - The S/R target must NOT become an exit. Honoring it as a take-profit on top of the 3x ATR trail drops Sharpe 2.04 -> 1.47 and halves CAGR. Win rate rises (37.5% -> 40.0%), which is the tell: it truncates the right tail where momentum's edge lives. - The clear-air fallback (synthesize a 3xATR target where no resistance exists, so 52-week-high breakouts stop being vetoed) looked strictly better in-sample (Sharpe 2.04 -> 2.07, CAGR 50.4% -> 62.3%, DD 21.4% -> 20.1%) but FAILED a real out-of-sample holdout: on entries after 2024-07-01 it is worse on Sharpe (2.78 -> 2.45) and Calmar, better only on raw CAGR. Not shipped. - The detector itself is weak vs best practice (POC/VAH/VAL computed then discarded, HVN = any above-mean bin, 1.48x volume double-counting, "touch" counts pass-throughs, no round numbers), but its only causal path to P&L is the entry gate. Fix it for the displayed levels, not for returns. Method note: nested lookback windows are NOT out-of-sample. The in-sample result was clean, large, and consistent across five windows, and still did not survive a proper entry-date split. All research paths are off by default and the default report is unchanged: BACKTEST_RESEARCH_EXITS=1 take-profit exit rows BACKTEST_ATR_TARGET_FALLBACK=k synthetic k*ATR target when S/R offers none BACKTEST_FALLBACK_CLEAR_AIR_ONLY=1 restrict that to genuinely clear air BACKTEST_HOLDOUT_SPLIT=YYYY-MM-DD train/test split by entry date Also fixes two reproducibility holes found while reconciling our local baseline against the live report: - create_backtest_snapshot.py now copies paper_% settings. The production monitor row replays the runtime exit policy via get_exit_policy(); without those keys a snapshot silently falls back to code defaults, so a live-tuned exit would never be reflected. - Migration 020 drops activation_min_expected_value and activation_min_target_probability. Both are orphans of the June EV-gate redesign, read by no code path, but prod carries min_target_probability = 50.0 which implies a probability floor that is not enforced (the real floor is the 20% constant in qualification.py). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,9 +1,9 @@
|
||||
"""Create a minimal local SQLite snapshot for offline backtest research.
|
||||
|
||||
Copies only the data required by app.services.backtest_service.run_backtest:
|
||||
tickers, OHLCV bars, SPY benchmark closes, and activation/recommendation
|
||||
settings. Other system settings are intentionally skipped to avoid copying
|
||||
secrets into local snapshot files.
|
||||
tickers, OHLCV bars, SPY benchmark closes, and the activation / recommendation /
|
||||
paper-exit settings the run reads. Other system settings are intentionally
|
||||
skipped to avoid copying secrets into local snapshot files.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -144,6 +144,12 @@ async def _main() -> None:
|
||||
where=or_(
|
||||
SystemSetting.key.like("activation_%"),
|
||||
SystemSetting.key.like("recommendation_%"),
|
||||
# The production portfolio-monitor row replays the RUNTIME
|
||||
# exit policy via get_exit_policy(). Without these keys a
|
||||
# snapshot silently falls back to the code defaults, so a
|
||||
# live-tuned exit would not be reflected — the snapshot run
|
||||
# would disagree with prod and give no hint why.
|
||||
SystemSetting.key.like("paper_%"),
|
||||
),
|
||||
),
|
||||
"benchmark_prices": await _copy_table(source, dest, BenchmarkPrice, batch_size=args.batch_size),
|
||||
|
||||
Reference in New Issue
Block a user