dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Finalize GTL and retire S/R research harness

Browse Source
This commit is contained in:
dennisthiessen
2026-07-13 17:58:04 +02:00
parent 9d362bd568
commit bee5a5ce89
35 changed files with 374 additions and 5385779 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/research/sr-levels-and-exits.md
+27 -89
View File
@@ -397,11 +397,10 @@ Note `--allow-spawn` is required on Windows: `_mp_context()` has no `fork`/
large, consistent across five nested windows — and still didn't survive a holdout.
Nested lookbacks are not out-of-sample. Split by entry date before believing anything.
## 7. S/R v2 research harness (implementation started 2026-07-12)
## 7. Archived S/R v2 investigation (2026-07-12/13)
The detector rewrite is decomposed into causal, research-only arms. The live
scanner does not read `BACKTEST_SR_VARIANT`; these switches exist only in the
offline snapshot harness:
The detector rewrite was decomposed into the causal arms below. They are names
in the historical experiment record, not supported runtime configuration:
| arm | behavior |
|---|---|
@@ -412,26 +411,10 @@ offline snapshot harness:
| `confirmed_rounds` | soft zones; standalone rounds need two rejection clusters |
| `gate_v2` | confirmed rounds plus uncapped gate evidence |
Detector evidence (`sources`, rejection count, last rejection age) stays in the
pure backtest objects. It is deliberately not migrated into the production DB
schema until a variant passes validation.
The cross-platform matrix runner is only an orchestrator around the existing
`run_backtest_snapshot.py`; it contains no duplicate backtest logic. On macOS:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py train --workers 14
```
Choose one arm and record that lock before running exactly control and that arm:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py validate \
--locked-arm confirmed_rounds --workers 14
```
Replace `confirmed_rounds` with the recorded winner. The validation command also
calls `scripts/compare_sr_variants.py` to produce the paired cohort CSV and JSON.
The matrix runner, comparator, environment switch, and candidate-level audit
were removed when the investigation closed. The compact comparison JSONs,
cohort CSVs, and this narrative retain the decisions; Git history retains the
raw implementation and reports for forensic reconstruction.
> Validation result: `confirmed_rounds` is rejected and is no longer a lockable
> arm. It remains in the corrected training matrix only to preserve the causal
@@ -493,14 +476,8 @@ changes one legacy component at a time:
- `legacy_pivots_only`: unfiltered full-history pivots, no VP grid;
- `legacy_traffic_grid_only`: deployed HVN+LVN grid, no pivots.
Run on macOS:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py traffic --workers 14
```
Do not validate any traffic arm yet. First establish whether geometry, pivots, or
the range-occupancy grid reproduces production on pre-2024 training data.
The traffic matrix first established whether geometry, pivots, or the
range-occupancy grid reproduced production on pre-2024 training data.
Corrected training result:
@@ -535,16 +512,6 @@ Two final training arms isolate the first two items explicitly:
- `legacy_range_grid_neutral`: identical centers, strength fixed at 50 after
clustering.
Run only these new arms on macOS:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py traffic \
--only-arm legacy_range_grid_touch --workers 14
.venv/bin/python scripts/run_sr_v2_matrix.py traffic \
--only-arm legacy_range_grid_neutral --workers 14
```
The touch arm reproduced `legacy_traffic_grid_only` exactly: all 121,464
candidates, 504 qualified setups, cohort membership, expectancy, and portfolio
metrics match. Volume contributes nothing. Neutral strength won the training
@@ -583,12 +550,6 @@ effect:
- `rewrite_range504_legacy_primary`: clean targets, frozen primary selection,
plus the identical range gate.
Run on pre-2024 training data only:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py factor --workers 14
```
Result:
| training arm | qualified | Sharpe | CAGR | MaxDD | net avg R | ex-top-5% |
@@ -620,27 +581,13 @@ two-arm residual matrix therefore holds the clean detector and range factor fixe
- `rewrite_range504_structural_primary2`: identical, but select the primary from
targets clearing 2.0R.
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py residual --workers 14
```
### Full-period production comparison
After freezing `rewrite_range504_structural_legacy_primary`, run it beside a
fresh `production_control` over the complete snapshot with identical portfolio
and exit settings:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py full --workers 14
```
The command deliberately supplies neither `--entry-start` nor `--entry-end`.
It writes both audited reports plus a paired cohort comparison:
- `reports/backtest-sr-full-production_control.json`
- `reports/backtest-sr-full-rewrite_range504_structural_legacy_primary.json`
- `reports/sr-full-production-vs-candidate-cohorts.csv`
- `reports/sr-full-production-vs-candidate-comparison.json`
The frozen `rewrite_range504_structural_legacy_primary` candidate was run beside
a fresh `production_control` over the complete snapshot with identical
portfolio and exit settings. The retained decision files are
`reports/sr-full-production-vs-candidate-comparison.json` and its cohort CSV;
the redundant full candidate-row reports were removed after consolidation.
This is an apples-to-apples full-history diagnostic against the current live
production path. It is not a new untouched holdout because the post-2024 data
@@ -667,13 +614,7 @@ overlay_rank = 95% * production_80_20_rank + 5% * structural_confirmation
There is deliberately no weight sweep and no union with clean-only setups. The
report must first reproduce the production qualified count and production book;
otherwise the comparison is invalid. Run the one-arm full-period diagnostic:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py overlay --workers 14
```
Output: `reports/backtest-sr-overlay-full.json`. The candidate advances only if
otherwise the comparison is invalid. The candidate advances only if
the overlay improves full-period Sharpe, does not worsen drawdown, and retains
at least 90% of production CAGR. The 1-year and 6-month rows must not both
deteriorate. This remains contaminated full-history research, not promotion
@@ -764,14 +705,10 @@ crossings, capped strength, side and source. The overlay is off by default,
loaded only on demand from `GET /gate-target-ladder/{symbol}`, and must remain
visually distinct from persisted Structural S/R.
The `explicit_target_ladder` arm therefore replaces only the irrelevant volume
pass with the complete range grid. It retains pivots, touch strength, merge
geometry, primary selection, qualification, ranking, and exit behavior. Grid
levels are labelled `range_grid`, making the internal purpose explicit.
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py ladder --workers 14
```
The production GTL replaces only the irrelevant volume pass with the complete
range grid. It retains pivots, touch strength, merge geometry, primary
selection, qualification, ranking, and exit behavior. Grid levels are labelled
`range_grid`, making the internal purpose explicit.
The arm advances only on exact parity with the full-period production control:
no added or removed qualified setups and identical production-book Sharpe,
@@ -793,7 +730,7 @@ human-facing S/R for charts and alerts. The scanner instead builds
for the current scan, and never writes those proposal levels to `SRLevel`. The
primary-target selector retains its independently researched 1.5 floor; the
later live activation gate remains 2.0, and the ATR-trailing exit is unchanged.
The `explicit_target_ladder` backtest arm calls the same pure helper as the live
The `production_gtl` backtest model calls the same pure helper as the live
scanner, so the final full-period rerun is an implementation-parity check rather
than another detector experiment.
@@ -912,15 +849,16 @@ model. This snapshot is now exhausted for GTL fitting; any future challenger
must be pre-registered and evaluated on genuinely new forward data rather than
another iteration over the same history.
The scheduled/Admin online backtest and the unconfigured local snapshot
backtest therefore default to `explicit_target_ladder`. Structural detector
variants remain explicit research overrides. After deployment, rerun the Admin
backtest once to replace any cached report produced by the former `rewrite`
default.
The scheduled backtest, Admin UI, and local snapshot runner therefore default
to `production_gtl`. A manual UI/local run may select `structural_sr` as an
explicit comparison; every report records the model and whether it is the
production path. After deployment, rerun the Admin backtest once to replace any
cached report produced by the former default.
The temporary GTL matrix scripts, configurable detector branches, and
confirmation hooks were removed after this decision. The normal snapshot
backtester and frozen `explicit_target_ladder` parity arm remain.
backtester now exposes only the production GTL and clean Structural S/R
comparison models.
The post-2024 window has been opened and is now analysis data, not a valid final
promotion holdout. These arms can isolate mechanism, but neither may ship without