Add full-period S/R production comparison

This commit is contained in:
2026-07-13 09:37:39 +02:00
parent 3280ed71f7
commit e6dc74df6f
14 changed files with 119 additions and 2575176 deletions
+22
View File
@@ -620,6 +620,28 @@ two-arm residual matrix therefore holds the clean detector and range factor fixe
.venv/bin/python scripts/run_sr_v2_matrix.py residual --workers 14
```
### Full-period production comparison
After freezing `rewrite_range504_structural_legacy_primary`, run it beside a
fresh `production_control` over the complete snapshot with identical portfolio
and exit settings:
```bash
.venv/bin/python scripts/run_sr_v2_matrix.py full --workers 14
```
The command deliberately supplies neither `--entry-start` nor `--entry-end`.
It writes both audited reports plus a paired cohort comparison:
- `reports/backtest-sr-full-production_control.json`
- `reports/backtest-sr-full-rewrite_range504_structural_legacy_primary.json`
- `reports/sr-full-production-vs-candidate-cohorts.csv`
- `reports/sr-full-production-vs-candidate-comparison.json`
This is an apples-to-apples full-history diagnostic against the current live
production path. It is not a new untouched holdout because the post-2024 data
was already inspected while isolating the range factor.
The post-2024 window has been opened and is now analysis data, not a valid final
promotion holdout. These arms can isolate mechanism, but neither may ship without
new future data or a separately pre-registered walk-forward protocol.
+43
View File
@@ -0,0 +1,43 @@
# Backtest report index
Reports dated 2026-07-11 or earlier are the historical production research
record and are intentionally left untouched.
The retained 2026-07-12/13 S/R reports mark a decision or provide the matching
control for one:
- The initial full-period clean-detector regression,
`backtest-20260712-sr-detector-rewrite.json`, remains a local-only report and
is intentionally not added to version control.
- `backtest-sr-v2-train-production_control.json`: pre-July-2024 production
control shared by the detector matrices.
- `backtest-sr-v2-train-rewrite.json` and
`backtest-sr-v2-train-rewrite_legacy_primary.json`: isolate detector geometry
from the primary-target selector.
- `backtest-sr-v2-train-rr_aligned_control.json` and
`backtest-sr-v2-validation-rr_aligned_control.json`: show why primary target
selection must not simply inherit the 2.0 activation floor.
- `backtest-sr-traffic-train-legacy_traffic_grid_only.json`: reproduces the
accidental legacy grid edge.
- `backtest-sr-traffic-train-legacy_range_grid_neutral.json`: proves that the
grid effect does not require volume or touch strength.
- `backtest-sr-v2-validation-production_control.json` and
`backtest-sr-v2-validation-legacy_range_grid_neutral.json`: the post-2024
control and failed range-grid replication. The compact paired result remains
in `sr-v2-validation-comparison.json` and `sr-v2-validation-cohorts.csv`.
- `backtest-sr-range-factor-train-production_range504.json` and
`backtest-sr-range-factor-train-rewrite_range504_legacy_primary.json`: isolate
the explicit 504-day multiplicative-range factor.
- `backtest-sr-range-residual-train-rewrite_range504_structural_legacy_primary.json`
and `backtest-sr-range-residual-train-rewrite_range504_structural_primary2.json`:
the final target-source and primary-floor decision.
The full-period production comparison writes:
- `backtest-sr-full-production_control.json`
- `backtest-sr-full-rewrite_range504_structural_legacy_primary.json`
- `sr-full-production-vs-candidate-comparison.json`
- `sr-full-production-vs-candidate-cohorts.csv`
Those outputs should be retained as the final decision point for this research
branch.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+47 -6
View File
@@ -42,6 +42,10 @@ RANGE_RESIDUAL_ARMS = (
"rewrite_range504_structural_legacy_primary",
"rewrite_range504_structural_primary2",
)
FULL_PERIOD_ARMS = (
"production_control",
"rewrite_range504_structural_legacy_primary",
)
def _add_common(parser: argparse.ArgumentParser) -> None:
@@ -91,6 +95,11 @@ def _args() -> argparse.Namespace:
default=None,
help="Run one residual target-geometry arm without repeating the pair.",
)
full = commands.add_parser(
"full",
help="Run current production and the frozen candidate over the full snapshot.",
)
_add_common(full)
validate = commands.add_parser(
"validate",
help="Run production control and one locked arm from 2024-07-01.",
@@ -105,23 +114,27 @@ def _run_arm(
snapshot: str,
workers: int,
*,
entry_flag: str,
entry_date: str,
entry_flag: str | None = None,
entry_date: str | None = None,
output: Path,
) -> None:
print(f"Running S/R arm: {arm}", flush=True)
subprocess.run(
[
command = [
sys.executable,
str(RUNNER),
snapshot,
"--workers", str(workers),
"--allow-spawn",
"--sr-variant", arm,
entry_flag, entry_date,
"--sr-audit",
"--out", str(output),
],
]
if entry_flag is not None and entry_date is not None:
command.extend((entry_flag, entry_date))
elif entry_flag is not None or entry_date is not None:
raise ValueError("entry_flag and entry_date must be provided together")
subprocess.run(
command,
cwd=ROOT,
check=True,
)
@@ -171,6 +184,32 @@ def _validate(args: argparse.Namespace) -> None:
)
def _full(args: argparse.Namespace) -> None:
reports: dict[str, Path] = {}
for arm in FULL_PERIOD_ARMS:
output = ROOT / "reports" / f"backtest-sr-full-{arm}.json"
reports[arm] = output
_run_arm(
arm,
args.snapshot,
args.workers,
output=output,
)
subprocess.run(
[
sys.executable,
str(COMPARE),
str(reports["production_control"]),
str(reports["rewrite_range504_structural_legacy_primary"]),
"--out-csv", str(ROOT / "reports" / "sr-full-production-vs-candidate-cohorts.csv"),
"--out-json", str(ROOT / "reports" / "sr-full-production-vs-candidate-comparison.json"),
],
cwd=ROOT,
check=True,
)
print("Full-period production comparison complete.")
def main() -> None:
args = _args()
if args.command == "train":
@@ -184,6 +223,8 @@ def main() -> None:
elif args.command == "residual":
arms = (args.only_arm,) if args.only_arm else RANGE_RESIDUAL_ARMS
_train(args, arms, "backtest-sr-range-residual-train")
elif args.command == "full":
_full(args)
else:
_validate(args)