dennisthiessenandClaude Opus 5
c3ae5ad949
v3 was calibrated by replaying the series offline, but that harness was never
committed -- so its published numbers could not be re-derived, and a v4 cut would
have had to choose anchors by argument rather than measurement. This is that
harness, and it reproduces the published figures.
scripts/run_regime_monitor_calibration.py replays State/Warning session by
session from the same inputs the live job uses (Alpaca for all 33 symbols, FRED
for VIX and HY OAS), with no database: breadth and divergence come from
breadth_service's pure helpers. It never reimplements an unchanged live sensor --
_compute_index, _score_pillars, P2, P4 and the Warning sensors are imported and
called. Only candidate formulas (proposed v4) and retired ones (v2, gone from the
codebase) are defined here and patched onto the module for a variant's duration.
Reproduction of the 408 sessions ending 2026-07-24, against the figures in
docs/research/regime-monitor-v3.md:
v2 State avg 22.6 -> 22.68
v2 State p80 35.1 -> 35.1 exact
v2 State max 91.2 -> 91.2 exact
v2 P3 pegged 39 -> 39 exact
v2 W1 live 108 -> 108 exact
v3 State max 87.4 -> 87.4 exact
v3 band shares 73.3/15.0/8.3/3.4 -> 73.0/15.4/8.1/3.4
Three things the harness had to get right to reach that, each of which was
initially wrong and caught by a gate rather than by inspection:
- "W1 live 108" counts NONZERO sessions, not non-null ones. v2's divergence
gate returned 0.0 during any decline (v3 tapers instead), so the retired
divergence formula had to be reconstructed too.
- v2 sliced HY_OAS_REFERENCE_YEARS = 10.0 per session, not v3's 700 days. The
percentile leg ranks against that window, so replaying it short shifted the
middle of the distribution while leaving the max exact.
- The published v2 numbers correspond to FULL OAS coverage. Replaying v2 with
the 400-calendar-day fetch it shipped with yields max 100.0 and 133
credit-less sessions -- so that truncation was not in force when the figures
were taken. Recorded rather than assumed.
The script refuses to emit a band recommendation unless every hard gate passes
(33 symbols fetched, per-symbol warm-up and final bar, full basket on every
session, calendar anchors, 100% coverage, and a row-wise state_v4 <= state_v3
invariant), and exits non-zero. It is meant to be structurally impossible to read
a calibration result out of a run whose pipeline did not validate. No v4 code
ships in this commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2.5 KiB
2.5 KiB
Regime Monitor v4 calibration
Generated 2026-08-08T20:02:03 at f22313d, 2024-12-05 → 2026-07-24.
Hard gates
| gate | expected | measured | |
|---|---|---|---|
| symbols_fetched | 33 | 33 | ok |
| per_symbol_warmup_252_bars | all | 33 | ok |
| per_symbol_reaches_last_session | 2026-07-24 | 33 | ok |
| breadth_counts_full_basket | 30 | 408/408 sessions | ok |
| sessions_scored | 408 | 408 | ok |
| last_scored_date | 2026-07-24 | 2026-07-24 | ok |
| w1_available_every_session | 408 | 408 | ok |
| state_coverage_100_every_row | 0 | 0 | ok |
| no_stale_inputs | 0 | 0 | ok |
| state_v4_le_v3_every_row | 0 | 0 | ok |
Distributions
| variant | avg | median | p80 | p90 | max |
|---|---|---|---|---|---|
| v2_reconstruction | 22.68 | 16.15 | 35.1 | 65.63 | 91.2 |
| v3 | 18.13 | 9.1 | 31.36 | 65.0 | 87.4 |
| v4 | 14.78 | 8.35 | 21.7 | 43.63 | 83.5 |
| v4-vix-b | 15.24 | 8.55 | 22.62 | 44.33 | 83.6 |
| v4-p1-capped | 14.73 | 8.35 | 21.7 | 43.63 | 80.1 |
Saturation census (sessions pegged at 100)
| variant | P1 | P2 | P3 | V1 |
|---|---|---|---|---|
| v2_reconstruction | 46 | 0 | 39 | 14 |
| v3 | 46 | 0 | 0 | 14 |
| v4 | 0 | 0 | 0 | 0 |
| v4-vix-b | 0 | 0 | 0 | 0 |
| v4-p1-capped | 0 | 0 | 0 | 0 |
Reproduction gates — v2_reconstruction
| figure | published | measured | |
|---|---|---|---|
| v2_state_avg | 22.6 | 22.68 | ok |
| v2_state_p80 | 35.1 | 35.1 | ok |
| v2_state_max | 91.2 | 91.2 | ok |
| v2_p3_pegged | 39 | 39 | ok |
| w1_live_sessions | 108 | 108 | ok |
Reproduction gates — v3
| figure | published | measured | |
|---|---|---|---|
| v3_state_max | 87.4 | 87.4 | ok |
v4 band-share grid (watch 20 / elevated 50)
| breaking | stable | watch | elevated | breaking |
|---|---|---|---|---|
| 60 | 78.9 | 13.0 | 2.9 | 5.1 |
| 65 | 78.9 | 13.0 | 4.7 | 3.4 |
| 70 | 78.9 | 13.0 | 6.9 | 1.2 |
Scenarios (pillar arithmetic, explicit sensor scores)
| scenario | price | breadth | C1 | V1 | State |
|---|---|---|---|---|---|
| S1 ordinary tape | 7.5 | 0.0 | 0.0 | 4.0 | 3.6 |
| S2 10% correction, calm credit | 31.25 | 62.5 | 0.0 | 34.4 | 33.28 |
| S3a 2022-style, calm credit, no death cross | 90.83 | 100.0 | 0.0 | 60.0 | 70.33 |
| S3b 2022-style, calm credit, death cross | 100.0 | 100.0 | 0.0 | 60.0 | 74.0 |
| S4 credit event on top | 100.0 | 100.0 | 75.0 | 86.67 | 93.0 |
| S5 March 2020, everything pegged | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 |
Recommendation: {'state_bands_candidate': [20.0, 50.0, 65.0], 'provisional': False, 'note': 'confirm against band_grid + scenarios before shipping'}