fix(research): make the calibration artifact live up to its refusal guarantees

Review of the v4 evidence path. The shipped sensors, bands, methodology bump and
categorical allowlist were found sound; these are gaps in the harness that
produced the evidence for them.

The recommendation gates were optional, so they were not gates. The calendar
anchor lived behind --expected-first-session, which defaulted to None -- so the
committed artifact had no first-date check at all, leaving only a session COUNT
that is tautological (the harness slices the tail of the price series to whatever
was asked for). And the state_v4 <= state_v3 invariant was appended only when
both variants were present, so `--methodology v3` alone could still emit a v4
recommendation having never evaluated v4. The anchor is now a published constant
asserted unconditionally, required explicitly whenever --end/--sessions are
overridden, and v3+v4 are mandatory. Both refusals exit 2.

The P1_SCORE_CAP decision was taken on the wrong population. The agreed rule was
"sole price argmax on >80% of sessions with State >= 40"; the harness reported
only all-session counts and the doc concluded from 17/408 = 4.2%. Measured on the
actual population: 47 qualifying sessions, P1 sole argmax on 17 = **36.2%** (P2
16, P3 14). Still well under 80, so the conclusion holds -- but it was reached
from a denominator that did not test the rule, and 36.2% is a materially
different number to have on the page.

Provenance did not identify the code that produced the artifact. It recorded
git_rev c3ae5ad while the live v4 variant depended on app changes that were still
uncommitted, so checking out that revision would not reproduce it. Now records
git_dirty plus sha256 of regime_monitor_service, breadth_service and the script
itself, and this artifact is regenerated from a clean tree.

The 400- vs 700-day OAS question was described as settled but was not
reproducible: the artifact carried only oas_fetch_days 4748, and
v2_reconstruction patches the per-session window to 3653 regardless, so
--oas-window-days 400 could not simulate it. Patching a window cannot stand in
for data that was simply absent, so v2_reconstruction_oas400 truncates the OAS
SOURCE series instead: avg 26.54, p80 42.52, max 100.00 against published
22.6 / 35.1 / 91.2. Full coverage reproduces all three, so the published figures
predate the truncation. Now recorded in the doc.

v4-vix-only and v4-p1-only had become no-ops: after the cutover the shipped
sensors ARE v4, so patching one candidate in left the other shipped and both
variants evaluated full v4. Each now restores the other sensor to its v3 formula,
and they separate properly (v3 18.13, v4-vix-only 16.64, v4-p1-only 16.28,
v4 14.78 -- each fix contributing about half the move).

Docs: the copy-paste invocation was mangled by a backslash-escaping bug and is
now a fenced, forward-slash command; "Unlike P3 and V1 ... P3's do not" corrected
to "Unlike P1 and V1"; the point-in-time section updated from 400 sessions to the
672-calendar-day / ~464-session window production actually replays; the exercised
52.33 VIX print recorded so the top anchors are not merely asserted.

Tests: band_for now pinned at 64.9/65 from both sides so a silent revert to 80
cannot pass, and the categorical carry-forward test stores locked=True and
asserts it survives -- losing it is half the failure mode, since
update_regime_monitor only auto-refreshes when locked is false.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-08 22:46:10 +02:00
co-authored by Claude Opus 5
parent 3143477a62
commit ec1b0acfad
5 changed files with 142 additions and 1510 deletions
+8
View File
@@ -52,6 +52,10 @@ def test_band_for_is_per_axis():
assert band_for(20, STATE_BANDS) == "watch"
assert band_for(50, STATE_BANDS) == "elevated"
assert band_for(80, STATE_BANDS) == "breaking"
# v4 moved the top band 80 -> 65; pin the new boundary from both sides so a
# silent revert cannot pass. band_for is inclusive at the threshold.
assert band_for(64.9, STATE_BANDS) == "elevated"
assert band_for(65, STATE_BANDS) == "breaking"
# Warning's realized range is far narrower, so it gets its own thresholds.
assert band_for(45, STATE_BANDS) == "watch"
assert band_for(45, WARNING_BANDS) == "elevated"
@@ -401,6 +405,7 @@ async def test_v2_observation_survives_the_methodology_bump(monkeypatch, stored_
"source": "gemini",
"fetched_at": "2026-07-24T14:25:47+00:00",
"effective_date": "2026-07-27",
"locked": True,
})
monkeypatch.setattr(rms.settings_store, "get_value", fake_value)
@@ -412,6 +417,9 @@ async def test_v2_observation_survives_the_methodology_bump(monkeypatch, stored_
assert result["effective_date"] == "2026-07-27"
assert result["f1_score"] == 37.5 # recomputed on the current scale, not the stored 0.0
assert result["fetched_at"] == "2026-07-24T14:25:47+00:00" # or a refresh loop starts
# locked is the operator saying "do not overwrite this". Losing it is half the
# failure mode: update_regime_monitor only auto-refreshes when locked is false.
assert result["locked"] is True
@pytest.mark.asyncio