feat(regime): cut the risk monitor to v4 — desaturate VIX and the trend break

Two sensors saturated in exactly the range where resolution matters, and the top
State band had no headroom. Calibrated with scripts/run_regime_monitor_calibration.py
over the 408 sessions ending 2026-07-24; the shipped code reproduces that run's
band shares exactly (78.9 / 13.0 / 4.7 / 3.4).

V1 read VIX 30, 50 and 82 as an identical 100 — the same defect v3 had just
removed from P3, left in place one sensor over. In the window it flattened five
distinct April-2025 prints (52.33, 46.98, 45.31, 40.72, 38.57) into one value.
Now an anchor table reaching full scale at 55, not at 2020's ~82: anchoring the
top at a once-in-a-generation print would make VIX 50 read only ~70. Pegged on
14 of 408 sessions before; none now.

_under_200 returned a bare 0/100, so P1 printed 100 the moment SMH and QQQ were
both under their average — and since the price pillar takes max(P1, P2, P3),
that pinned the pillar and stopped P3's ladder resolving for the whole of a
selloff. Now graded by depth below the 200-DMA, with a deliberate floor of 20 at
the crossing: the break is a genuine binary event, only its depth is graded.
Pegged on 46 of 408 sessions before; none now. A 2% break reads ~30, not 100.

max() was KEPT — the defect was the step function feeding it, not the vote, and
v3's "one capped vote for correlated reads" rationale still holds. P1 is the sole
price argmax on 17 of 408 sessions (4.2%), so the P1_SCORE_CAP fallback drafted
during design was measured as unnecessary and not shipped.

STATE_BANDS breaking 80 -> 65, and only that threshold. Credit returns 0.0 (not
None) when calm, so it holds its 20 points pinned at zero and price + breadth +
volatility at literal maximum summed to exactly 80.0 — v3's threshold to the
decimal, with nothing above it. The sensor is deliberately unchanged: a
calm-credit selloff genuinely is less stressed. What was stale is the band, fit
on v2 while credit's since-removed percentile leg still contributed. A
2022-style AI/tech drawdown with calm credit computes to 70.3 (no death cross) or
74.0 (with one); 70 would have left 0.33 points of headroom, reproducing the
defect. Chosen by scenario arithmetic, and the realized breaking share then lands
on 3.4% — the same as v3's, arrived at independently.

"v4" added to CATEGORICAL_FUNDAMENTAL_METHODOLOGIES in this same commit, which is
load-bearing: that set is checked against the STORED blob, so bumping without it
discards the collected observation on first write, leaving fetched_at null and
locked false — and update_regime_monitor then fires a paid LLM refresh on every
run, forever. Now guarded by a test parametrised over v2 and v3 stored blobs.

SENSOR_REVISION deliberately stays 2: a METHODOLOGY change already forces a full
reseed via _parse_snapshot, and bumping both would imply the reseed was
revision-driven.

QUADRANT_STATE_DIVIDER stays 50 because only breaking moved, so alert_service,
RegimeChart and the quadrant tests need no change. A new test enforces
divider == band boundary on both axes, which nothing did before.

Doc renamed to regime-monitor-v4.md with a tombstone at the old path (commit
messages cite it), the three open questions converted to resolved with the
reasoning that closed them, and indexed in docs/research/README.md for the first
time. The P2 limit is stated honestly: _death_cross pegs at a -5% MA gap, so a
deep selloff still reaches 100 via P2 — v4 repairs the shallow-to-moderate break,
not "the price pillar no longer pegs".

DEPLOY: the first run reseeds ~464 sessions. Expect one phantom quadrant alert
(the dedup key carries basket_hash, not methodology) and re-run the Event Study
manually — its cached report self-invalidates but does not self-regenerate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tests/unit/test_regime_calibration_script.py

@@ -37,13 +37,14 @@ class TestVariantPatching:
     def test_patching_restores_even_when_the_body_raises(self):
         original = rms.p5_volatility
         with pytest.raises(RuntimeError):
-            with calib.patched(calib.VARIANTS["v4"]):
+            with calib.patched(calib.VARIANTS["v3"]):
                 raise RuntimeError("boom")
         assert rms.p5_volatility is original
 
-    def test_v3_variant_patches_nothing(self):
-        """The reproduction gate must run against live code, not a copy."""
-        assert calib.VARIANTS["v3"] == {}
+    def test_v4_variant_patches_nothing(self):
+        """The shipped methodology must be exercised as live code, not a copy,
+        or the harness and the service can drift apart silently."""
+        assert calib.VARIANTS["v4"] == {}
 
 
 class TestCandidateFormulas:
@@ -52,24 +53,24 @@ class TestCandidateFormulas:
         closes = [100.0] * 200
         for last in (105.0, 100.0, 99.0, 92.0, 80.0, 50.0):
             series = closes[:-1] + [last]
-            v4 = calib._candidate_under_200(series)
-            v3 = rms._under_200(series)
+            v4 = rms._under_200(series)          # shipped
+            v3 = calib._v3_under_200(series)      # retired
             assert v4 <= v3, f"close={last}: v4 {v4} > v3 {v3}"
 
     def test_anchored_vix_never_exceeds_the_live_formula(self):
-        p5 = calib._candidate_p5(calib.P5_VIX_ANCHORS_A)
         for vix in (10, 15, 17, 20, 25, 30, 40, 55, 82):
-            assert p5(vix) <= rms.p5_volatility(vix), f"vix={vix}"
+            assert rms.p5_volatility(vix) <= calib._v3_p5_volatility(vix), f"vix={vix}"
 
     def test_the_vix_table_keeps_resolving_past_thirty(self):
-        p5 = calib._candidate_p5(calib.P5_VIX_ANCHORS_A)
-        assert p5(30) < p5(40) < p5(50) < p5(55) == 100.0
-        assert rms.p5_volatility(30) == rms.p5_volatility(82) == 100.0  # the defect
+        assert rms.p5_volatility(30) < rms.p5_volatility(40) < rms.p5_volatility(50)
+        assert rms.p5_volatility(55) == 100.0
+        # the retired formula's defect, kept as the contrast
+        assert calib._v3_p5_volatility(30) == calib._v3_p5_volatility(82) == 100.0
 
     def test_a_shallow_break_no_longer_pegs(self):
         closes = [100.0] * 199 + [98.0]  # ~2% below a flat 200-DMA
-        assert rms._under_200(closes) == 100.0
-        assert calib._candidate_under_200(closes) < 40.0
+        assert calib._v3_under_200(closes) == 100.0   # retired: pegged
+        assert rms._under_200(closes) < 40.0          # shipped: graded
 
     def test_candidate_tables_are_well_formed(self):
         for table in (calib.P5_VIX_ANCHORS_A, calib.P5_VIX_ANCHORS_B,

tests/unit/test_regime_monitor.py

@@ -1,4 +1,4 @@
-"""Pure-function tests for the v3 AI/Tech Risk Monitor contract."""
+"""Pure-function tests for the v4 AI/Tech Risk Monitor contract."""
 
 from __future__ import annotations
 
@@ -172,7 +172,7 @@ def test_relative_strength_flat_or_better_is_zero():
 
 def test_volatility_and_breadth_zero_points():
     assert p5_volatility(15) == 0
-    assert p5_volatility(30) == 100
+    assert p5_volatility(30) == 55
     assert breadth_level_score(60) == 0
     assert breadth_level_score(20) == 100
     assert breadth_level_score(None) is None
@@ -363,7 +363,7 @@ def test_fundamental_api_rejects_numeric_ordinal_overrides():
 
 
 @pytest.mark.asyncio
-async def test_legacy_numeric_fundamentals_do_not_leak_into_v3(monkeypatch):
+async def test_legacy_numeric_fundamentals_do_not_leak_into_v4(monkeypatch):
     async def fake_value(_db, _key):
         return json.dumps({"f1_score": 75.0, "f3_score": 75.0, "source": "manual"})
 
@@ -371,24 +371,29 @@ async def test_legacy_numeric_fundamentals_do_not_leak_into_v3(monkeypatch):
 
     result = await rms.get_fundamental_overrides(object())
 
-    assert result["methodology"] == "v3"
+    assert result["methodology"] == "v4"
     assert result["f1_score"] is None
     assert result["f3_score"] is None
     assert result["good_news_stock_down"] == "mixed"
 
 
 @pytest.mark.asyncio
-async def test_v2_observation_survives_the_methodology_bump(monkeypatch):
+@pytest.mark.parametrize("stored_methodology", ["v2", "v3"])
+async def test_v2_observation_survives_the_methodology_bump(monkeypatch, stored_methodology):
     """A snapshot reseed must not throw away a hand/LLM-collected observation.
 
     The categorical format is unchanged, so the stored capex map is still valid;
     only the capex scale moved, and f1 is recomputed from the categories.
+
+    Parametrised over every methodology that could be sitting in the settings row
+    at cutover time -- "v3" is the one the v4 bump actually meets in production,
+    and losing it would silently start a paid LLM refresh on every run.
     """
     names = DEFAULT_CONFIG["tickers"]["hyperscalers"]
 
     async def fake_value(_db, _key):
         return json.dumps({
-            "methodology": "v2",
+            "methodology": stored_methodology,
             "f1_score": 0.0,  # stale v2 scale, must be recomputed
             "f3_score": 100.0,
             "capex": {names[0]: "raising", **dict.fromkeys(names[1:], "holding")},
@@ -405,7 +410,8 @@ async def test_v2_observation_survives_the_methodology_bump(monkeypatch):
     assert result["source"] == "gemini"
     assert result["good_news_stock_down"] == "yes"
     assert result["effective_date"] == "2026-07-27"
-    assert result["f1_score"] == 37.5  # recomputed on the v3 scale, not the stored 0.0
+    assert result["f1_score"] == 37.5  # recomputed on the current scale, not the stored 0.0
+    assert result["fetched_at"] == "2026-07-24T14:25:47+00:00"  # or a refresh loop starts
 
 
 @pytest.mark.asyncio
@@ -484,7 +490,7 @@ async def test_manual_fundamentals_are_categorical_and_derived(monkeypatch):
 async def test_prior_snapshot_is_immutable_without_explicit_rebuild(db_session):
     snapshot_date = date(2026, 6, 26)
     first = {
-        "methodology": "v3",
+        "methodology": "v4",
         "date": snapshot_date.isoformat(),
         "state": {"score": 10.0, "band": "stable"},
         "warning": {"score": 20.0, "band": "stable"},
@@ -540,7 +546,7 @@ async def test_routine_can_refresh_latest_trading_session_after_civil_day_rolls(
         return {}, {}
 
     async def fake_latest(_db):
-        return object(), {"methodology": "v3", "sensor_revision": rms.SENSOR_REVISION}
+        return object(), {"methodology": "v4", "sensor_revision": rms.SENSOR_REVISION}
 
     async def fake_upsert(_db, result, *, rewrite_existing):
         rewrites.append(rewrite_existing)
@@ -568,9 +574,9 @@ async def test_routine_can_refresh_latest_trading_session_after_civil_day_rolls(
 @pytest.mark.parametrize(
     ("stored", "expect_reseed"),
     [
-        ({"methodology": "v3"}, True),  # written before the marker existed
-        ({"methodology": "v3", "sensor_revision": 1}, True),
-        ({"methodology": "v3", "sensor_revision": rms.SENSOR_REVISION}, False),
+        ({"methodology": "v4"}, True),  # written before the marker existed
+        ({"methodology": "v4", "sensor_revision": 1}, True),
+        ({"methodology": "v4", "sensor_revision": rms.SENSOR_REVISION}, False),
     ],
 )
 async def test_a_stale_sensor_revision_reseeds_stored_history(
@@ -708,6 +714,91 @@ def test_compute_index_uses_one_max_price_vote_and_has_no_combined_score():
     price = next(p for p in result["state"]["pillars"] if p["id"] == "price")
     sensor_scores = [sensor["score"] for sensor in price["sensors"] if sensor["score"] is not None]
     assert price["score"] == max(sensor_scores)
-    assert result["methodology"] == "v3"
+    assert result["methodology"] == "v4"
     assert "combined" not in result
     assert result["basket"]["members_available"] == 25
+
+
+def test_v4_carries_categorical_fundamental_observations():
+    """The costliest failure mode in the v3 -> v4 cut.
+
+    CATEGORICAL_FUNDAMENTAL_METHODOLOGIES is checked against the *stored* blob.
+    Omit the current methodology and the first write discards the observation;
+    the default that replaces it has fetched_at None and locked False, so
+    _fundamentals_stale is true and update_regime_monitor fires a paid LLM
+    refresh on every run, forever, with the operator's locked read gone.
+    """
+    assert rms.METHODOLOGY in rms.CATEGORICAL_FUNDAMENTAL_METHODOLOGIES
+    # Older categorical blobs must still carry forward across the bump.
+    assert {"v2", "v3"} <= rms.CATEGORICAL_FUNDAMENTAL_METHODOLOGIES
+
+
+def test_quadrant_dividers_match_the_band_boundaries():
+    """The doc asserts dividers sit at each axis's watch/elevated boundary.
+
+    Nothing enforced it, and alert_service keeps its own fallback copies -- so a
+    band move could silently leave the alert path classifying on the old grid.
+    """
+    from app.services import alert_service
+
+    assert rms.QUADRANT_STATE_DIVIDER == STATE_BANDS[1]
+    assert rms.QUADRANT_WARNING_DIVIDER == WARNING_BANDS[1]
+    assert alert_service.QUAD_X_DIV == rms.QUADRANT_STATE_DIVIDER
+    assert alert_service.QUAD_Y_DIV == rms.QUADRANT_WARNING_DIVIDER
+
+
+def test_the_vix_sensor_keeps_headroom_past_a_thirty_print():
+    """v3 read VIX 30, 50 and 82 as an identical 100 -- the same saturation v3
+    itself had just removed from P3."""
+    assert p5_volatility(30) < p5_volatility(40) < p5_volatility(50)
+    assert p5_volatility(55) == 100.0
+    assert p5_volatility(82) == 100.0
+    assert p5_volatility(15) == 0.0
+    assert p5_volatility(10) == 0.0
+
+
+def test_a_shallow_trend_break_does_not_peg_the_price_pillar():
+    """v3's binary _under_200 printed 100 the moment price crossed, pinning the
+    pillar's max() and stopping P3's ladder resolving for the whole selloff."""
+    end = date(2026, 6, 26)
+    # ~2% below a flat 200-DMA, with a shallow drawdown to match.
+    flat = [100.0] * 260
+    shallow = flat[:-1] + [98.0]
+    prices = {
+        "SMH": _dated(shallow, end),
+        "QQQ": _dated(shallow, end),
+        "SPY": _dated(flat, end),
+    }
+    result = _compute_index(
+        prices, [(end, 16.0)], [(end, 2.8)], {},
+        copy.deepcopy(DEFAULT_CONFIG), end, [(end, 55.0)], [(end, 0.0)], {end: 30},
+    )
+    price = next(p for p in result["state"]["pillars"] if p["id"] == "price")
+    assert price["score"] < 40.0, "a 2% break must not read as maximum stress"
+    p1 = next(s for s in price["sensors"] if s["id"] == "P1")
+    assert 0.0 < p1["score"] < 40.0
+
+
+def test_anchor_tables_are_well_formed():
+    """Cheap guard against a fat-fingered edit to any interpolation table."""
+    tables = {
+        "P3_DRAWDOWN_ANCHORS": rms.P3_DRAWDOWN_ANCHORS,
+        "P1_TREND_BREAK_ANCHORS": rms.P1_TREND_BREAK_ANCHORS,
+        "P5_VIX_ANCHORS": rms.P5_VIX_ANCHORS,
+    }
+    for name, table in tables.items():
+        xs = [x for x, _ in table]
+        ys = [y for _, y in table]
+        assert xs == sorted(xs) and len(set(xs)) == len(xs), f"{name}: x not increasing"
+        assert ys == sorted(ys), f"{name}: y not non-decreasing"
+        assert 0.0 <= min(ys) and max(ys) <= 100.0, f"{name}: out of [0,100]"
+
+    # Slopes ease off only on the two v4 tables. P3 is deliberately gentle at the
+    # onset then steepens (2.5, 3.75, 3.125, 2.33, 1.83), so it is excluded.
+    for name in ("P1_TREND_BREAK_ANCHORS", "P5_VIX_ANCHORS"):
+        table = tables[name]
+        slopes = [
+            (table[i + 1][1] - table[i][1]) / (table[i + 1][0] - table[i][0])
+            for i in range(len(table) - 1)
+        ]
+        assert all(a >= b for a, b in zip(slopes, slopes[1:])), f"{name}: {slopes}"