fix(regime): reseed stored history on a sensor change, and stop faking an observation

Two review findings on 46ace50.

[P1] Raising HY_OAS_WINDOW_DAYS to 700 only reached newly computed rows. A
routine run recomputes the latest trading date alone, and `rebuilding` was keyed
on "no v3 snapshot exists at all", which is false once the cutover has run --
so every row already written kept the credit gap the wider window exists to
close, indefinitely.

Adds SENSOR_REVISION: stamped into each snapshot, absent on pre-marker rows
(read as 1), and a stored revision below the current one triggers exactly one
reseed. Deliberately not METHODOLOGY, which would partition the history API and
discard the cached event study -- neither warranted, since the study recomputes
its Warning series from source rather than reading snapshots and so cannot be
staled by a reseed.

The reseed is bounded by REBUILD_LOOKBACK_DAYS in calendar days rather than a
session count, because the binding constraint is the OAS fetch: each replayed
row needs W3's 20-business-day lookback inside HY_OAS_WINDOW_DAYS. Replaying by
session count would have left the oldest stored rows unrepaired -- the exact
rows the fix targets. At 672 days the replay covers ~464 sessions, W3's oldest
requirement lands on the first fetched OAS day, and the ~400-session series the
cutover wrote is fully covered. A test asserts that relationship so the two
constants cannot drift back into recreating the gap.

[P2] With nothing ever collected, current_observation returned available=true
and the default placeholders -- "unknown" for every hyperscaler, "mixed" for the
reaction -- so the card announced a reading that never happened. Those are the
absence of an observation, not an observation of absence. Gated on `observed`
(non-null fetched_at, the one field every path writing real content stamps),
which blanks the content and drives a proper empty state naming where an admin
collects one. This was a regression from 46ace50; fundamental_overlay never had
it, since no observation means no effective date means pending.

Also renames the leftover v2 identifiers in the touched paths
(rewrite_existing_v2, latest_v2).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app/services/regime_monitor_service.py

@@ -52,7 +52,14 @@ METHODOLOGY = "v3"
 # Snapshots are reseeded on a methodology bump, but fundamental observations are
 # collected by hand/LLM and carried across it when the format is compatible.
 CATEGORICAL_FUNDAMENTAL_METHODOLOGIES = frozenset({"v2", "v3"})
-REBUILD_SESSIONS = 400
+
+# Bumped when a fix changes what historical rows *should* contain without
+# changing the live formula, so stored history needs one reseed.  Deliberately
+# not METHODOLOGY: that partitions the history API and discards the cached event
+# study, neither of which is warranted here -- the study recomputes its Warning
+# series from source rather than reading snapshots, so a reseed cannot stale it.
+# Snapshots written before this marker existed carry no key and read as 1.
+SENSOR_REVISION = 2
 MIN_COVERAGE = 75.0
 SOURCE_MAX_LAG_DAYS = 7
 
@@ -91,6 +98,14 @@ HY_OAS_STRESSED = 7.0
 # live scores are unchanged and this needs no methodology bump.  Stays under
 # ICE's ~3-year cap so FRED still honours the request.
 HY_OAS_WINDOW_DAYS = 700
+
+# A rebuild replays every session inside this window.  Bounded by calendar days
+# rather than a session count because the binding constraint is the OAS fetch:
+# each replayed row needs W3's 20-business-day lookback (~28 calendar days)
+# inside HY_OAS_WINDOW_DAYS, so replaying further back would recreate the exact
+# credit gap a reseed exists to close.  672 days is ~464 trading sessions, which
+# comfortably covers the 400-session series the v3 cutover wrote.
+REBUILD_LOOKBACK_DAYS = HY_OAS_WINDOW_DAYS - 28
 W3_OAS_LOOKBACK = 20
 W3_OAS_FULL_SCALE_PCT = 35.0
 
@@ -536,19 +551,27 @@ def current_observation(overrides: dict, config: dict, as_of: date) -> dict:
     published number; the stored snapshot keeps the gate.
     """
     effective, pending, age, stale = _overlay_timing(overrides, config, as_of)
+    # The default override carries "unknown"/"mixed" placeholders for every
+    # hyperscaler.  Those are the absence of an observation, not an observation
+    # of absence, and must never be presented as collected.  ``fetched_at`` is
+    # the collection timestamp and is the only field written on every path that
+    # produces real content (LLM refresh and manual save both stamp it).
+    observed = bool(overrides.get("fetched_at"))
     return {
+        "observed": observed,
         # Live availability is about usefulness, not effectiveness: a pending
-        # observation is the freshest thing we have.
-        "available": not stale,
+        # observation is the freshest thing we have -- but nothing collected is
+        # never available.
+        "available": observed and not stale,
         "pending": pending,
         "stale": stale,
         "effective_date": effective.isoformat() if effective else None,
         "age_days": age,
-        "capex": overrides.get("capex"),
-        "good_news_stock_down": overrides.get("good_news_stock_down"),
-        "capex_stress": overrides.get("f1_score"),
-        "earnings_stress": overrides.get("f3_score"),
-        "reasoning": overrides.get("reasoning"),
+        "capex": overrides.get("capex") if observed else None,
+        "good_news_stock_down": overrides.get("good_news_stock_down") if observed else None,
+        "capex_stress": overrides.get("f1_score") if observed else None,
+        "earnings_stress": overrides.get("f3_score") if observed else None,
+        "reasoning": overrides.get("reasoning") if observed else None,
         "source": overrides.get("source"),
         "fetched_at": overrides.get("fetched_at"),
     }
@@ -680,6 +703,8 @@ def _compute_index(
 
     return {
         "methodology": METHODOLOGY,
+        # Not part of the history filter -- only the reseed trigger.
+        "sensor_revision": SENSOR_REVISION,
         "date": as_of.isoformat(),
         "state": state,
         "warning": warning,
@@ -939,7 +964,7 @@ async def _upsert_snapshot(
     db: AsyncSession,
     result: dict,
     *,
-    rewrite_existing_v2: bool,
+    rewrite_existing: bool,
 ) -> tuple[bool, dict]:
     snapshot_date = date.fromisoformat(result["date"])
     existing = await db.execute(select(RegimeSnapshot).where(RegimeSnapshot.date == snapshot_date))
@@ -956,15 +981,23 @@ async def _upsert_snapshot(
             created_at=datetime.now(timezone.utc),
         ))
     else:
-        existing_v2 = _parse_snapshot(row.breakdown_json)
-        if existing_v2 is not None and not rewrite_existing_v2:
-            return False, existing_v2
+        existing_parsed = _parse_snapshot(row.breakdown_json)
+        if existing_parsed is not None and not rewrite_existing:
+            return False, existing_parsed
         row.total_score = float(state_score or 0.0)
         row.band = state_band or "unavailable"
         row.breakdown_json = payload
     return True, result
 
 
+def _snapshot_revision(snapshot: dict) -> int:
+    """Sensor revision of a stored snapshot; pre-marker rows read as 1."""
+    try:
+        return int(snapshot.get("sensor_revision") or 1)
+    except (TypeError, ValueError):
+        return 1
+
+
 def _parse_snapshot(raw: str) -> dict | None:
     try:
         parsed = json.loads(raw)
@@ -984,7 +1017,9 @@ async def _latest_snapshot_row(db: AsyncSession) -> tuple[RegimeSnapshot, dict]
     return None
 
 
-async def update_regime_monitor(db: AsyncSession, rebuild_sessions: int = REBUILD_SESSIONS) -> dict:
+async def update_regime_monitor(
+    db: AsyncSession, rebuild_lookback_days: int = REBUILD_LOOKBACK_DAYS
+) -> dict:
     config = await get_regime_config(db)
     overrides = await get_fundamental_overrides(db)
     if _fundamentals_stale(overrides, config) and not overrides.get("locked"):
@@ -1018,10 +1053,18 @@ async def update_regime_monitor(db: AsyncSession, rebuild_sessions: int = REBUIL
         logger.warning("Regime monitor: fixed-basket breadth skipped: %s", exc)
         breadth, breadth_counts, divergence = {}, {}, {}
 
-    latest_v2 = await _latest_snapshot_row(db)
-    rebuilding = latest_v2 is None and bool(leader_series)
+    latest_snapshot = await _latest_snapshot_row(db)
+    # A stored series written under an older sensor revision is reseeded once.
+    # Without this, raising HY_OAS_WINDOW_DAYS would only ever reach newly
+    # computed rows: routine runs touch the latest date alone, so every older row
+    # would keep the credit gap indefinitely.
+    rebuilding = bool(leader_series) and (
+        latest_snapshot is None
+        or _snapshot_revision(latest_snapshot[1]) < SENSOR_REVISION
+    )
     if rebuilding:
-        dates = [d for d, _ in leader_series[-max(1, rebuild_sessions):]]
+        floor = end - timedelta(days=rebuild_lookback_days)
+        dates = [d for d, _ in leader_series if d >= floor] or [latest_date]
     else:
         # Routine PIT rule: only the latest trading date may be inserted/updated.
         dates = [latest_date]
@@ -1045,7 +1088,9 @@ async def update_regime_monitor(db: AsyncSession, rebuild_sessions: int = REBUIL
         written, latest_result = await _upsert_snapshot(
             db,
             computed,
-            rewrite_existing_v2=rebuilding or snapshot_date == latest_date,
+            # True for *every* replayed date on a reseed, or it would write one
+            # row and leave the rest at the old revision.
+            rewrite_existing=rebuilding or snapshot_date == latest_date,
         )
         snapshots_written += int(written)
     await db.commit()

docs/research/regime-monitor-v3.md

@@ -154,6 +154,14 @@ and refreshing appeared to do nothing. That was the opposite of what this sectio
 already claimed. Showing it early cannot leak into a published number, because
 nothing in the overlay is scored (see "Fundamentals left the score").
 
+`current_observation` gates on `observed` (a non-null `fetched_at`, the one field
+every path writing real content stamps). Without it, the default override —
+`unknown` for every hyperscaler and `mixed` for the reaction — was reported as a
+live observation with `available: true`, so the card presented placeholders as a
+collected reading. Those are the absence of an observation, not an observation of
+absence. `fundamental_overlay` never had this problem: no observation means no
+effective date, which means `pending`, which already blanks the content.
+
 Each snapshot stores the fixed basket symbols, hash, and freeze date.
 Reconstructed history before that freeze date is retrospective/exploratory.
 
@@ -276,6 +284,27 @@ the widened request, not new upstream history — and it makes the chip a better
 truncation canary, since a 700-day request returning ~1095 days' worth is now
 the visible ceiling.
 
+**Widening the window alone does not repair stored history.** Routine runs
+recompute only the latest trading date, and `rebuilding` was keyed on "no v3
+snapshot exists at all" — which is false once the cutover has run — so every row
+already written would have kept its credit gap indefinitely. `SENSOR_REVISION`
+fixes that: it is stamped into each snapshot, snapshots predating it read as 1,
+and a stored revision below the current one triggers exactly one reseed.
+
+It is deliberately not `METHODOLOGY`. That constant partitions the history API
+and discards the cached event study; neither is warranted here, because the study
+recomputes its Warning series from source (`_warning_series` calls
+`warning_sensor_scores` against freshly fetched prices and OAS) rather than
+reading snapshots, so a reseed cannot stale it.
+
+The reseed is bounded by `REBUILD_LOOKBACK_DAYS` in calendar days rather than a
+session count, because the binding constraint is the OAS fetch: each replayed row
+needs W3's 20-business-day lookback inside `HY_OAS_WINDOW_DAYS`. At 672 days the
+replay reaches ~464 sessions, W3's oldest requirement lands exactly on the first
+fetched OAS day, and the ~400-session series the v3 cutover wrote is fully
+covered. A test asserts that relationship so the two constants cannot drift into
+recreating the gap.
+
 The fix was sequenced deliberately: acting on items 1–3 above bumps
 `METHODOLOGY`, which fires `rebuilding`, which would have baked the credit-less
 rows into the fresh series. Fixing the window afterwards would mean reseeding

frontend/src/lib/types.ts

@@ -500,6 +500,9 @@ export interface RegimeFundamentalOverlay {
   reasoning: string | null;
   source: string | null;
   fetched_at: string | null;
+  /** Whether anything was actually collected. Live reading only; the snapshot's
+   *  point-in-time overlay omits it. */
+  observed?: boolean;
   observed_in_snapshot?: boolean;
 }
 

frontend/src/pages/RegimePage.tsx

@@ -125,15 +125,30 @@ const CAPEX_TONE: Record<CapexState, string> = {
   unknown: 'text-gray-500',
 };
 
+const OVERLAY_TITLE = 'Fundamental overlay · context, not scored';
+
 function FundamentalOverlayCard({ overlay }: { overlay: RegimeFundamentalOverlay }) {
   const capex = overlay.capex ?? {};
   const reaction = overlay.good_news_stock_down;
+
+  // Nothing collected: the stored default is "unknown" for every hyperscaler
+  // and "mixed" for the reaction, which are placeholders, not a reading.
+  if (overlay.observed === false) {
+    return (
+      <div className="glass border border-white/[0.06] p-5">
+        <div className="text-[11px] uppercase tracking-wider text-gray-500">{OVERLAY_TITLE}</div>
+        <p className="mt-3 text-xs text-gray-500">
+          No observation collected yet. An admin can collect one under Admin · Monitor settings. It is
+          context only — it never enters State or Warning.
+        </p>
+      </div>
+    );
+  }
+
   return (
     <div className="glass border border-white/[0.06] p-5">
       <div className="flex flex-wrap items-baseline justify-between gap-2">
-        <div className="text-[11px] uppercase tracking-wider text-gray-500">
-          Fundamental overlay · context, not scored
-        </div>
+        <div className="text-[11px] uppercase tracking-wider text-gray-500">{OVERLAY_TITLE}</div>
         <div className="flex flex-wrap items-center gap-2 text-[11px] text-gray-500">
           {overlay.source && <span>{overlay.source}</span>}
           {/* When pending, the line below is the single carrier of this date. */}

tests/unit/test_regime_monitor.py

@@ -279,6 +279,37 @@ def test_live_observation_is_visible_before_its_effective_date():
     assert current_observation(overrides, config, date(2026, 8, 22))["available"] is False
 
 
+def test_an_uncollected_observation_is_not_reported_as_collected():
+    """The default override is placeholders, not a reading.
+
+    ``capex`` defaults to "unknown" for every hyperscaler and the reaction to
+    "mixed".  Surfacing those as an observation made the card claim a read that
+    never happened.
+    """
+    names = DEFAULT_CONFIG["tickers"]["hyperscalers"]
+    nothing_collected = {
+        "f1_score": None,
+        "f3_score": None,
+        "capex": {name: "unknown" for name in names},
+        "good_news_stock_down": "mixed",
+        "reasoning": None,
+        "fetched_at": None,
+        "effective_date": None,
+        "source": "default",
+    }
+
+    blank = current_observation(nothing_collected, DEFAULT_CONFIG, date(2026, 8, 7))
+    assert blank["observed"] is False
+    assert blank["available"] is False
+    assert blank["capex"] is None
+    assert blank["good_news_stock_down"] is None
+    assert blank["reasoning"] is None
+
+    # One real observation flips it, placeholders and all.
+    collected = {**nothing_collected, "fetched_at": "2026-08-07T10:00:00+00:00", "source": "gemini"}
+    assert current_observation(collected, DEFAULT_CONFIG, date(2026, 8, 7))["observed"] is True
+
+
 def test_fundamentals_do_not_move_the_warning_score():
     """The v3 complaint: a maxed-out LLM read must not silently do nothing.
 
@@ -462,11 +493,11 @@ async def test_prior_snapshot_is_immutable_without_explicit_rebuild(db_session):
     changed["state"] = {"score": 90.0, "band": "breaking"}
 
     written, _ = await rms._upsert_snapshot(
-        db_session, first, rewrite_existing_v2=True
+        db_session, first, rewrite_existing=True
     )
     await db_session.flush()
     rewritten, persisted = await rms._upsert_snapshot(
-        db_session, changed, rewrite_existing_v2=False
+        db_session, changed, rewrite_existing=False
     )
     row = (
         await db_session.execute(
@@ -509,10 +540,10 @@ async def test_routine_can_refresh_latest_trading_session_after_civil_day_rolls(
         return {}, {}
 
     async def fake_latest(_db):
-        return object(), {"methodology": "v3"}
+        return object(), {"methodology": "v3", "sensor_revision": rms.SENSOR_REVISION}
 
-    async def fake_upsert(_db, result, *, rewrite_existing_v2):
-        rewrites.append(rewrite_existing_v2)
+    async def fake_upsert(_db, result, *, rewrite_existing):
+        rewrites.append(rewrite_existing)
         return True, result
 
     class FakeDB:
@@ -533,6 +564,91 @@ async def test_routine_can_refresh_latest_trading_session_after_civil_day_rolls(
     assert rewrites == [True]
 
 
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("stored", "expect_reseed"),
+    [
+        ({"methodology": "v3"}, True),  # written before the marker existed
+        ({"methodology": "v3", "sensor_revision": 1}, True),
+        ({"methodology": "v3", "sensor_revision": rms.SENSOR_REVISION}, False),
+    ],
+)
+async def test_a_stale_sensor_revision_reseeds_stored_history(
+    monkeypatch, stored, expect_reseed
+):
+    """Widening the OAS window has to reach rows that are already stored.
+
+    Routine runs recompute only the latest date, so without this trigger every
+    older row would keep the credit gap the wider window exists to close.
+    """
+    sessions = [date.today() - timedelta(days=offset) for offset in reversed(range(10))]
+    prices = {symbol: [(day, 100.0) for day in sessions] for symbol in ("SMH", "QQQ", "SPY")}
+    written: list[date] = []
+    revisions: list[int] = []
+
+    async def fake_config(_db):
+        return copy.deepcopy(DEFAULT_CONFIG)
+
+    async def fake_overrides(_db):
+        return {"locked": True, "fetched_at": None, "effective_date": None}
+
+    async def fake_prices(_config, _start, _end):
+        return prices
+
+    async def fake_fred(_series_id, _start, _end):
+        return None
+
+    async def fake_breadth(_db, _symbols, window, min_tickers):
+        return {}, {}
+
+    async def fake_latest(_db):
+        return object(), stored
+
+    async def fake_upsert(_db, result, *, rewrite_existing):
+        written.append(date.fromisoformat(result["date"]))
+        revisions.append(result["sensor_revision"])
+        # Every replayed row must be rewritable, or a reseed writes one row.
+        assert rewrite_existing is True
+        return True, result
+
+    class FakeDB:
+        async def commit(self):
+            return None
+
+    for name, value in (
+        ("get_regime_config", fake_config),
+        ("get_fundamental_overrides", fake_overrides),
+        ("_fetch_prices", fake_prices),
+        ("_fetch_fred_series", fake_fred),
+        ("_latest_snapshot_row", fake_latest),
+        ("_upsert_snapshot", fake_upsert),
+    ):
+        monkeypatch.setattr(rms, name, value)
+    monkeypatch.setattr(rms.breadth_service, "compute_breadth_details", fake_breadth)
+
+    await rms.update_regime_monitor(FakeDB())
+
+    if expect_reseed:
+        assert written == sessions, "a reseed must replay the whole stored span"
+    else:
+        assert written == [sessions[-1]], "a current revision must not reseed"
+    assert set(revisions) == {rms.SENSOR_REVISION}
+
+
+def test_the_rebuild_span_stays_inside_the_oas_window():
+    """The reseed must not replay rows it cannot compute credit for.
+
+    Each replayed row needs W3's lookback inside the fetched OAS window; if the
+    replay reached further back than the fetch, the reseed would recreate the
+    very gap it exists to close.
+    """
+    replay_calendar_days = rms.REBUILD_LOOKBACK_DAYS
+    w3_lookback_calendar = rms.W3_OAS_LOOKBACK * 7 / 5  # business days -> calendar
+    assert replay_calendar_days + w3_lookback_calendar <= rms.HY_OAS_WINDOW_DAYS
+    # ...and still covers the 400-session series the v3 cutover wrote.
+    assert replay_calendar_days >= 400 * 365 / 252
+
+
 @pytest.mark.asyncio
 async def test_manual_llm_refresh_recomputes_latest_regime_snapshot(monkeypatch):
     calls: list[str] = []