dennisthiessenandClaude Opus 5
83c0555e52
The v3 cutover run scored 2/4 corrections warned against v2's 3/4, which reads like a regression and is not one. Only 4 of the 11 detected corrections fall in the holdout, so recall is one event from a different headline -- and the event that flips is decided by threshold placement, not by what the score saw. "v3 without the credit sensor" catches 2025-02-21 at a *higher* threshold (35.5) than shipped v3 misses it at (32.3), because the alarm rule needs a rising edge and a lower threshold can fire outside the horizon then never reset below. Two caveats are now computed and surfaced rather than left for the reader to infer: - Holdout event count against MIN_EVENTS_FOR_CONFIDENCE. The summary sentence states how many of the detected corrections actually fall in the test period. - Warning-sensor coverage across the split. The score renormalises over what is available, so a training window predating a sensor's history freezes the threshold on a different construct than the holdout is measured against. At the cutover that is 39% of training sessions with all three sensors versus 100% of the test period, credit history beginning 2023-07-25. Restricting the threshold to sensor-matched training sessions was tested and rejected: those sessions are a calm recent stretch, so the threshold falls from 32.3 to 22.5 and false alarms rise from 3.3 to 8.6/yr. It swaps a coverage bias for a regime-selection bias. The report states its limits instead. _warning_series now returns per-session sensor counts alongside the scores. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
203 lines
11 KiB
Markdown
203 lines
11 KiB
Markdown
# Regime Monitor v3 methodology
|
||
|
||
The Regime Monitor is an observational AI/Tech risk thermometer. It does not
|
||
gate entries, exits, position size, ranking, or alerts about individual setups.
|
||
|
||
v3 supersedes v2. Every parameter below was calibrated against the 408 v2
|
||
sessions ending 2026-07-24, reproduced offline from the same Alpaca and FRED
|
||
inputs the live job uses; the reproduction matched the stored prod distribution
|
||
exactly (State avg 22.6/22.7, p80 35.1, max 91.2, P3 pegged 39, W1 live 108).
|
||
|
||
## What changed and why
|
||
|
||
**Fundamentals left the score.** F1 (capex) and F3 (good-news-stock-down)
|
||
carried 12 + 8 of 100 Warning points. Pegged at maximum stress they produced a
|
||
Warning of exactly 20.0 — below the event study's 25.3 alarm threshold, and
|
||
still inside the "stable" band. The sourced observation could not change any
|
||
published conclusion, so refreshing it looked like it did nothing. They are now
|
||
a qualitative overlay reported beside the scores. Capex also stopped scoring
|
||
`raising` and `holding` identically at 0: `holding` is the deceleration case and
|
||
now scores 50, so a boom no longer reads the same as a stall.
|
||
|
||
**The drawdown sensor stopped saturating.** v2 used `dd_pct * 5`, reaching 100 at
|
||
a 20% drawdown — the 90th percentile of the observed distribution. 39 of 408
|
||
sessions sat at exactly 100 with no resolution left, and the price pillar showed
|
||
the top band on 13.5% of sessions. v3 uses named anchors with headroom past the
|
||
observed 36% maximum, and blends leader/confirm 2:1 as P1 and P2 already did
|
||
instead of taking `max()`. P3's realized share of State falls from 65% to 40%,
|
||
matching its nominal weight.
|
||
|
||
**Warning gained a sensor with range.** The HY OAS *level* is pinned at zero
|
||
below the 3.5 mild anchor (2.77 at the cutover), so credit contributed nothing
|
||
in a calm tape. Its 20-session rate of change still does, and spread widening is
|
||
a classic lead.
|
||
|
||
**The credit percentile leg was removed.** Its reference window silently shrank
|
||
from 10 years to 3 when ICE restricted the upstream series in April 2026, after
|
||
which it scored 20 points of stress at a spread the same sensor's anchors call
|
||
"mild". See Calibration below.
|
||
|
||
**Breadth loss counts during declines.** v2's divergence gate was
|
||
`price_ret >= 0`, so the sensor zeroed during every selloff. On 2026-07-24 the
|
||
basket shed 10 points of participation in 20 sessions while SMH fell 11.9% and
|
||
Warning printed exactly 0. v3 tapers to a floor instead: deterioration counts
|
||
fully when price masks it (true divergence, the dangerous pre-top case) and at
|
||
35% when price confirms it. Breadth *level* lives in State, but breadth
|
||
*velocity* appears nowhere else, so this is not double counting.
|
||
|
||
**Bands are per axis.** v2 Warning never exceeded 64.9 in 408 sessions while
|
||
State reached 91.2, yet both used 30/60/80 with quadrant dividers at 60. The
|
||
upper half of the Warning axis was unreachable.
|
||
|
||
## Outputs
|
||
|
||
**State** — current structural stress:
|
||
|
||
- Price structure, 40%: `max(P1, P2, P3)`, one capped vote for correlated reads.
|
||
- Fixed-basket breadth level, 25%.
|
||
- HY option-adjusted credit spread level, 20%.
|
||
- VIX level, 15%.
|
||
|
||
**Warning** — deterioration and divergence:
|
||
|
||
- Fixed-basket breadth divergence, 45%.
|
||
- 60-session SMH/SPY relative-strength deterioration, 30%.
|
||
- HY OAS 20-session widening, 25%.
|
||
|
||
Combined, RSP/SPY (former F4), and the NVDA canary (former P6) do not enter v3.
|
||
|
||
## Calibration
|
||
|
||
P3 drawdown anchors, as (drawdown %, score): 0→0, 4→10, 8→25, 16→50, 28→78,
|
||
40→100, flat outside. Credit impulse is relative (+35% over 20 sessions = 100)
|
||
rather than absolute, because +0.5pp means something very different at an OAS of
|
||
2.7 than at 8.0.
|
||
|
||
Bands are round, meaning-anchored numbers, not percentile fits — percentile
|
||
thresholds would drift on every rebuild and silently rewrite what past snapshots
|
||
meant. Realized shares over the calibration window:
|
||
|
||
| Axis | stable | watch | elevated | breaking | thresholds |
|
||
|------|--------|-------|----------|----------|------------|
|
||
| State | 73.3% | 15.0% | 8.3% | 3.4% | 20 / 50 / 80 |
|
||
| Warning | 69.4% | 19.6% | 7.6% | 3.4% | 20 / 40 / 60 |
|
||
|
||
Quadrant dividers sit at each axis's watch/elevated boundary: State 50,
|
||
Warning 40.
|
||
|
||
Scores renormalize over available fixed weights, but a band is published only at
|
||
75% or greater coverage. Trend deltas are suppressed when the participating
|
||
pillar set changes. Zero means ordinary/healthy; only stress contributes.
|
||
|
||
Credit level is the named HY OAS anchors alone: 3.5 mild, 5.0 elevated, 7.0
|
||
stressed, linear between, and nothing else. v2 blended those anchors at 70% with
|
||
a 30% upper-tail percentile over a nominally 10-year window.
|
||
|
||
That leg was removed rather than repaired. ICE restricted FRED to a rolling
|
||
3-year window for `BAMLH0A0HYM2` in April 2026 — the series metadata states it
|
||
outright ("Starting in April 2026, this series will only include 3 years of
|
||
observations"), and an unbounded request returns the same 795 observations as a
|
||
30-year one. The v2 percentile therefore ranked the current spread against three
|
||
uniformly tight years (range 2.59–4.61 over the calibration window), which made
|
||
it fire early and saturate absurdly: at an OAS of 3.50 — the level the anchors
|
||
call *mild*, scoring zero stress — the blended sensor read 20.1, and the
|
||
percentile leg pegged at 100 by an OAS of 4.5. Across the 408 sessions it
|
||
roughly tripled the credit sensor's average (2.70 vs 1.00) and more than doubled
|
||
its nonzero days (60 vs 27).
|
||
|
||
The anchors already encode the long-run distribution as constants, so the
|
||
percentile was a second, noisier estimate of the same thing. What it was
|
||
genuinely reaching for — "unusual versus recent history" — is now W3 on the
|
||
Warning axis, computed as a rate of change, which is where deterioration
|
||
belongs. Removing it moved State's average by −0.4 and its maximum by −3.8, left
|
||
Warning bit-identical, and did not shift any band threshold.
|
||
|
||
A long-history alternative (`BAA10Y`, Fed-published, 7,712 observations back to
|
||
1997) was considered and rejected: ranking an HY spread against investment-grade
|
||
history is not a coherent statistic, and it would rescue a leg that is redundant
|
||
anyway.
|
||
|
||
Every snapshot now records `data_quality.credit_history_days` and
|
||
`vix_history_days`. This defect was invisible for roughly three months because
|
||
nothing asserted the window the code claimed; the spans make a future upstream
|
||
truncation show up in the record instead of quietly reshaping a sensor.
|
||
|
||
**Survivorship caveat.** The basket was frozen 2026-07-15 but the calibration
|
||
window reaches back to 2024, so names were partly selected for having done well.
|
||
Every distribution above inherits that bias. It is the same bias v2 carried, so
|
||
the v2/v3 comparison is like-for-like, but the absolute band shares are
|
||
optimistic.
|
||
|
||
## Point-in-time record
|
||
|
||
The first run under a new `METHODOLOGY` rebuilds the latest 400 trading sessions
|
||
with sufficient sensor warm-up; routine runs thereafter insert/update only the
|
||
latest trading date. The history API and main chart show only snapshots matching
|
||
the current methodology, so a bump reseeds the series rather than splicing two
|
||
formulas into one line.
|
||
|
||
The fundamental overlay keeps its effective date (normally the next session after
|
||
collection) and is never replayed backward, so a rebuild cannot stamp today's
|
||
observation onto historical snapshots. Because the observation is stored in a
|
||
single slot, a refresh replaces the previously effective record: the snapshot
|
||
therefore reports the overlay as `pending` until the new effective date, and the
|
||
live reading additionally carries `fundamental_context` so a just-collected
|
||
observation is visible immediately rather than appearing to have done nothing.
|
||
|
||
Each snapshot stores the fixed basket symbols, hash, and freeze date.
|
||
Reconstructed history before that freeze date is retrospective/exploratory.
|
||
|
||
## Warning study
|
||
|
||
The study calls the outcome a **10% correction**, not a regime break. The first
|
||
70% of sessions freezes the 80th-percentile warning threshold; alarm episodes are
|
||
measured on the final 30%. Because v3 dropped fundamentals from the score, the
|
||
study now measures exactly the live Warning score rather than a technical-only
|
||
approximation of it, and both are computed from one shared sensor definition
|
||
(`warning_sensor_scores`) so they cannot drift apart.
|
||
|
||
A cached report is discarded when its methodology no longer matches, so the panel
|
||
reverts to "not run yet" after a bump rather than showing stale numbers. **Re-run
|
||
the Event Study job after cutting over to v3.**
|
||
|
||
### Reading the result
|
||
|
||
The report carries a `reliability` block and the UI renders its warnings, because
|
||
the headline numbers invite over-reading in two specific ways.
|
||
|
||
**The holdout is thin.** The study detects 11 corrections across 5 years but the
|
||
70/30 split leaves only 4 in the test period. Recall is therefore one event away
|
||
from a materially different headline, and in practice the event that flips is
|
||
decided by where the frozen threshold happens to land rather than by whether the
|
||
score saw anything. The v3 cutover run illustrates it: v3 scored 2/4 against v2's
|
||
3/4, but "v3 without the credit sensor" scores 3/4 at a *higher* threshold
|
||
(35.5) than shipped v3 misses it at (32.3) — because the alarm rule needs a
|
||
rising edge, and a lower threshold can mean the alarm already fired outside the
|
||
20-session horizon and never reset below. Below `MIN_EVENTS_FOR_CONFIDENCE`
|
||
holdout events the report says so explicitly.
|
||
|
||
Some events carry no information at all for comparison: in that run every
|
||
variant caught 2026-03-06, every variant missed 2026-06-05, and every variant
|
||
"caught" 2025-11-20 with a 1-session lead, which is coincident rather than a
|
||
warning.
|
||
|
||
**Sensor coverage can straddle the split.** The score renormalises over available
|
||
sensors, so a training window predating a sensor's history freezes the threshold
|
||
on a different construct than the holdout is measured against. At the v3 cutover
|
||
only 39% of training sessions had all three Warning sensors versus 100% of the
|
||
test period, because credit history begins 2023-07-25.
|
||
|
||
Restricting the threshold to sensor-matched training sessions was tried and is
|
||
*not* the fix: those sessions are a calm recent stretch, so the threshold drops
|
||
from 32.3 to 22.5 and false alarms rise from 3.3 to 8.6 per year. It trades a
|
||
coverage bias for a regime-selection bias. The honest position is that the
|
||
threshold is hypersensitive to window choice at this sample size; the report
|
||
states its limits rather than pretending to a precision it does not have.
|
||
|
||
## Operator rule
|
||
|
||
Quadrant alerts default off for new/reset configurations. When enabled they
|
||
require fresh inputs, at least 75% coverage on both axes, two consecutive daily
|
||
confirmations, hysteresis, and cooldown. Every alert states: **Risk thermometer —
|
||
not a trade signal.**
|