feat(risk-monitor): measure the rule that fires, and give fundamentals their own channel
Deploy / lint (push) Failing after 11s
Deploy / test (push) Skipped
Deploy / deploy (push) Skipped

The Warning study measured a fitted percentile crossing that nothing consumes.
What reaches Telegram is a quadrant change: fixed 50/40 dividers, hysteresis,
two-session confirmation, 3-day cooldown. Those thresholds are constants, not
fits, so there is no training set to protect and all 11 detected corrections are
evaluable instead of the 4 that fell in a holdout.

Replaying it: 1/10 corrections, 0.9 false alarms/year. Random alarms at the same
firing rate match or beat that in 65% of draws. The panel now carries ablations
(does the quadrant machinery earn its place?), external baselines (does the score
earn its complexity?), and that null, because a bare "2 of 4" was unreadable in
either direction. Nothing in the alert path was retuned on the strength of it.

Fundamentals become a third channel rather than a term in either score. v3 cut
them arguing 12+8 of 100 points "could not change any published conclusion" --
true only when every technical sensor reads zero; weighted they moved the bar for
the 40 divider from 40 to 25. But no fusion weight is measurable either: with ~10
events and no fundamental history, any weight is a policy preference presented as
a measurement. So the read is a categorical state (supportive/neutral/adverse/
unknown) with an evidence grade, derived by fixed rules from stored facts, read
by confluence. The LLM extracts and explains; it does not score.

Absence stays absence throughout. `unknown` is unreachable by averaging, a stale
or empty observation may display but never confirm, extraction failures map to
`unknown` rather than `mixed`, and the study rows are coverage-matched and marked
not-measurable until enough corrections are covered -- otherwise a fortnight of
observations renders as 0/10 and reads as a failed test.

Observations become a real time series (migration 033); they lived in a single
overwritten settings slot, so no history existed to replay. Pre-rename snapshots
are adapted rather than discarded. METHODOLOGY stays v4 -- no score changed --
so no reseed; STUDY_SCHEMA moves to 3 and discards the cached report.

Post-deploy: re-run Event Study from Admin -> Jobs. The panel reads "not run yet"
until then.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-13 11:15:09 +02:00
co-authored by Claude Opus 5
parent 3033ad83fd
commit 333989eeab
18 changed files with 3494 additions and 403 deletions
+410 -39
View File
@@ -39,6 +39,162 @@ session, the calendar anchors, 100% coverage on every row, and a row-wise
`state_v4 <= state_v3` invariant. Reading a calibration result out of a run whose
pipeline did not validate is meant to be structurally impossible.
## The fundamental channel (2026-08-12)
The monitor has **three channels**, not two scores with a decoration:
- **State** — current observable technical stress (price, breadth, credit, volatility).
- **Warning** — observable deterioration that may precede stress (breadth
divergence, relative strength, credit impulse).
- **Fundamental context** — a categorical state (`supportive` / `neutral` /
`adverse` / `unknown`) with an `evidence_quality` grade.
The third is **never a term in the other two**. They are read together by
confluence:
| Warning | Fundamentals | Reading |
|---|---|---|
| Calm | Supportive/neutral | Normal |
| Elevated | Supportive/neutral | Technical warning, not fundamentally confirmed |
| Calm | Adverse | Fundamental concern; tape has not confirmed |
| Elevated | Adverse | Confluence — highest attention |
`METHODOLOGY` stays **v4**: no score changed, so partitioning the history API and
discarding the event study cache would be churn. `STUDY_SCHEMA` moved to 3
instead, and is now the only thing that discards a stale report.
### Why the read is a channel and not a weight
Two things are true at once, and only this shape honours both.
**v3's reason for removing fundamentals from the score was wrong.** Not stale —
wrong. v3 argued that F1 (capex) and F3 (good-news-stock-down), carrying 12 + 8
of 100 Warning points, "could not change any published conclusion" because pegged
they produced a Warning of exactly 20.0, below the alarm threshold. That
arithmetic holds only when *every* technical sensor reads exactly zero, which is
the one case that never matters. Warning is a weighted average, so the sensors
add:
| technical Warning | without fundamentals | with them pegged | delta |
|---|---|---|---|
| 0 | 0.0 | 20.0 | +20.0 |
| 20 | 20.0 | 36.0 | +16.0 |
| 25 | 25.0 | **40.0** | +15.0 |
| 35 | 35.0 | **48.0** | +13.0 |
| 50 | 50.0 | 60.0 | +10.0 |
| 80 | 80.0 | 84.0 | +4.0 |
Pegged fundamentals lowered the technical Warning needed to reach the 40 quadrant
divider from 40 to 25. That is a 15-point shift in where the alert fires, which
is emphatically a changed conclusion. The v3 section below is kept as written,
with this correction attached, because its reasoning is cited elsewhere in this
file and a silent overwrite would hide that the error was ever made.
**But no weight is measurable either.** A weighted modifier was built and
reverted: 025 points added onto the technical Warning, sized so a maxed-out read
carried a calm tape over the 40 divider on its own. Nothing could justify the 25.
With ~10 correction events and essentially no fundamental history, any fusion
weight is a policy preference presented as a measurement — and the debate it
invites ("does the read deserve 10%, 20%, 30%?") has no evidence that can settle
it. Adding a slow categorical judgement to a fast continuous score also
manufactures precision by summing unlike things, and it forces a missing
observation to silently redistribute its weight onto the technical sensors, which
is the opposite of leaving it unknown.
So: the read gets a channel, not a coefficient. Both facts survive — the v3
removal was badly argued *and* no weight is defensible — because "report it
separately" is the only design that neither buries the observation nor invents a
number for it.
### Derivation
Deterministic, from the stored categorical facts. The LLM is an **extraction and
explanation layer**: it finds the capex guidance, classifies it, and cites it.
Fixed rules turn those facts into a state, so the same observation always yields
the same category.
`capex_signal`: any `cutting` → adverse; else any `holding` → neutral; else all
known `raising` → supportive; nothing known → unknown.
`reaction_signal`: `yes` → adverse, `mixed` → neutral, `no` → supportive,
`unknown` → unknown.
`mixed` and `unknown` are different reaction states and were merged until
2026-08-13. A failed LLM parse fell back to `mixed`, so an extraction error
became *neutral evidence* — an observation of normality manufactured out of a
bug. `mixed` now means an observed mixed reaction; anything unreadable, missing
or unattempted is `unknown` and contributes nothing.
Combined by precedence, never by averaging: **any adverse read carries**; both
unknown → unknown; every observed signal supportive → supportive; otherwise
neutral.
`unknown` is deliberately unreachable by combination. Averaging would let two
`cutting` reads and two `unknown` ones land on "neutral", presenting missing
evidence as evidence of normality — the same conflation `current_observation`
already refuses between "no observation" and "an observation of zero". Two cuts
and two unknowns read **adverse with `evidence_quality: partial`**.
`evidence_quality` is ordered by what an operator needs first: `unavailable`
(nothing collected) → `stale` (past `fundamental_staleness_days`) → `manual`
(hand override) → `complete` / `partial`.
### Presentation and alerts
The Path view colours each dot by the fundamental state recorded that day; the
axes are untouched, because context is confluence information rather than a
position on either axis. The card leads with the state and evidence grade.
Alerts stay **separate**, off one toggle:
- quadrant change — the market axes moved (existing);
- `regime_fundamental` — the context changed, e.g. neutral → adverse;
- `regime_confluence` — Warning elevated *and* fundamentals adverse.
`unknown` never alerts: an absence of evidence is not a change in the evidence,
and alerting on it would train the reader to ignore the channel. Both new
triggers seed silently on first run, as the quadrant alert does.
### The observation is now a real time series
`regime_fundamental_observations` (migration 033), one row per `effective_date`,
upserted. Before this it lived in a single `SystemSetting` slot that every
refresh overwrote, so no history existed at all — which made the read impossible
to replay, impossible to backtest, and meant a rebuild recorded every historical
session as if nothing had been observed. `update_regime_monitor` carries the
pre-existing single-slot observation into the series on its next run.
### What this does not establish
The table starts empty and fills one observation at a time, so the fundamental
rows are **untested, not failed**. Two things enforce that rather than one:
- they are **coverage-matched** — scored only on sessions where the channel had
usable context and on corrections whose warning horizon fell inside it, with a
market-only comparator over the identical window so any difference between them
is the channel and not the window;
- `measurable` stays false until `MIN_EVENTS_FOR_CONFIDENCE` corrections are
covered, and the panel prints "insufficient exposure" rather than a ratio.
Without the first, one day of coverage would render as 0/10 — recreating, one
observation later, exactly the tested-versus-unavailable confusion the flag was
added to prevent. The market rows are unchanged, and the 1/10 shipped-rule figure
remains a verdict on the technical sensors and the alert machinery alone.
The rationale for expecting the read to matter is the operator's: hyperscaler
capex is the demand side of the entire AI trade, and good earnings being sold is
a classic late-cycle tell. Both are plausible. Neither is measured here, and this
file's convention is that published numbers are reproducible.
**The path forward is accumulation, then a test — in that order.** Once enough
point-in-time observations exist, test whether the state improves prediction
*conditional on* Warning. If it does, a fitted and calibrated model has something
to fit; until then there is nothing to calibrate against. Backfilling would get
there faster: capex direction is derivable from the 10-Q/10-K capex line, which
the SEC fundamentals import already carries, and "good news, stock down" from
earnings dates plus next-day returns, which the Dolt earnings import already
carries. That last one is worth computing deterministically rather than asking
the LLM to judge, for the same reason the state derivation is rule-based.
## What changed in v4
**V1 stopped saturating at VIX 30.** `(vix - 15) / 15` reached 100 at VIX 30 —
@@ -81,6 +237,16 @@ a qualitative overlay reported beside the scores. Capex also stopped scoring
`raising` and `holding` identically at 0: `holding` is the deceleration case and
now scores 50, so a boom no longer reads the same as a stall.
> **Corrected 2026-08-12.** The claim in this paragraph is false. "Pegged
> they produced a Warning of exactly 20.0" describes only the case where every
> technical sensor reads zero; Warning is a weighted average, so in the general
> case those 20 points added +10 to +20 and moved the technical score needed to
> reach the 40 quadrant divider from 40 to 25. The observation was removed for
> being *underweighted*, on reasoning that mistook a corner case for the whole
> range. See "The fundamental channel" above for what replaced it — a separate
> categorical channel, not a restored weight. The capex `holding` rescale in the second half
> of this paragraph stands and is still live.
**The drawdown sensor stopped saturating.** v2 used `dd_pct * 5`, reaching 100 at
a 20% drawdown — the 90th percentile of the observed distribution. 39 of 408
sessions sat at exactly 100 with no resolution left, and the price pillar showed
@@ -126,7 +292,11 @@ upper half of the Warning axis was unreachable.
- 60-session SMH/SPY relative-strength deterioration, 30%.
- HY OAS 20-session widening, 25%.
Combined, RSP/SPY (former F4), and the NVDA canary (former P6) do not enter v3 or v4.
**Fundamental context** — a categorical third channel, not a term in either
score. See "The fundamental channel" above.
Combined, RSP/SPY (former F4), and the NVDA canary (former P6) do not enter v3
or v4.
## Calibration
@@ -279,31 +449,73 @@ reseed exists to close. The history API and main chart show only snapshots match
the current methodology, so a bump reseeds the series rather than splicing two
formulas into one line.
The fundamental overlay keeps its effective date (normally the next session after
The fundamental channel keeps its effective date (normally the next session after
collection) and is never replayed backward, so a rebuild cannot stamp today's
observation onto historical snapshots. Because the observation is stored in a
single slot, a refresh replaces the previously effective record: the snapshot
therefore reports the overlay as `pending` until the new effective date.
observation onto historical snapshots. Since the observations became a real
series (`regime_fundamental_observations`, migration 033), the effective-date
lookup *is* the gate: a replayed session gets whichever observation was live on
it, and sessions before the first one read `unknown`.
Two functions, deliberately: `fundamental_overlay` is the **record** and keeps
Two functions, deliberately: `fundamental_context` is the **record** and keeps
the gate — it runs for every replayed date during a rebuild, so it must never
grow a bypass flag. `current_observation` is the **live reading** behind
`fundamental_context`, and *reports* the effective date instead of blanking the
`fundamental_live`, and *reports* the effective date instead of blanking the
content.
Until 2026-08-07 the live reading called the gated function, so a just-collected
observation stayed hidden until the next weekday — three days over a weekend —
and refreshing appeared to do nothing. That was the opposite of what this section
already claimed. Showing it early cannot leak into a published number, because
nothing in the overlay is scored (see "Fundamentals left the score").
already claimed. Showing it early cannot leak into a published score, because
nothing in the channel is scored.
`current_observation` gates on `observed` (a non-null `fetched_at`, the one field
every path writing real content stamps). Without it, the default override —
`unknown` for every hyperscaler and `mixed` for the reaction — was reported as a
live observation with `available: true`, so the card presented placeholders as a
collected reading. Those are the absence of an observation, not an observation of
absence. `fundamental_overlay` never had this problem: no observation means no
effective date, which means `pending`, which already blanks the content.
`unknown` for every hyperscaler and, since 2026-08-13, `unknown` for the reaction
— was reported as a live observation with `available: true`, so the card
presented placeholders as a collected reading. Those are the absence of an
observation, not an observation of absence. `fundamental_context` never had this
problem: no observation means no effective date, which means `pending`, which
already blanks the content.
**`usable` is what may confirm; `available` is only what to display.** Three
distinct things, and collapsing any two of them is a bug:
- `state` — the last thing observed. Survives going stale, so the card can show it.
- `available`*timing*: there is an effective, non-stale record to display.
- `usable`*content*: available **and** the observation actually determined
something (`state != "unknown"`).
The confluence alert and all three coverage-matched study rules gate on `usable`.
Gating on `available` instead has two failure modes, and both were live at some
point in this design:
1. a reading past `fundamental_staleness_days` would corroborate every Warning
crossing indefinitely — the strongest claim this channel makes, from the data
with the least right to make it;
2. an LLM run that failed to extract anything produces a perfectly fresh
observation that knows nothing. Counting it as exposure means repeated
extraction failures slowly accumulate coverage until the fundamental rows flip
to a *measurable* 0/8 — a failed result published for a channel that never saw
a thing, which is precisely what coverage-matching exists to prevent.
**Pre-rename snapshots are adapted, not discarded.** The channel was stored as
`fundamental_overlay` until 2026-08-12. The rename shipped without a methodology
bump — no score changed — so those rows are still served and were never reseeded.
Reading only the new key would have turned every one of them into `unknown`,
silently dropping real recorded evidence: historical Path colours, and exposure
the event study can legitimately count. `_parse_snapshot` derives the channel
from a legacy overlay's own stored facts (its capex map supplies the basket, so
the derivation uses the names observed at the time rather than today's config).
Normalising there rather than at each call site means no reader can receive an
un-adapted row. Delete only after a reseed has rewritten the whole window.
**The blob and the series row are one transaction.** They are the same
observation seen by the live card and by the point-in-time replay; committing
them separately leaves a window where a failure publishes one and not the other,
and the two then disagree permanently with nothing to detect it. Both writers use
`settings_store.upsert_setting` (which does not commit) plus a single commit;
`record_fundamental_observation` deliberately takes no commit of its own so
`update_regime_monitor` keeps its own transaction boundary.
Each snapshot stores the fixed basket symbols, hash, and freeze date.
Reconstructed history before that freeze date is retrospective/exploratory.
@@ -321,39 +533,185 @@ today's number. The quadrant dividers rendered in Path view come from
## Warning study
The study calls the outcome a **10% correction**, not a regime break. The first
70% of sessions freezes the 80th-percentile warning threshold; alarm episodes are
measured on the final 30%. Because v3 dropped fundamentals from the score, the
study now measures exactly the live Warning score rather than a technical-only
approximation of it, and both are computed from one shared sensor definition
(`warning_sensor_scores`) so they cannot drift apart.
The study calls the outcome a **10% correction**, not a regime break. It measures
two rules against that outcome, plus enough context to tell whether either number
is any good.
A cached report is discarded when its methodology no longer matches, so the panel
reverts to "not run yet" after a bump rather than showing stale numbers. **Re-run
the Event Study job after cutting over to v4.**
A cached report is discarded when its methodology no longer matches *or* when
`STUDY_SCHEMA` moves, so the panel reverts to "not run yet" rather than showing
stale numbers or a report missing half its blocks. **Re-run the Event Study job
after a methodology cutover or a schema bump.**
### The headline is the rule that actually fires
Until 2026-08-12 the study measured a bare rising-edge crossing of an
80th-percentile threshold fitted on the first 70% of sessions. **Nothing consumes
that rule.** What reaches Telegram is `_collect_regime_quadrant`: a quadrant
change with State ≥ 50 and Warning ≥ 40 as fixed dividers, a ±5 hysteresis
deadband, a two-session confirmation, a 3-day cooldown, and a 75% coverage gate
on both axes. The two differ on every one of those axes, including the threshold
itself (a fitted ~32 against a shipped 40).
`replay_quadrant_changes` replays the shipped state machine over the whole
sample. Three details are reproduced rather than cleaned up, because a state
machine written from first principles gets each of them wrong:
- the prior session is classified against the **current baseline**, not against
its own predecessor, so confirmation asks "did yesterday already look like this
change" rather than "did yesterday change too";
- the baseline advances only when an alert actually fires, so a change blocked by
confirmation or cooldown is re-evaluated against the old quadrant next session;
- one cooldown is shared by every quadrant change, so a 3→4 alert can swallow a
4→2 alert three days later.
Two consequences worth stating. The alarm is dated at the **confirmation**, not
at the first crossing, which costs one session of lead by construction. And the
rule alerts on changes in both directions, so the replay's exits are recorded but
filtered out by `entry_alarms` — only entering a Warning-high quadrant is a
warning about anything.
The replay reuses `_compute_index` rather than re-deriving the axes. That is the
same anti-drift argument that produced `warning_sensor_scores`: the v2 study
re-derived Warning by hand and would have kept measuring the old construct
through a scoring change. State has no equivalent shared helper, so the snapshot
builder itself is the shared definition.
**Nothing is fitted, so nothing needs protecting from a training set.** There is
no split, and every detected correction is evaluable instead of the four that
happen to land in the last 30%. The `underpowered` and "threshold frozen on a
different construct" caveats do not apply to this variant.
### Reading the result
The report carries a `reliability` block and the UI renders its warnings, because
the headline numbers invite over-reading in two specific ways.
A bare "2 of 4" is unreadable in either direction, so the report scores four more
rules through the same `evaluate_alarms` harness over the same events and
sessions, and adds a null. All use fixed thresholds — a threshold fitted on the
full sample would have lookahead the shipped rule does not, and one fitted on a
split could only be scored on the holdout events.
| kind | rules | the question |
|---|---|---|
| ablation | Warning ≥ 40 bare, State ≥ 50 bare | does the quadrant machinery earn its place? |
| baseline | leader below its 50-DMA, VIX ≥ 20 | does the score earn its complexity? |
| null | K random alarms at the observed firing rate | is any of this better than chance? |
The two kinds must not be read as one list. If a baseline matches the score, the
composite is not earning its complexity and that is the finding — it does not
mean the monitor is worthless, since State and Warning exist to be *read*, but it
caps how much further calibration is justified. If the bare Warning crossing
beats the shipped rule, the machinery (not the sensor) is what is costing recall.
The null draws only from sessions a rule could actually have fired on. Over the
whole sample it would be diluted by warm-up sessions and would understate what
chance achieves — which matters, because with ~11 events and a 20-session horizon
roughly a sixth of the sample already sits inside a hit window. It is seeded, so
a re-run cannot move the report. Corrections cluster and uniform placement does
not, so it is the **floor, not the bar**: an alarm process that clustered would
beat it for reasons unrelated to foresight.
### First result (2026-08-12): the shipped rule is not distinguishable from chance
Replayed over 2021-07-14 → 2026-08-12. The 200-DMA warm-up means the baseline
only seeds on 2022-05-26, so 1056 of 1276 sessions are evaluable and 10 of the 11
detected corrections fall inside them.
| rule | kind | warned | FA/yr | median lead |
|---|---|---|---|---|
| **Quadrant alert (shipped)** | | **1/10** | **0.9** | 19d |
| Quadrant alert, both axes high | ablation | 0/10 | 0.9 | — |
| Warning ≥ 40, bare crossing | ablation | 3/10 | 4.8 | 20d |
| State ≥ 50, bare crossing | ablation | 0/10 | 0.7 | — |
| SMH below its 50-DMA | baseline | 7/10 | 6.7 | 8d |
| VIX ≥ 20 | baseline | 4/10 | 7.2 | 9.5d |
| Random alarms, same firing rate | null | 0.9 ± 0.8 | — | — |
**P(chance ≥ 1/10) = 0.65.** Alarms scattered at random over the same sessions at
the rule's own firing rate match or beat it two times in three. Whatever the
score knows, this rule is not transmitting it.
Three readings, in order of how much they should change:
**The machinery costs more than it protects.** The bare Warning crossing catches
3 with a 20-session lead; wrapping it in the quadrant rule drops that to 1. The
State condition is the largest single cost — requiring both axes high catches
nothing at all, which is what a coincident axis gating a leading one predicts.
Hysteresis, the two-session confirmation and the shared cooldown between them
take the rest, and the cooldown is shared across *every* quadrant change, so
exits consume the budget that entries need. Only 5 of the 15 replayed changes are
Warning-high entries.
**The crude baselines beat everything on recall, at a price.** SMH below its
50-DMA catches 7 of 10 — but at 6.7 false alarms a year against the shipped
rule's 0.9. That is a 7× recall improvement for 7× the noise, so it is not a
clean dominance and this table cannot settle it; the missing axis is what a false
alarm actually costs, which nothing here measures. What it does settle is that
the composite is not buying recall the 50-DMA does not already have.
**The 0.9 false alarms/year is not the achievement it looks like.** A rule that
almost never fires has few false alarms by construction. Read the two columns
together or not at all.
Recorded from an offline replay (live Alpaca + FRED, no database, breadth
computed from the same Alpaca closes rather than the stored universe). The job in
Admin → Jobs is the canonical path and reads breadth from the DB, so re-run it to
confirm these figures before treating them as the record.
**This is a verdict on the market channels only.** The fundamental and confluence
rows in the same table are marked `measurable: false` and print "not measurable"
rather than a ratio: with an empty observation series they never fire, and a 0/10
sitting in a comparison column would read as tested-and-failed. `false` here means
the input does not exist yet, not that the rule lost.
(The figures above were also produced under a briefly-built weighted modifier and
came back bit-identical, which is what confirmed the modifier was inert over the
whole window — the numbers depend on the technical sensors alone either way.)
**Not acted on.** Nothing in the alert path was changed on the strength of this.
The obvious candidates — dropping the State condition from the entry test,
separating the entry and exit cooldowns, or lowering the Warning divider — are
threshold changes to a live alerting rule and want their own decision.
### The coverage gap relocates, it does not close
Dropping the fitted threshold makes the whole sample evaluable, but most of the
extra events predate 2023-08. W3 does not exist there, so Warning renormalises to
`(W1×45 + W2×30)/75` and the fixed 40 divider is applied to a different construct
than it was reasoned about. The report therefore splits shipped-rule metrics at
the credit sensor's first session and the panel states both, because replacing
one misleading headline with a differently misleading one would be no gain.
Convenient side effect: the pre-credit era *is* the "Warning without W3"
ablation, measured on real sessions rather than simulated ones, so that ablation
is not run separately.
Alarms and events are assigned to eras by index, so an alarm days before the
boundary matching an event days after it lands in the earlier era. With the eras
years long and the events sparse, that costs nothing.
### The fitted variant, kept for continuity
The 70/30 percentile study is still computed and still reported, collapsed, with
its `reliability` block intact — it is a genuinely different question, and it is
what earlier revisions of this document report. Its caveats stand:
**The holdout is thin.** The study detects 11 corrections across 5 years but the
70/30 split leaves only 4 in the test period. Recall is therefore one event away
from a materially different headline, and in practice the event that flips is
decided by where the frozen threshold happens to land rather than by whether the
score saw anything. The v3 cutover run illustrates it: v3 scored 2/4 against v2's
3/4, but "v3 without the credit sensor" scores 3/4 at a *higher* threshold
(35.5) than shipped v3 misses it at (32.3) — because the alarm rule needs a
rising edge, and a lower threshold can mean the alarm already fired outside the
20-session horizon and never reset below. Below `MIN_EVENTS_FOR_CONFIDENCE`
holdout events the report says so explicitly.
70/30 split leaves only 4 in the test period. Recall is one event away from a
materially different headline, and in practice the event that flips is decided by
where the frozen threshold happens to land rather than by whether the score saw
anything. The v3 cutover run illustrates it: v3 scored 2/4 against v2's 3/4, but
"v3 without the credit sensor" scores 3/4 at a *higher* threshold (35.5) than
shipped v3 misses it at (32.3) — because the alarm rule needs a rising edge, and a
lower threshold can mean the alarm already fired outside the 20-session horizon
and never reset below. Below `MIN_EVENTS_FOR_CONFIDENCE` holdout events the
report says so explicitly.
Some events carry no information at all for comparison: in that run every
variant caught 2026-03-06, every variant missed 2026-06-05, and every variant
"caught" 2025-11-20 with a 1-session lead, which is coincident rather than a
warning.
warning. The headline recall does not currently discount those; a minimum-lead
rule is the obvious next change and has not been made.
**Sensor coverage can straddle the split.** The score renormalises over available
**Sensor coverage straddles the split.** The score renormalises over available
sensors, so a training window predating a sensor's history freezes the threshold
on a different construct than the holdout is measured against. At the v3 cutover
only 39% of training sessions had all three Warning sensors versus 100% of the
@@ -362,9 +720,22 @@ test period, because credit history begins 2023-07-25.
Restricting the threshold to sensor-matched training sessions was tried and is
*not* the fix: those sessions are a calm recent stretch, so the threshold drops
from 32.3 to 22.5 and false alarms rise from 3.3 to 8.6 per year. It trades a
coverage bias for a regime-selection bias. The honest position is that the
threshold is hypersensitive to window choice at this sample size; the report
states its limits rather than pretending to a precision it does not have.
coverage bias for a regime-selection bias. The honest position is that a fitted
threshold is hypersensitive to window choice at this sample size — which is the
strongest argument for making the unfitted shipped rule the headline.
### Considered and not done
**An ETF credit proxy (HYG/IEF) to extend W3 back over the whole sample.** It
would trade "two sensors versus three" for "proxy sensor versus real sensor" —
still a construct straddle, but no longer flagged by the coverage split. This is
the same objection that rejected `BAA10Y` as a percentile reference. If ever
revisited, check the impulse correlation on the three years of real-OAS overlap
first and report it as a sensitivity, never as the headline.
**A depth sweep (5%/7%/15% corrections) for more events.** `EVENT_COOLDOWN_DAYS`
is 40, so at shallower thresholds re-triggers inside a single decline merge or
drop and the denominator moves for cooldown reasons rather than market ones.
## Resolved in v4 (raised 2026-08-07, shipped 2026-08-08)