Event study: report its own statistical limits

The v3 cutover run scored 2/4 corrections warned against v2's 3/4, which reads
like a regression and is not one. Only 4 of the 11 detected corrections fall in
the holdout, so recall is one event from a different headline -- and the event
that flips is decided by threshold placement, not by what the score saw. "v3
without the credit sensor" catches 2025-02-21 at a *higher* threshold (35.5)
than shipped v3 misses it at (32.3), because the alarm rule needs a rising edge
and a lower threshold can fire outside the horizon then never reset below.

Two caveats are now computed and surfaced rather than left for the reader to
infer:

- Holdout event count against MIN_EVENTS_FOR_CONFIDENCE. The summary sentence
  states how many of the detected corrections actually fall in the test period.
- Warning-sensor coverage across the split. The score renormalises over what is
  available, so a training window predating a sensor's history freezes the
  threshold on a different construct than the holdout is measured against. At
  the cutover that is 39% of training sessions with all three sensors versus
  100% of the test period, credit history beginning 2023-07-25.

Restricting the threshold to sensor-matched training sessions was tested and
rejected: those sessions are a calm recent stretch, so the threshold falls from
32.3 to 22.5 and false alarms rise from 3.3 to 8.6/yr. It swaps a coverage bias
for a regime-selection bias. The report states its limits instead.

_warning_series now returns per-session sensor counts alongside the scores.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

frontend/src/lib/types.ts

@@ -604,6 +604,18 @@ export interface EventStudyReport {
     warn_threshold: number;
     basket_hash: string;
     basket_asof: string;
+    credit_sensor_from?: string | null;
+  };
+  /** How far the headline metrics can be trusted. See _reliability(). */
+  reliability?: {
+    events_detected: number;
+    events_in_holdout: number;
+    minimum_events: number;
+    underpowered: boolean;
+    sensors_expected: number;
+    train_full_sensor_share: number;
+    holdout_full_sensor_share: number;
+    sensor_coverage_mismatch: boolean;
   };
   sample?: {
     start: string;

frontend/src/pages/RegimePage.tsx

@@ -271,6 +271,32 @@ function EventStudyBody({ report }: { report: EventStudyReport }) {
           </table>
         </div>
       )}
+      {report.reliability && (report.reliability.underpowered || report.reliability.sensor_coverage_mismatch) && (
+        <Callout variant="warning">
+          <div className="space-y-1.5">
+            {report.reliability.underpowered && (
+              <p>
+                <strong>Underpowered.</strong> Only {report.reliability.events_in_holdout} of{' '}
+                {report.reliability.events_detected} detected corrections fall in the test period (
+                {report.reliability.minimum_events}+ needed). Recall is one event away from a materially
+                different headline, and which events flip is usually decided by where the frozen threshold
+                lands rather than by what the score saw. Read the direction, not the ratio.
+              </p>
+            )}
+            {report.reliability.sensor_coverage_mismatch && (
+              <p>
+                <strong>Sensor coverage differs across the split.</strong>{' '}
+                {report.reliability.train_full_sensor_share}% of training sessions had all{' '}
+                {report.reliability.sensors_expected} Warning sensors versus{' '}
+                {report.reliability.holdout_full_sensor_share}% of test sessions
+                {report.params?.credit_sensor_from && ` — credit history begins ${report.params.credit_sensor_from}`}
+                . The score renormalises over what is available, so the threshold was frozen on a partly
+                different construct than it is measured against.
+              </p>
+            )}
+          </div>
+        </Callout>
+      )}
       <p className="text-[11px] leading-relaxed text-gray-600">
         The threshold is frozen on the training period and measured on the chronological test period. Reconstructed
         pre-freeze basket history remains exploratory.