feat(risk-monitor): measure the rule that fires, and give fundamentals their own channel
The Warning study measured a fitted percentile crossing that nothing consumes. What reaches Telegram is a quadrant change: fixed 50/40 dividers, hysteresis, two-session confirmation, 3-day cooldown. Those thresholds are constants, not fits, so there is no training set to protect and all 11 detected corrections are evaluable instead of the 4 that fell in a holdout. Replaying it: 1/10 corrections, 0.9 false alarms/year. Random alarms at the same firing rate match or beat that in 65% of draws. The panel now carries ablations (does the quadrant machinery earn its place?), external baselines (does the score earn its complexity?), and that null, because a bare "2 of 4" was unreadable in either direction. Nothing in the alert path was retuned on the strength of it. Fundamentals become a third channel rather than a term in either score. v3 cut them arguing 12+8 of 100 points "could not change any published conclusion" -- true only when every technical sensor reads zero; weighted they moved the bar for the 40 divider from 40 to 25. But no fusion weight is measurable either: with ~10 events and no fundamental history, any weight is a policy preference presented as a measurement. So the read is a categorical state (supportive/neutral/adverse/ unknown) with an evidence grade, derived by fixed rules from stored facts, read by confluence. The LLM extracts and explains; it does not score. Absence stays absence throughout. `unknown` is unreachable by averaging, a stale or empty observation may display but never confirm, extraction failures map to `unknown` rather than `mixed`, and the study rows are coverage-matched and marked not-measurable until enough corrections are covered -- otherwise a fortnight of observations renders as 0/10 and reads as a failed test. Observations become a real time series (migration 033); they lived in a single overwritten settings slot, so no history existed to replay. Pre-rename snapshots are adapted rather than discarded. METHODOLOGY stays v4 -- no score changed -- so no reseed; STUDY_SCHEMA moves to 3 and discards the cached report. Post-deploy: re-run Event Study from Admin -> Jobs. The panel reads "not run yet" until then. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -14,6 +14,7 @@ from app.models.settings import SystemSetting, IngestionProgress
|
||||
from app.models.alert import AlertLog
|
||||
from app.models.paper_trade import PaperTrade
|
||||
from app.models.regime_snapshot import RegimeSnapshot
|
||||
from app.models.regime_fundamental_observation import RegimeFundamentalObservation
|
||||
from app.models.benchmark_price import BenchmarkPrice
|
||||
from app.models.signal_context_snapshot import SignalContextSnapshot
|
||||
from app.models.system_event import SystemEvent
|
||||
@@ -39,6 +40,7 @@ __all__ = [
|
||||
"AlertLog",
|
||||
"PaperTrade",
|
||||
"RegimeSnapshot",
|
||||
"RegimeFundamentalObservation",
|
||||
"BenchmarkPrice",
|
||||
"SignalContextSnapshot",
|
||||
"SystemEvent",
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
from datetime import date as date_type
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Date, DateTime, Float, String, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class RegimeFundamentalObservation(Base):
|
||||
"""Point-in-time record of the sourced hyperscaler capex / earnings read.
|
||||
|
||||
One row per ``effective_date`` (unique, upserted). Before this table the
|
||||
observation lived in a single ``SystemSetting`` slot, so every refresh
|
||||
overwrote the previous one and no history existed at all — which made the
|
||||
read impossible to replay, impossible to backtest, and meant a snapshot
|
||||
rebuild could only ever score historical sessions as if nothing had been
|
||||
observed.
|
||||
|
||||
The read is a categorical channel reported beside State and Warning, never a
|
||||
term in either, so this series is not a scoring input. It is the record that
|
||||
makes the channel replayable at all -- and the only route to eventually
|
||||
testing whether it improves prediction conditional on Warning, which is the
|
||||
one thing that could justify combining the channels later.
|
||||
|
||||
``effective_date`` rather than ``fetched_at`` is the key: it is the session
|
||||
the observation becomes usable on (normally the next weekday), and the gate
|
||||
that stops a rebuild stamping today's reading onto historical rows.
|
||||
"""
|
||||
|
||||
__tablename__ = "regime_fundamental_observations"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
effective_date: Mapped[date_type] = mapped_column(
|
||||
Date, nullable=False, unique=True, index=True
|
||||
)
|
||||
f1_score: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
f3_score: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
capex_json: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
good_news_stock_down: Mapped[str] = mapped_column(String(10), nullable=False)
|
||||
reasoning: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
source: Mapped[str] = mapped_column(String(30), nullable=False)
|
||||
fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
+8
-2
@@ -1351,8 +1351,11 @@ async def run_event_study_job() -> None:
|
||||
report = await run_event_study_and_store(db)
|
||||
|
||||
_runtime_progress(job_name, processed=1, total=1)
|
||||
shipped = report.get("shipped") or {}
|
||||
if report.get("available"):
|
||||
metrics = report.get("metrics") or {}
|
||||
# The shipped quadrant rule is the headline; the fitted-threshold
|
||||
# variant lives under report["fitted"] and is not what fires.
|
||||
metrics = shipped.get("metrics") or {}
|
||||
msg = (
|
||||
f"{metrics.get('events_warned', 0)}/{metrics.get('events', 0)} warned, "
|
||||
f"{metrics.get('false_alarms_per_year', 0)} false alarms/year"
|
||||
@@ -1360,7 +1363,10 @@ async def run_event_study_job() -> None:
|
||||
else:
|
||||
msg = report.get("reason", "no data")
|
||||
_runtime_finish(job_name, "completed", processed=1, total=1, message=msg)
|
||||
_log_event(logging.INFO, "job_complete", job=job_name, events=len(report.get("events", [])))
|
||||
_log_event(
|
||||
logging.INFO, "job_complete", job=job_name,
|
||||
events=len(shipped.get("events") or []),
|
||||
)
|
||||
except Exception as exc:
|
||||
_runtime_finish(job_name, "error", processed=0, total=1, message=str(exc))
|
||||
_log_event(logging.ERROR, "job_error", job=job_name, error_type=type(exc).__name__, message=str(exc))
|
||||
|
||||
@@ -97,6 +97,14 @@ SIGNAL_BUNDLE_MAX_CHARS = 3900 # Telegram limit is 4096; keep room for HTML par
|
||||
# Hysteresis (a deadband around each divider) stops a point sitting on a boundary
|
||||
# from flip-flopping; the cooldown caps how often a genuine change can re-alert.
|
||||
QUAD_TYPE = "regime_quadrant"
|
||||
# The fundamental channel gets its own alerts rather than shifting a score:
|
||||
# "the context changed" and "both channels are elevated" are different facts from
|
||||
# "the market axes moved", and fusing them into one number would destroy exactly
|
||||
# the information an operator uses to decide how much the alert is worth.
|
||||
FUND_TYPE = "regime_fundamental"
|
||||
CONFLUENCE_TYPE = "regime_confluence"
|
||||
# States that count as fundamental risk for the confluence test.
|
||||
FUND_ADVERSE = "adverse"
|
||||
QUAD_X_DIV = 50.0 # v3 State divider (backend response is authoritative)
|
||||
QUAD_Y_DIV = 40.0 # v3 Warning divider; the axes have different ranges
|
||||
QUAD_MARGIN = 5.0 # half-width of the hysteresis deadband around each divider
|
||||
@@ -859,16 +867,111 @@ async def _collect_regime_quadrant(db: AsyncSession) -> list[tuple[str, str]]:
|
||||
)
|
||||
else:
|
||||
metrics = f"State {x:.0f} · Warning {y:.0f}"
|
||||
# The fundamental channel is reported, never added in: this alert is about
|
||||
# the two market axes, and the context is stated beside them so a reader can
|
||||
# judge confluence themselves rather than being handed a fused number.
|
||||
context = data.get("fundamental_context") or {}
|
||||
context_line = (
|
||||
f"fundamentals: {context.get('state', 'unknown')} "
|
||||
f"({context.get('evidence_quality', 'unavailable')})\n"
|
||||
)
|
||||
text = (
|
||||
f"🧭 <b>AI/Tech risk quadrant change</b>\n"
|
||||
f"{QUAD_LABELS.get(prev, prev)} → {QUAD_LABELS.get(new_q, new_q)}\n"
|
||||
f"{metrics}\n"
|
||||
f"{context_line}"
|
||||
f"coverage: state {state.get('coverage'):.0f}% / warning {warning.get('coverage'):.0f}%\n"
|
||||
f"<i>Risk thermometer - not a trade signal.</i>"
|
||||
)
|
||||
return [(_quadrant_log_key(new_q, x, y, basket_hash), text)]
|
||||
|
||||
|
||||
async def _last_logged_key(db: AsyncSession, alert_type: str) -> str | None:
|
||||
"""Most recent logged key for a type, our baseline for change detection."""
|
||||
result = await db.execute(
|
||||
select(AlertLog.dedup_key)
|
||||
.where(AlertLog.alert_type == alert_type)
|
||||
.order_by(AlertLog.created_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
row = result.first()
|
||||
return row[0] if row else None
|
||||
|
||||
|
||||
async def _collect_regime_fundamental(db: AsyncSession) -> list[tuple[str, str, str]]:
|
||||
"""Fundamental-context changes and market/fundamental confluence.
|
||||
|
||||
Two triggers, deliberately separate from the quadrant alert and from each
|
||||
other, because they answer different questions: *what the evidence says* and
|
||||
*whether both channels agree*. Neither is derived by moving a score.
|
||||
|
||||
``unknown`` never alerts. An absence of evidence is not a change in the
|
||||
evidence, and alerting on it would train the reader to ignore the channel.
|
||||
Both seed silently on first run, exactly as the quadrant alert does.
|
||||
"""
|
||||
from app.services.regime_monitor_service import get_regime_monitor
|
||||
|
||||
data = await get_regime_monitor(db)
|
||||
if not data.get("available"):
|
||||
return []
|
||||
warning = data.get("warning") or {}
|
||||
context = data.get("fundamental_context") or {}
|
||||
state = str(context.get("state") or "unknown")
|
||||
# `usable`, not `available`: the state is deliberately preserved past its
|
||||
# staleness horizon so the card can keep showing the last thing observed, and
|
||||
# an observation whose extraction failed is fresh but knows nothing. Neither
|
||||
# may confirm anything — without this gate a months-old adverse read silently
|
||||
# corroborates every new Warning crossing forever, which is the strongest
|
||||
# claim this channel makes and the one it has least right to make.
|
||||
usable = bool(context.get("usable"))
|
||||
score = warning.get("score")
|
||||
|
||||
quality = data.get("data_quality") or {}
|
||||
if not quality.get("is_fresh") or float(warning.get("coverage") or 0) < 75:
|
||||
return []
|
||||
|
||||
quadrant_cfg = data.get("quadrant_config") or {}
|
||||
y_div = float(quadrant_cfg.get("warning_divider", QUAD_Y_DIV))
|
||||
warning_elevated = score is not None and float(score) >= y_div
|
||||
|
||||
out: list[tuple[str, str, str]] = []
|
||||
|
||||
previous_state = await _last_logged_key(db, FUND_TYPE)
|
||||
if previous_state is None:
|
||||
_log_alert(db, FUND_TYPE, state) # seed
|
||||
elif previous_state != state and state != "unknown" and usable:
|
||||
effective = context.get("effective_date")
|
||||
out.append((
|
||||
FUND_TYPE,
|
||||
state,
|
||||
f"📋 <b>Fundamental context changed</b>\n"
|
||||
f"{previous_state} → {state}\n"
|
||||
f"evidence: {context.get('evidence_quality', 'unavailable')}"
|
||||
+ (f" · effective {effective}" if effective else "")
|
||||
+ "\n<i>Context channel — not a score, not a trade signal.</i>",
|
||||
))
|
||||
|
||||
confluence = "yes" if (warning_elevated and state == FUND_ADVERSE and usable) else "no"
|
||||
previous_confluence = await _last_logged_key(db, CONFLUENCE_TYPE)
|
||||
if previous_confluence is None:
|
||||
_log_alert(db, CONFLUENCE_TYPE, confluence) # seed
|
||||
elif previous_confluence != confluence and confluence == "yes":
|
||||
out.append((
|
||||
CONFLUENCE_TYPE,
|
||||
confluence,
|
||||
f"⚠️ <b>Confluence: market and fundamental risk both elevated</b>\n"
|
||||
f"Warning {float(score):.0f} (≥ {y_div:.0f}) with fundamentals {state}\n"
|
||||
f"evidence: {context.get('evidence_quality', 'unavailable')}\n"
|
||||
f"<i>Highest attention. Still a thermometer — not a trade signal.</i>",
|
||||
))
|
||||
elif previous_confluence != confluence:
|
||||
# Falling out of confluence is a state change worth recording as the new
|
||||
# baseline, but not worth a message.
|
||||
_log_alert(db, CONFLUENCE_TYPE, confluence)
|
||||
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dispatch
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -961,6 +1064,11 @@ async def dispatch_alerts(db: AsyncSession) -> dict:
|
||||
# cooldown/hysteresis handled in the collector (like score drops)
|
||||
for key, text in await _collect_regime_quadrant(db):
|
||||
outgoing.append((QUAD_TYPE, key, text))
|
||||
# Deliberately three separate messages off one toggle, not one fused
|
||||
# signal: the market axes and the fundamental channel are different kinds
|
||||
# of evidence, and an operator needs to know which one moved.
|
||||
for alert_type, key, text in await _collect_regime_fundamental(db):
|
||||
outgoing.append((alert_type, key, text))
|
||||
|
||||
if cfg["trade_closed"]:
|
||||
for key, text, pnl_usd in await _collect_closed_trades(db):
|
||||
|
||||
@@ -1,15 +1,48 @@
|
||||
"""Compact chronological validation for the AI/Tech Risk Monitor warning score.
|
||||
"""Chronological validation for the AI/Tech Risk Monitor warning score.
|
||||
|
||||
The study calls its outcome a 10% correction, uses the first 70% of sessions to
|
||||
freeze an 80th-percentile warning threshold, and reports alarm episodes only on
|
||||
the final 30%. It is still labelled exploratory while the fixed breadth basket
|
||||
is reconstructed before its freeze date.
|
||||
The outcome is a 10% correction in the leader, never a regime break. Two rules
|
||||
are measured against it, and they answer different questions:
|
||||
|
||||
* **shipped** -- the quadrant-change rule that actually reaches Telegram
|
||||
(``alert_service._collect_regime_quadrant``). Its thresholds are fixed
|
||||
constants chosen by scenario arithmetic, so nothing is fitted, so there is no
|
||||
training set to protect and the whole sample is evaluable. This is the
|
||||
headline.
|
||||
* **fitted** -- the original study: an 80th-percentile Warning threshold frozen
|
||||
on the first 70% of sessions and measured on the last 30%. Kept because it is
|
||||
what the methodology document reports, and because a fitted threshold is a
|
||||
genuinely different question -- but it is measured on the four corrections that
|
||||
happen to fall in the holdout, which is too few to read as a property of the
|
||||
score.
|
||||
|
||||
Both are scored by the same ``evaluate_alarms`` harness, alongside ablations
|
||||
(does the quadrant machinery earn its place?), external baselines (does the
|
||||
score earn its complexity?), and a random-alarm null (is any of this better than
|
||||
chance?). Without those rows a bare "2 of 4" is unreadable in either direction.
|
||||
|
||||
The fundamental channel is compared, never fused. It appears as its own rule
|
||||
(transitions into an adverse state), as a confluence gate (a market crossing kept
|
||||
only when the state agrees), and as a market-only comparator over the identical
|
||||
window -- because with ~10 correction events and almost no fundamental history,
|
||||
any weight that combined it with the market axes would be a policy preference
|
||||
presented as a measurement.
|
||||
|
||||
Those three rows are **coverage-matched**: scored only on the sessions where the
|
||||
channel had usable context and on the corrections whose warning horizon fell
|
||||
inside it, and marked ``measurable: false`` until enough corrections are covered.
|
||||
A fundamental rule scores zero whether it is wrong or merely absent, so scoring
|
||||
it over the market rows' full sample would turn a fortnight of observations into
|
||||
a 0/10 that reads as a failed test.
|
||||
|
||||
Still labelled exploratory while the fixed breadth basket is reconstructed
|
||||
before its freeze date.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import random
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
@@ -17,11 +50,26 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.services import breadth_service, settings_store
|
||||
from app.services import regime_monitor_service as rms
|
||||
from app.services.admin_service import update_setting
|
||||
from app.services.alert_service import (
|
||||
QUAD_COOLDOWN_DAYS,
|
||||
QUAD_MARGIN,
|
||||
QUAD_X_DIV,
|
||||
QUAD_Y_DIV,
|
||||
_classify_quadrant,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
KEY_REPORT = "regime_event_study"
|
||||
|
||||
# Report shape, independent of METHODOLOGY. A cached report from an older shape
|
||||
# parses fine and reports the current methodology, so without this check the
|
||||
# panel would render a report missing half its blocks. Bumping discards the cache
|
||||
# the way a methodology change does -- and it is the *only* thing that does so
|
||||
# here, because the fundamental-channel rework left METHODOLOGY on v4 (the scores
|
||||
# did not change), so the methodology check cannot catch a stale report.
|
||||
STUDY_SCHEMA = 3
|
||||
|
||||
EVENT_THRESHOLD_PCT = 10.0
|
||||
EVENT_COOLDOWN_DAYS = 40
|
||||
DRAWDOWN_LOOKBACK = 252
|
||||
@@ -33,6 +81,21 @@ TRAIN_FRACTION = 0.70
|
||||
MIN_EVENTS_FOR_CONFIDENCE = 8
|
||||
SENSOR_MISMATCH_TOLERANCE = 0.10
|
||||
|
||||
# _collect_regime_quadrant confirms against get_regime_history(db, days=14), so a
|
||||
# prior session older than that window is not available to confirm with.
|
||||
QUAD_HISTORY_DAYS = 14
|
||||
# Quadrants with Warning above its divider: "1" early warning, "2" active stress.
|
||||
WARNING_QUADRANTS = ("1", "2")
|
||||
STRESS_QUADRANT = ("2",)
|
||||
|
||||
# Draws for the random-alarm null. Seeded, because a cached report that moves
|
||||
# on re-run for RNG reasons is worse than no report.
|
||||
NULL_DRAWS = 2000
|
||||
NULL_SEED = 20260812
|
||||
|
||||
BASELINE_SMA_WINDOW = 50
|
||||
BASELINE_VIX_LEVEL = 20.0
|
||||
|
||||
|
||||
def _median(values: list[float]) -> float | None:
|
||||
if not values:
|
||||
@@ -148,41 +211,355 @@ def evaluate_alarms(
|
||||
}
|
||||
|
||||
|
||||
def _warning_series(
|
||||
def _score_rule(
|
||||
alarm_indices: list[int],
|
||||
event_indices: list[int],
|
||||
dates: list[date],
|
||||
horizon: int,
|
||||
sessions: int,
|
||||
) -> dict:
|
||||
"""``evaluate_alarms`` plus the annualised false-alarm rate for one rule.
|
||||
|
||||
The rate is ``None`` when the rule had no eligible sessions. Dividing by a
|
||||
tiny floor instead produced 5e9 alarms/year for a coverage-matched rule with
|
||||
an empty window -- a number that means "undefined" while looking like a
|
||||
measurement, which is the failure mode this whole panel is built to avoid.
|
||||
"""
|
||||
metrics = evaluate_alarms(alarm_indices, event_indices, dates, horizon)
|
||||
metrics["false_alarms_per_year"] = (
|
||||
round(metrics["false_alarms"] / (sessions / 252.0), 2) if sessions > 0 else None
|
||||
)
|
||||
return metrics
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The shipped rule
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _axis_rows(
|
||||
prices: dict[str, rms.Series],
|
||||
breadth_divergence: dict[date, float],
|
||||
vix_series: rms.Series | None,
|
||||
oas_series: rms.Series | None,
|
||||
breadth_series: rms.Series | None,
|
||||
divergence_series: rms.Series | None,
|
||||
dates: list[date],
|
||||
config: dict,
|
||||
oas_series: rms.Series | None = None,
|
||||
) -> tuple[dict[date, float], dict[date, int]]:
|
||||
"""Warning score per session plus how many sensors backed it.
|
||||
observations: list[dict] | None = None,
|
||||
) -> dict[date, dict]:
|
||||
"""State and Warning per session, from the function that writes snapshots.
|
||||
|
||||
v2 re-derived this by hand from ``WARNING_WEIGHTS`` and so would have kept
|
||||
measuring the old construct after a scoring change. Since v3 dropped
|
||||
fundamentals from the score, this is now exactly the live Warning score
|
||||
rather than a technical-only approximation of it.
|
||||
Calling ``_compute_index`` rather than re-deriving the two axes is the same
|
||||
anti-drift argument that produced ``warning_sensor_scores``: the v2 study
|
||||
re-derived Warning by hand and would have kept measuring the old construct
|
||||
through a scoring change. State has no such shared helper, so the whole
|
||||
snapshot builder is the shared definition.
|
||||
|
||||
The sensor count matters because the score renormalises over whatever is
|
||||
available: a session backed by two sensors is not drawn from the same
|
||||
distribution as one backed by three, and the frozen threshold assumes it is.
|
||||
``observations`` is the point-in-time fundamental series. It does not enter
|
||||
either score -- the fundamental channel is categorical and read by confluence
|
||||
-- but the per-session ``fundamental_state`` it produces is what the
|
||||
confluence rule below is measured on, so it has to be the same series
|
||||
production reports from. Every variant in this module reads its Warning from
|
||||
these rows, so there is no second derivation to fall out of step.
|
||||
"""
|
||||
tickers = config["tickers"]
|
||||
smh_full = prices.get(tickers["leaders"][0], [])
|
||||
spy_full = prices.get(tickers["market"], [])
|
||||
out: dict[date, float] = {}
|
||||
backing: dict[date, int] = {}
|
||||
rows: dict[date, dict] = {}
|
||||
for session in dates:
|
||||
sensors = rms.warning_sensor_scores(
|
||||
breadth_divergence.get(session),
|
||||
rms._closes_asof(smh_full, session),
|
||||
rms._closes_asof(spy_full, session),
|
||||
rms._window_asof(oas_series, session, rms.HY_OAS_WINDOW_DAYS),
|
||||
snapshot = rms._compute_index(
|
||||
prices,
|
||||
vix_series,
|
||||
oas_series,
|
||||
{},
|
||||
config,
|
||||
session,
|
||||
breadth_series=breadth_series,
|
||||
divergence_series=divergence_series,
|
||||
observations=observations or [],
|
||||
)
|
||||
score = rms.score_warning_sensors(sensors)
|
||||
if score is not None:
|
||||
out[session] = round(score, 2)
|
||||
backing[session] = sum(1 for value in sensors.values() if value is not None)
|
||||
return out, backing
|
||||
state = snapshot["state"]
|
||||
warning = snapshot["warning"]
|
||||
rows[session] = {
|
||||
"state": state.get("score"),
|
||||
"warning": warning.get("score"),
|
||||
"fundamental_state": (snapshot.get("fundamental_context") or {}).get("state"),
|
||||
# `usable`, not `available`: a stale observation keeps its state for
|
||||
# display but stops counting as evidence, and an observation whose
|
||||
# extraction failed on everything is fresh but knows nothing. Either
|
||||
# one counted here would inflate the covered window with sessions the
|
||||
# channel could not have contributed to.
|
||||
"fundamental_usable": bool(
|
||||
(snapshot.get("fundamental_context") or {}).get("usable")
|
||||
),
|
||||
"state_coverage": state.get("coverage") or 0.0,
|
||||
"warning_coverage": warning.get("coverage") or 0.0,
|
||||
# The score renormalises over available sensors, so a session backed
|
||||
# by two is not drawn from the same distribution as one backed by
|
||||
# three, and a frozen threshold assumes it is.
|
||||
"warning_sensors": len(warning.get("available_pillars") or []),
|
||||
"inputs_fresh": bool((snapshot.get("data_quality") or {}).get("inputs_fresh")),
|
||||
}
|
||||
return rows
|
||||
|
||||
|
||||
def _publishable(row: dict | None) -> bool:
|
||||
"""What ``get_regime_history`` leaves for the alert to confirm against.
|
||||
|
||||
Deliberately not freshness-gated: ``_collect_regime_quadrant`` checks
|
||||
``is_fresh`` on today's live reading only, while the prior session comes from
|
||||
stored history where the only filter is a published band on both axes.
|
||||
"""
|
||||
return (
|
||||
row is not None
|
||||
and row["state"] is not None
|
||||
and row["warning"] is not None
|
||||
and row["state_coverage"] >= rms.MIN_COVERAGE
|
||||
and row["warning_coverage"] >= rms.MIN_COVERAGE
|
||||
)
|
||||
|
||||
|
||||
def _prior_publishable(
|
||||
rows: dict[date, dict], dates: list[date], index: int, history_days: int
|
||||
) -> dict | None:
|
||||
"""``valid[-2]``: the previous published session inside the 14-day window.
|
||||
|
||||
The monitor writes today's snapshot before the alert step runs
|
||||
(``job_catalog._DAILY_PIPELINE_STEPS``), so ``valid[-1]`` is today and this
|
||||
is genuinely the prior session rather than t-2.
|
||||
"""
|
||||
cutoff = dates[index] - timedelta(days=history_days)
|
||||
for position in range(index - 1, -1, -1):
|
||||
if dates[position] < cutoff:
|
||||
return None
|
||||
candidate = rows.get(dates[position])
|
||||
if _publishable(candidate):
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def replay_quadrant_changes(
|
||||
rows: dict[date, dict],
|
||||
dates: list[date],
|
||||
state_divider: float = QUAD_X_DIV,
|
||||
warning_divider: float = QUAD_Y_DIV,
|
||||
margin: float = QUAD_MARGIN,
|
||||
cooldown_days: int = QUAD_COOLDOWN_DAYS,
|
||||
history_days: int = QUAD_HISTORY_DAYS,
|
||||
) -> list[dict]:
|
||||
"""Every quadrant change the shipped alert would have sent, in order.
|
||||
|
||||
A faithful replay of ``_collect_regime_quadrant``, including three details a
|
||||
state machine written from first principles gets wrong:
|
||||
|
||||
* the prior session is classified against the *current baseline*, not against
|
||||
its own predecessor, so confirmation asks "did yesterday already look like
|
||||
this change" rather than "did yesterday change too";
|
||||
* the baseline advances only when an alert actually fires, so a change that
|
||||
fails confirmation or cooldown is re-evaluated against the old quadrant on
|
||||
the next session rather than being forgotten;
|
||||
* one cooldown is shared by every quadrant change, so a 3->4 alert can
|
||||
swallow a 4->2 alert three days later.
|
||||
|
||||
Returns the fires themselves rather than alarm indices, because which
|
||||
transitions count as a *warning* is the caller's question: entering
|
||||
Warning-high territory and entering both-high territory are different rules
|
||||
over the same replay.
|
||||
"""
|
||||
fires: list[dict] = []
|
||||
baseline: str | None = None
|
||||
baseline_date: date | None = None
|
||||
|
||||
for index, session in enumerate(dates):
|
||||
row = rows.get(session)
|
||||
if not _publishable(row) or not row["inputs_fresh"]:
|
||||
continue
|
||||
x, y = float(row["state"]), float(row["warning"])
|
||||
|
||||
if baseline is None: # seeds silently, exactly as a fresh install does
|
||||
baseline = _classify_quadrant(x, y, None, margin, state_divider, warning_divider)
|
||||
baseline_date = session
|
||||
continue
|
||||
|
||||
new_quadrant = _classify_quadrant(x, y, baseline, margin, state_divider, warning_divider)
|
||||
if new_quadrant == baseline:
|
||||
continue
|
||||
|
||||
prior = _prior_publishable(rows, dates, index, history_days)
|
||||
if prior is None:
|
||||
continue
|
||||
prior_quadrant = _classify_quadrant(
|
||||
float(prior["state"]), float(prior["warning"]),
|
||||
baseline, margin, state_divider, warning_divider,
|
||||
)
|
||||
if prior_quadrant != new_quadrant:
|
||||
continue
|
||||
|
||||
if baseline_date is not None and (session - baseline_date).days < cooldown_days:
|
||||
continue
|
||||
|
||||
fires.append({
|
||||
"index": index,
|
||||
"date": session.isoformat(),
|
||||
"from": baseline,
|
||||
"to": new_quadrant,
|
||||
"state": x,
|
||||
"warning": y,
|
||||
})
|
||||
baseline, baseline_date = new_quadrant, session
|
||||
|
||||
return fires
|
||||
|
||||
|
||||
def entry_alarms(fires: list[dict], entry: tuple[str, ...]) -> list[int]:
|
||||
"""Fires that *enter* the given quadrant set from outside it."""
|
||||
return [f["index"] for f in fires if f["to"] in entry and f["from"] not in entry]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ablations, baselines, null
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _usable_adverse(rows: dict[date, dict], session: date) -> bool:
|
||||
"""Adverse *and* still within its staleness horizon.
|
||||
|
||||
Both callers need this pair, and neither may use the state alone: the state
|
||||
survives going stale so the card can show it, which would otherwise let a
|
||||
months-old read confirm crossings indefinitely.
|
||||
"""
|
||||
row = rows.get(session) or {}
|
||||
return row.get("fundamental_state") == "adverse" and bool(row.get("fundamental_usable"))
|
||||
|
||||
|
||||
def adverse_episodes(
|
||||
rows: dict[date, dict], dates: list[date], start_index: int
|
||||
) -> list[int]:
|
||||
"""Sessions where the fundamental state *becomes* usably adverse.
|
||||
|
||||
The market rules alarm on a rising-edge crossing; a categorical state has no
|
||||
crossing, so its analogue is the transition into ``adverse``. That keeps the
|
||||
row comparable with every other row in the table rather than counting every
|
||||
day the state happens to sit there.
|
||||
"""
|
||||
alarms: list[int] = []
|
||||
was_adverse = start_index > 0 and _usable_adverse(rows, dates[start_index - 1])
|
||||
for index in range(start_index, len(dates)):
|
||||
if dates[index] not in rows:
|
||||
continue
|
||||
adverse = _usable_adverse(rows, dates[index])
|
||||
if adverse and not was_adverse:
|
||||
alarms.append(index)
|
||||
was_adverse = adverse
|
||||
return alarms
|
||||
|
||||
|
||||
def confluence_episodes(
|
||||
warning_alarms: list[int], rows: dict[date, dict], dates: list[date]
|
||||
) -> list[int]:
|
||||
"""Warning crossings that happen while the fundamental state is usably adverse.
|
||||
|
||||
Deliberately gated on the market crossing rather than on either channel
|
||||
moving: it preserves the rising-edge semantics every other row uses, so the
|
||||
column measures "does requiring fundamental agreement help?" instead of a
|
||||
differently-shaped rule that cannot be compared with the others.
|
||||
"""
|
||||
return [index for index in warning_alarms if _usable_adverse(rows, dates[index])]
|
||||
|
||||
|
||||
def covered_events(
|
||||
event_indices: list[int],
|
||||
rows: dict[date, dict],
|
||||
dates: list[date],
|
||||
horizon: int,
|
||||
) -> list[int]:
|
||||
"""Corrections a fundamental rule actually had a chance to warn about.
|
||||
|
||||
An alarm counts only if it fires in ``[event - horizon, event - 1]``, so a
|
||||
correction is *coverable* only if the channel had usable context somewhere in
|
||||
that window. Scoring these rules against every correction instead would make
|
||||
one day of observation render as 0/10 -- an untested rule reported as a
|
||||
failed one, which is the exact mistake the ``measurable`` flag exists to
|
||||
prevent for the empty-table case.
|
||||
"""
|
||||
covered: list[int] = []
|
||||
for event_index in event_indices:
|
||||
window = range(max(0, event_index - horizon), event_index)
|
||||
if any(
|
||||
bool((rows.get(dates[index]) or {}).get("fundamental_usable"))
|
||||
for index in window
|
||||
):
|
||||
covered.append(event_index)
|
||||
return covered
|
||||
|
||||
|
||||
def eligible_sessions(
|
||||
rows: dict[date, dict], dates: list[date], start_index: int
|
||||
) -> int:
|
||||
"""Sessions a fundamental rule could have fired on, for the FA/year rate.
|
||||
|
||||
Annualising over the whole window instead would divide a rule's false alarms
|
||||
by years in which it was structurally incapable of firing, reporting a
|
||||
flattering rate that means nothing.
|
||||
"""
|
||||
return sum(
|
||||
1
|
||||
for session in dates[start_index:]
|
||||
if bool((rows.get(session) or {}).get("fundamental_usable"))
|
||||
)
|
||||
|
||||
|
||||
def below_average_series(
|
||||
series: rms.Series, window: int = BASELINE_SMA_WINDOW
|
||||
) -> dict[date, float]:
|
||||
"""100 while the close sits under its ``window``-session average, else 0."""
|
||||
out: dict[date, float] = {}
|
||||
closes = [value for _, value in series]
|
||||
for index, (session, close) in enumerate(series):
|
||||
if index + 1 < window:
|
||||
continue
|
||||
average = sum(closes[index + 1 - window: index + 1]) / window
|
||||
out[session] = 100.0 if close < average else 0.0
|
||||
return out
|
||||
|
||||
|
||||
def _null_model(
|
||||
alarm_count: int,
|
||||
event_indices: list[int],
|
||||
dates: list[date],
|
||||
horizon: int,
|
||||
start_index: int,
|
||||
observed_warned: int,
|
||||
draws: int = NULL_DRAWS,
|
||||
seed: int = NULL_SEED,
|
||||
) -> dict | None:
|
||||
"""Recall from alarms scattered at random over the same evaluable sessions.
|
||||
|
||||
Drawn only from sessions a real rule could have fired on: over the whole
|
||||
sample the null would be diluted by warm-up sessions and would understate
|
||||
what chance achieves. That matters here -- with ~11 events and a 20-session
|
||||
horizon, a sixth of the sample already sits inside a hit window.
|
||||
|
||||
Corrections cluster, and uniform placement does not, so this is the floor
|
||||
rather than the bar: an alarm process that clusters would beat it for
|
||||
reasons that have nothing to do with foresight.
|
||||
"""
|
||||
population = range(start_index, len(dates))
|
||||
if alarm_count <= 0 or not event_indices or alarm_count > len(population):
|
||||
return None
|
||||
rng = random.Random(seed)
|
||||
recalls: list[int] = []
|
||||
for _ in range(draws):
|
||||
picks = sorted(rng.sample(population, alarm_count))
|
||||
recalls.append(evaluate_alarms(picks, event_indices, dates, horizon)["events_warned"])
|
||||
mean = sum(recalls) / len(recalls)
|
||||
variance = sum((value - mean) ** 2 for value in recalls) / len(recalls)
|
||||
return {
|
||||
"draws": draws,
|
||||
"alarms_per_draw": alarm_count,
|
||||
"events": len(event_indices),
|
||||
"mean_warned": round(mean, 2),
|
||||
"sd_warned": round(variance ** 0.5, 2),
|
||||
"observed_warned": observed_warned,
|
||||
"p_at_least_observed": round(
|
||||
sum(1 for value in recalls if value >= observed_warned) / len(recalls), 3
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _reliability(
|
||||
@@ -192,9 +569,9 @@ def _reliability(
|
||||
events_detected: int,
|
||||
events_in_holdout: int,
|
||||
) -> dict:
|
||||
"""How far the headline metrics can actually be trusted.
|
||||
"""How far the *fitted* variant's headline metrics can be trusted.
|
||||
|
||||
Two things repeatedly invite over-reading this report:
|
||||
Two things repeatedly invite over-reading it:
|
||||
|
||||
* The holdout carries only the corrections that fall in the last 30% of the
|
||||
sample. A "2/4" is one event away from "3/4", and in practice the events
|
||||
@@ -203,6 +580,9 @@ def _reliability(
|
||||
* The score renormalises over available sensors, so a training window that
|
||||
predates a sensor's history freezes a threshold on a different construct
|
||||
than the holdout is measured against.
|
||||
|
||||
Neither applies to the shipped rule, whose thresholds are fixed constants --
|
||||
but the second one does not vanish, it relocates: see ``_era_split``.
|
||||
"""
|
||||
expected = len(rms.WARNING_WEIGHTS)
|
||||
train = [backing[d] for d in dates[:split] if d in backing]
|
||||
@@ -221,6 +601,126 @@ def _reliability(
|
||||
}
|
||||
|
||||
|
||||
def _era_split(
|
||||
alarms: list[int],
|
||||
event_indices: list[int],
|
||||
dates: list[date],
|
||||
horizon: int,
|
||||
start_index: int,
|
||||
credit_from: date | None,
|
||||
) -> dict | None:
|
||||
"""Shipped-rule metrics either side of the credit sensor's first session.
|
||||
|
||||
Dropping the fitted threshold makes the whole sample evaluable, which is the
|
||||
point -- but most of the extra events sit before 2023-08, where W3 does not
|
||||
exist and Warning renormalises to ``(W1*45 + W2*30)/75``. The fixed 40
|
||||
divider is then applied to a different construct than it was reasoned about,
|
||||
so the coverage caveat does not disappear with the split; it relocates from
|
||||
the threshold to the score. Reporting the two eras separately is what keeps
|
||||
the fuller sample from being a differently misleading headline.
|
||||
|
||||
The pre-credit era is close to a "Warning without W3" ablation on real
|
||||
sessions -- and a clean one, because the fundamental channel is not a term in
|
||||
Warning at all, so the two eras differ by W3 and nothing else. That stays
|
||||
true however much fundamental history accumulates.
|
||||
|
||||
Alarms and events are assigned to eras by index, so an alarm days before the
|
||||
boundary that matched an event days after it lands in the earlier era. With
|
||||
the eras years long and the events sparse, that costs nothing.
|
||||
"""
|
||||
if credit_from is None:
|
||||
return None
|
||||
boundary = next(
|
||||
(index for index, session in enumerate(dates) if session >= credit_from), None
|
||||
)
|
||||
if boundary is None or boundary <= start_index or boundary >= len(dates):
|
||||
return None
|
||||
|
||||
def slice_metrics(low: int, high: int) -> dict:
|
||||
sessions = max(0, high - low)
|
||||
metrics = _score_rule(
|
||||
[a for a in alarms if low <= a < high],
|
||||
[e for e in event_indices if low <= e < high],
|
||||
dates, horizon, sessions,
|
||||
)
|
||||
metrics.pop("per_event", None)
|
||||
metrics["sessions"] = sessions
|
||||
return metrics
|
||||
|
||||
return {
|
||||
"credit_from": credit_from.isoformat(),
|
||||
"pre_credit": {
|
||||
"label": "W1+W2 only",
|
||||
"start": dates[start_index].isoformat(),
|
||||
"end": dates[boundary - 1].isoformat(),
|
||||
**slice_metrics(start_index, boundary),
|
||||
},
|
||||
"full_coverage": {
|
||||
"label": "all three sensors",
|
||||
"start": dates[boundary].isoformat(),
|
||||
"end": dates[-1].isoformat(),
|
||||
**slice_metrics(boundary, len(dates)),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _warning_from_rows(
|
||||
rows: dict[date, dict], dates: list[date]
|
||||
) -> tuple[dict[date, float], dict[date, int]]:
|
||||
"""Published Warning per session plus how many sensors backed it.
|
||||
|
||||
Read off ``_axis_rows`` rather than recomputed. v2 re-derived Warning by hand
|
||||
from ``WARNING_WEIGHTS`` and would have kept measuring the old construct
|
||||
after a scoring change; a second derivation here would have done the same to
|
||||
any later change to how Warning is assembled -- silently, in the fitted
|
||||
variant and the ``warning_bare`` ablation, while the shipped replay moved on
|
||||
without it.
|
||||
"""
|
||||
out: dict[date, float] = {}
|
||||
backing: dict[date, int] = {}
|
||||
for session in dates:
|
||||
row = rows.get(session)
|
||||
if row is None or row["warning"] is None:
|
||||
continue
|
||||
out[session] = float(row["warning"])
|
||||
backing[session] = int(row["warning_sensors"])
|
||||
return out, backing
|
||||
|
||||
|
||||
def _rule_row(
|
||||
rule_id: str,
|
||||
label: str,
|
||||
kind: str,
|
||||
note: str,
|
||||
alarms: list[int],
|
||||
event_indices: list[int],
|
||||
dates: list[date],
|
||||
horizon: int,
|
||||
sessions: int,
|
||||
measurable: bool = True,
|
||||
) -> dict:
|
||||
"""One comparison row.
|
||||
|
||||
``measurable=False`` marks a rule whose *input* is too thin to have been
|
||||
tested, not one that failed. A fundamental rule scores 0/N whether it is
|
||||
wrong or merely absent, and a 0/N sitting in this table would read as
|
||||
tested-and-failed -- the same false precision the whole restructure exists to
|
||||
remove. It stays false until the channel has covered
|
||||
``MIN_EVENTS_FOR_CONFIDENCE`` corrections, because a 1/1 or 0/2 over a
|
||||
two-week exposure is not a result either.
|
||||
"""
|
||||
metrics = _score_rule(alarms, event_indices, dates, horizon, sessions)
|
||||
metrics.pop("per_event", None)
|
||||
return {
|
||||
"id": rule_id,
|
||||
"label": label,
|
||||
"kind": kind,
|
||||
"note": note,
|
||||
"measurable": measurable,
|
||||
**metrics,
|
||||
}
|
||||
|
||||
|
||||
async def run_event_study(
|
||||
db: AsyncSession,
|
||||
threshold_pct: float = EVENT_THRESHOLD_PCT,
|
||||
@@ -242,55 +742,202 @@ async def run_event_study(
|
||||
)
|
||||
divergence = breadth_service.compute_divergence_series(breadth, benchmark)
|
||||
oas_series = await rms._fetch_fred_series("BAMLH0A0HYM2", start, end)
|
||||
warning, backing = _warning_series(prices, divergence, dates, config, oas_series)
|
||||
# State needs volatility, which the Warning-only study never fetched.
|
||||
vix_series = await rms._fetch_fred_series("VIXCLS", start, end)
|
||||
# The point-in-time fundamental series. It is not in either score; it drives
|
||||
# the categorical channel the confluence rule below is measured on.
|
||||
observations = await rms.get_fundamental_observations(db)
|
||||
# The credit sensor cannot reach back as far as the price history does (the
|
||||
# upstream series is capped at ~3 years), so the earlier part of the sample
|
||||
# scores on W1+W2 alone via renormalisation. Report where W3 starts rather
|
||||
# than letting the threshold quietly straddle two sensor sets.
|
||||
credit_from = oas_series[0][0].isoformat() if oas_series else None
|
||||
credit_from = oas_series[0][0] if oas_series else None
|
||||
|
||||
all_events = detect_events(closes, dates, threshold_pct)
|
||||
all_event_indices = [event["index"] for event in all_events]
|
||||
|
||||
# --- one pass; every rule below reads its Warning from these rows ----
|
||||
rows = _axis_rows(
|
||||
prices,
|
||||
vix_series,
|
||||
oas_series,
|
||||
rms._mapping_series(breadth),
|
||||
rms._mapping_series(divergence),
|
||||
dates,
|
||||
config,
|
||||
observations,
|
||||
)
|
||||
warning, backing = _warning_from_rows(rows, dates)
|
||||
fires = replay_quadrant_changes(rows, dates)
|
||||
# Nothing can alarm before the baseline seeds, so every rule is measured from
|
||||
# the same session and the comparison stays like-for-like.
|
||||
seeded = next(
|
||||
(
|
||||
index
|
||||
for index, session in enumerate(dates)
|
||||
if _publishable(rows.get(session)) and rows[session]["inputs_fresh"]
|
||||
),
|
||||
None,
|
||||
)
|
||||
if seeded is None:
|
||||
return {"available": False, "reason": "no session with publishable coverage"}
|
||||
evaluable_start = seeded + 1
|
||||
evaluable_sessions = max(1, len(dates) - evaluable_start)
|
||||
evaluable_events = [index for index in all_event_indices if index >= evaluable_start]
|
||||
|
||||
warning_alarms = entry_alarms(fires, WARNING_QUADRANTS)
|
||||
shipped_metrics = _score_rule(
|
||||
warning_alarms, evaluable_events, dates, horizon, evaluable_sessions
|
||||
)
|
||||
shipped_events = shipped_metrics.pop("per_event")
|
||||
|
||||
# --- the fitted variant, kept for continuity -------------------------
|
||||
split = max(1, min(len(dates) - 1, int(len(dates) * TRAIN_FRACTION)))
|
||||
train_values = [warning[d] for d in dates[:split] if d in warning]
|
||||
warn_threshold = _percentile(train_values, WARN_PERCENTILE)
|
||||
if warn_threshold is None:
|
||||
return {"available": False, "reason": "insufficient warning history"}
|
||||
|
||||
all_events = detect_events(closes, dates, threshold_pct)
|
||||
holdout_events = [event["index"] for event in all_events if event["index"] >= split]
|
||||
alarms = alarm_episodes(warning, dates, warn_threshold, start_index=split)
|
||||
metrics = evaluate_alarms(alarms, holdout_events, dates, horizon)
|
||||
holdout_events = [index for index in all_event_indices if index >= split]
|
||||
fitted_alarms = alarm_episodes(warning, dates, warn_threshold, start_index=split)
|
||||
holdout_sessions = max(1, len(dates) - split)
|
||||
metrics["false_alarms_per_year"] = round(
|
||||
metrics["false_alarms"] / (holdout_sessions / 252.0), 2
|
||||
fitted_metrics = _score_rule(
|
||||
fitted_alarms, holdout_events, dates, horizon, holdout_sessions
|
||||
)
|
||||
|
||||
fitted_events = fitted_metrics.pop("per_event")
|
||||
reliability = _reliability(dates, split, backing, len(all_events), len(holdout_events))
|
||||
|
||||
# --- ablations and baselines, all on fixed thresholds ----------------
|
||||
# Fitted thresholds are deliberately excluded here: a threshold fitted on the
|
||||
# full sample would have lookahead the shipped rule does not, and one fitted
|
||||
# on a training split could only be scored on the four holdout events. Fixed
|
||||
# constants keep every row on the same events over the same sessions.
|
||||
state_series = {
|
||||
session: row["state"] for session, row in rows.items() if row["state"] is not None
|
||||
}
|
||||
vix_indicator = {
|
||||
session: value
|
||||
for session in dates
|
||||
if (value := rms._value_asof(vix_series, session)) is not None
|
||||
}
|
||||
# The fundamental channel is categorical and never enters a score, so it is
|
||||
# compared as its own rule and as a confluence gate rather than tuned as a
|
||||
# weight. With an empty observation series both are unmeasurable, and say so.
|
||||
fundamental_alarms = adverse_episodes(rows, dates, evaluable_start)
|
||||
confluence_alarms = confluence_episodes(warning_alarms, rows, dates)
|
||||
# Coverage-matched denominators. These rules only existed on the sessions the
|
||||
# channel had usable context, so scoring them over the whole window would
|
||||
# report an exposure they never had -- and one day of coverage would render
|
||||
# as 0/10.
|
||||
fundamental_events = covered_events(evaluable_events, rows, dates, horizon)
|
||||
fundamental_sessions = eligible_sessions(rows, dates, evaluable_start)
|
||||
fundamental_measurable = len(fundamental_events) >= MIN_EVENTS_FOR_CONFIDENCE
|
||||
comparison = [
|
||||
_rule_row(
|
||||
"fundamental_adverse", "Fundamental context turns adverse", "fundamental",
|
||||
"The third channel on its own: transitions into an adverse capex / "
|
||||
"earnings-reaction state, with no market input at all.",
|
||||
fundamental_alarms, fundamental_events, dates, horizon, fundamental_sessions,
|
||||
measurable=fundamental_measurable,
|
||||
),
|
||||
_rule_row(
|
||||
"confluence", "Confluence: Warning crossing while adverse", "fundamental",
|
||||
"The shipped market crossing, kept only when the fundamental channel "
|
||||
"agrees. Answers whether requiring agreement buys precision, at what "
|
||||
"cost in recall.",
|
||||
confluence_alarms, fundamental_events, dates, horizon, fundamental_sessions,
|
||||
measurable=fundamental_measurable,
|
||||
),
|
||||
_rule_row(
|
||||
"market_over_covered", "Quadrant alert, covered window only", "fundamental",
|
||||
"The shipped market rule scored on exactly the events, sessions and "
|
||||
"alarms the two rows above were scored on. Without it, any difference "
|
||||
"between them and the headline could be the window rather than the "
|
||||
"channel.",
|
||||
# Alarms are restricted to the covered window too: counting crossings
|
||||
# that fired when the channel had no context would compare the market
|
||||
# rule's full exposure against the channel's partial one.
|
||||
[
|
||||
index
|
||||
for index in warning_alarms
|
||||
if index >= evaluable_start
|
||||
and bool((rows.get(dates[index]) or {}).get("fundamental_usable"))
|
||||
],
|
||||
fundamental_events, dates, horizon, fundamental_sessions,
|
||||
measurable=fundamental_measurable,
|
||||
),
|
||||
_rule_row(
|
||||
"quadrant_stress_entry", "Quadrant alert, both axes high", "ablation",
|
||||
"The same replay, recording only entries into the both-high quadrant. "
|
||||
"State is coincident by construction, so requiring it should convert "
|
||||
"leads into confirmations.",
|
||||
entry_alarms(fires, STRESS_QUADRANT),
|
||||
evaluable_events, dates, horizon, evaluable_sessions,
|
||||
),
|
||||
_rule_row(
|
||||
"warning_bare", f"Warning >= {QUAD_Y_DIV:.0f} (bare crossing)", "ablation",
|
||||
"The shipped divider with none of the quadrant machinery: no State "
|
||||
"condition, no hysteresis, no confirmation, no cooldown.",
|
||||
alarm_episodes(warning, dates, QUAD_Y_DIV, start_index=evaluable_start),
|
||||
evaluable_events, dates, horizon, evaluable_sessions,
|
||||
),
|
||||
_rule_row(
|
||||
"state_bare", f"State >= {QUAD_X_DIV:.0f} (bare crossing)", "ablation",
|
||||
"The coincident axis alone. State measures stress that has already "
|
||||
"arrived, so a competitive lead here would be surprising.",
|
||||
alarm_episodes(state_series, dates, QUAD_X_DIV, start_index=evaluable_start),
|
||||
evaluable_events, dates, horizon, evaluable_sessions,
|
||||
),
|
||||
_rule_row(
|
||||
"smh_below_50dma", f"{leader} below its {BASELINE_SMA_WINDOW}-DMA", "baseline",
|
||||
"The crudest possible trend rule, and free.",
|
||||
alarm_episodes(
|
||||
below_average_series(benchmark, BASELINE_SMA_WINDOW), dates,
|
||||
50.0, start_index=evaluable_start,
|
||||
),
|
||||
evaluable_events, dates, horizon, evaluable_sessions,
|
||||
),
|
||||
_rule_row(
|
||||
"vix_level", f"VIX >= {BASELINE_VIX_LEVEL:.0f}", "baseline",
|
||||
"The market's own risk gauge, unweighted and unmodelled.",
|
||||
alarm_episodes(
|
||||
vix_indicator, dates, BASELINE_VIX_LEVEL, start_index=evaluable_start
|
||||
),
|
||||
evaluable_events, dates, horizon, evaluable_sessions,
|
||||
),
|
||||
]
|
||||
|
||||
null_model = _null_model(
|
||||
len(warning_alarms), evaluable_events, dates, horizon,
|
||||
evaluable_start, shipped_metrics["events_warned"],
|
||||
# Passed rather than defaulted: a default argument binds the constant at
|
||||
# import, so overriding it (in tests) would silently do nothing.
|
||||
draws=NULL_DRAWS, seed=NULL_SEED,
|
||||
)
|
||||
eras = _era_split(
|
||||
warning_alarms, evaluable_events, dates, horizon, evaluable_start, credit_from
|
||||
)
|
||||
basket_asof = date.fromisoformat(config["basket_asof"])
|
||||
retrospective = dates[split] < basket_asof
|
||||
retrospective = dates[evaluable_start] < basket_asof
|
||||
evaluation = "exploratory" if retrospective else "holdout"
|
||||
lead_text = (
|
||||
f"median lead {metrics['median_lead_days']:.0f} sessions"
|
||||
if metrics["median_lead_days"] is not None
|
||||
f"median lead {shipped_metrics['median_lead_days']:.0f} sessions"
|
||||
if shipped_metrics["median_lead_days"] is not None
|
||||
else "no successful warning lead"
|
||||
)
|
||||
summary = (
|
||||
f"{evaluation.capitalize()} chronological test: warning episodes preceded "
|
||||
f"{metrics['events_warned']}/{metrics['events']} 10% corrections; "
|
||||
f"{metrics['events_missed']} missed, {metrics['false_alarms_per_year']:.1f} "
|
||||
f"false alarms/year, {lead_text}. "
|
||||
f"{metrics['events']} of {reliability['events_detected']} detected corrections "
|
||||
f"fall in the test period"
|
||||
+ (
|
||||
"; too few to read recall as a property of the score."
|
||||
if reliability["underpowered"]
|
||||
else "."
|
||||
)
|
||||
f"{evaluation.capitalize()} replay of the shipped quadrant alert over "
|
||||
f"{evaluable_sessions} sessions: it entered Warning-high territory ahead of "
|
||||
f"{shipped_metrics['events_warned']} of {shipped_metrics['events']} 10% "
|
||||
f"corrections, with {shipped_metrics['false_alarms_per_year']:.1f} false "
|
||||
f"alarms/year and {lead_text}. Its dividers are fixed constants rather than "
|
||||
f"fitted, so there is no training split and every detected correction is "
|
||||
f"evaluable — compare it against the ablations and baselines below before "
|
||||
f"reading the ratio as good or bad."
|
||||
)
|
||||
per_event = metrics.pop("per_event")
|
||||
|
||||
report = {
|
||||
"available": True,
|
||||
"schema": STUDY_SCHEMA,
|
||||
"methodology": rms.METHODOLOGY,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"evaluation": evaluation,
|
||||
@@ -301,24 +948,69 @@ async def run_event_study(
|
||||
"event_threshold_pct": threshold_pct,
|
||||
"event_cooldown_days": EVENT_COOLDOWN_DAYS,
|
||||
"horizon_days": horizon,
|
||||
"train_fraction": TRAIN_FRACTION,
|
||||
"warn_percentile": WARN_PERCENTILE,
|
||||
"warn_threshold": round(warn_threshold, 1),
|
||||
"credit_sensor_from": credit_from,
|
||||
"credit_sensor_from": credit_from.isoformat() if credit_from else None,
|
||||
"basket_hash": rms._basket_hash(config["breadth_basket"]),
|
||||
"basket_asof": config["basket_asof"],
|
||||
},
|
||||
# The channel's actual exposure, which is what its rows are scored on.
|
||||
# The series starts empty -- the observation lived in a single
|
||||
# overwritten settings slot until 2026-08-12 -- and it accumulates one
|
||||
# observation at a time, so for a long while these rows are unmeasurable
|
||||
# rather than unsuccessful. Stating the exposure is what stops the table
|
||||
# inventing a failed result out of a thin one.
|
||||
"fundamental_coverage": {
|
||||
"observations": len(observations),
|
||||
"sessions_eligible": fundamental_sessions,
|
||||
"evaluable_sessions": evaluable_sessions,
|
||||
"events_covered": len(fundamental_events),
|
||||
"events_evaluable": len(evaluable_events),
|
||||
"minimum_events": MIN_EVENTS_FOR_CONFIDENCE,
|
||||
"measurable": fundamental_measurable,
|
||||
},
|
||||
"sample": {
|
||||
"start": dates[0].isoformat(),
|
||||
"end": dates[-1].isoformat(),
|
||||
"train_end": dates[split - 1].isoformat(),
|
||||
"test_start": dates[split].isoformat(),
|
||||
"sessions": len(dates),
|
||||
"holdout_sessions": holdout_sessions,
|
||||
# Not "test_start": the shipped rule fits nothing, so this is where
|
||||
# the baseline seeds and every rule becomes measurable, not where a
|
||||
# holdout begins. The fitted variant's split lives under "fitted".
|
||||
"evaluable_from": dates[evaluable_start].isoformat(),
|
||||
"evaluable_sessions": evaluable_sessions,
|
||||
"events_detected": len(all_events),
|
||||
"events_evaluable": len(evaluable_events),
|
||||
},
|
||||
"shipped": {
|
||||
"rule": {
|
||||
"state_divider": QUAD_X_DIV,
|
||||
"warning_divider": QUAD_Y_DIV,
|
||||
"margin": QUAD_MARGIN,
|
||||
"confirm_sessions": 2,
|
||||
"cooldown_days": QUAD_COOLDOWN_DAYS,
|
||||
"entry": "Warning-high quadrant (early warning or active stress)",
|
||||
},
|
||||
"metrics": shipped_metrics,
|
||||
"events": shipped_events,
|
||||
"quadrant_changes": len(fires),
|
||||
"fires": fires,
|
||||
"by_era": eras,
|
||||
},
|
||||
"comparison": comparison,
|
||||
"null_model": null_model,
|
||||
"fitted": {
|
||||
"params": {
|
||||
"train_fraction": TRAIN_FRACTION,
|
||||
"warn_percentile": WARN_PERCENTILE,
|
||||
"warn_threshold": round(warn_threshold, 1),
|
||||
},
|
||||
"sample": {
|
||||
"train_end": dates[split - 1].isoformat(),
|
||||
"test_start": dates[split].isoformat(),
|
||||
"holdout_sessions": holdout_sessions,
|
||||
},
|
||||
"metrics": fitted_metrics,
|
||||
"events": fitted_events,
|
||||
},
|
||||
"metrics": metrics,
|
||||
"reliability": reliability,
|
||||
"events": per_event,
|
||||
"recent_breadth": [
|
||||
{"date": d.isoformat(), "breadth": breadth[d], "warning": warning.get(d)}
|
||||
for d in dates[-90:]
|
||||
@@ -328,10 +1020,13 @@ async def run_event_study(
|
||||
logger.info(json.dumps({
|
||||
"event": "regime_event_study_complete",
|
||||
"evaluation": evaluation,
|
||||
"events": metrics["events"],
|
||||
"events_detected": reliability["events_detected"],
|
||||
"warned": metrics["events_warned"],
|
||||
"false_alarms_per_year": metrics["false_alarms_per_year"],
|
||||
"shipped_events": shipped_metrics["events"],
|
||||
"shipped_warned": shipped_metrics["events_warned"],
|
||||
"shipped_false_alarms_per_year": shipped_metrics["false_alarms_per_year"],
|
||||
"quadrant_changes": len(fires),
|
||||
"fitted_events": fitted_metrics["events"],
|
||||
"fitted_warned": fitted_metrics["events_warned"],
|
||||
"null_p_at_least_observed": (null_model or {}).get("p_at_least_observed"),
|
||||
"underpowered": reliability["underpowered"],
|
||||
"sensor_coverage_mismatch": reliability["sensor_coverage_mismatch"],
|
||||
}))
|
||||
@@ -352,4 +1047,8 @@ async def get_event_study_report(db: AsyncSession) -> dict | None:
|
||||
report = json.loads(setting.value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return report if report.get("methodology") == rms.METHODOLOGY else None
|
||||
if report.get("methodology") != rms.METHODOLOGY:
|
||||
return None
|
||||
# A pre-replay report parses fine and carries the current methodology, so the
|
||||
# shape has to be checked separately or the panel renders a headline-less v4.
|
||||
return report if report.get("schema") == STUDY_SCHEMA else None
|
||||
|
||||
@@ -7,11 +7,17 @@ two deliberately separate outputs:
|
||||
* Warning: deterioration/divergence that may precede State (breadth divergence,
|
||||
relative strength, credit impulse).
|
||||
|
||||
Both scores are quantitative and daily. The sourced hyperscaler capex and
|
||||
earnings-reaction observations are a qualitative *overlay* since v3 rather than
|
||||
weighted sensors: at a combined 20 points they could not reach the event
|
||||
study's alarm threshold even when both pegged, so refreshing them appeared to
|
||||
do nothing. They are reported next to the scores instead of inside them.
|
||||
* Fundamental context: a categorical channel (supportive / neutral / adverse /
|
||||
unknown) with an evidence-quality grade, derived by fixed rules from the
|
||||
sourced hyperscaler capex and earnings-reaction observations.
|
||||
|
||||
Both scores are quantitative and daily. The fundamental channel is deliberately
|
||||
**not** a term in either: the three are read together by confluence, because
|
||||
adding a slow categorical judgement to a fast continuous score manufactures
|
||||
precision by summing unlike things, and any fusion weight would be a policy
|
||||
preference presented as a measurement until there is enough point-in-time
|
||||
history to fit one. A missing observation therefore stays ``unknown`` instead of
|
||||
silently redistributing its weight onto the technical sensors.
|
||||
|
||||
Daily snapshots are the point-in-time record. The first run under a new
|
||||
``METHODOLOGY`` rewrites every session inside ``REBUILD_LOOKBACK_DAYS`` once;
|
||||
@@ -35,6 +41,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.config import settings
|
||||
from app.exceptions import ProviderError, ValidationError
|
||||
from app.models.regime_fundamental_observation import RegimeFundamentalObservation
|
||||
from app.models.regime_snapshot import RegimeSnapshot
|
||||
from app.providers.alpaca import AlpacaOHLCVProvider
|
||||
from app.services import breadth_service, settings_store
|
||||
@@ -55,7 +62,11 @@ METHODOLOGY = "v4"
|
||||
# against the *stored* blob, so omitting the current one discards the observation
|
||||
# on its first write, which leaves fetched_at null and locked false -- and then
|
||||
# update_regime_monitor refreshes it via the LLM on every single run, forever.
|
||||
CATEGORICAL_FUNDAMENTAL_METHODOLOGIES = frozenset({"v2", "v3", "v4"})
|
||||
# "v5" is listed although no v5 scoring exists: a v5 was briefly built (a weighted
|
||||
# fundamental modifier on Warning) and reverted, so a development box can have
|
||||
# that string sitting in its settings blob. Keeping it costs nothing; omitting it
|
||||
# costs the failure above.
|
||||
CATEGORICAL_FUNDAMENTAL_METHODOLOGIES = frozenset({"v2", "v3", "v4", "v5"})
|
||||
|
||||
# Bumped when a fix changes what historical rows *should* contain without
|
||||
# changing the live formula, so stored history needs one reseed. Deliberately
|
||||
@@ -173,6 +184,34 @@ WARNING_WEIGHTS = {
|
||||
"credit_impulse": 25.0,
|
||||
}
|
||||
|
||||
# The sourced fundamental read is a **separate channel**, never a term in either
|
||||
# score. It is reported as a categorical state beside State and Warning, and the
|
||||
# three are read together by confluence rather than added up.
|
||||
#
|
||||
# Two things had to be true at once and only this shape gets both.
|
||||
#
|
||||
# **v3's reason for removing it was wrong.** v3 argued that F1+F3, at 12+8 of 100
|
||||
# Warning points, "could not change any published conclusion" because pegged they
|
||||
# produced a Warning of exactly 20.0. That holds only when every technical sensor
|
||||
# reads exactly zero. Weighted, those points added +10 to +20 across the
|
||||
# realistic range and moved the technical score needed to reach the 40 quadrant
|
||||
# divider from 40 to 25. So the observation was not inert, and demoting it to
|
||||
# decoration was not justified by that argument.
|
||||
#
|
||||
# **But no weight is measurable either.** A weighted modifier was built (v5,
|
||||
# reverted) and its size could not be derived from anything: with ~10 correction
|
||||
# events and essentially no fundamental history, any fusion weight is a policy
|
||||
# preference presented as a measurement. Adding a slow categorical judgement to a
|
||||
# fast continuous score also manufactures precision by summing unlike things, and
|
||||
# it forces a missing observation to silently redistribute its weight onto the
|
||||
# technical sensors -- the opposite of leaving it unknown.
|
||||
#
|
||||
# So the read gets a channel, not a coefficient. Revisit only with enough
|
||||
# point-in-time history to test whether the state improves prediction
|
||||
# *conditional on* Warning; a fitted model then has something to fit.
|
||||
FUNDAMENTAL_STATES = ("supportive", "neutral", "adverse", "unknown")
|
||||
EVIDENCE_QUALITY = ("complete", "partial", "stale", "manual", "unavailable")
|
||||
|
||||
# Fixed at the v2 launch. These are liquid S&P 500/Nasdaq AI, semiconductor,
|
||||
# infrastructure, cloud, and enterprise-software names that the platform's
|
||||
# normal universe sync already stores.
|
||||
@@ -196,7 +235,12 @@ DEFAULT_CONFIG: dict = {
|
||||
}
|
||||
|
||||
CAPEX_STATES = ("raising", "holding", "cutting", "unknown")
|
||||
GNSD_STATES = ("yes", "no", "mixed")
|
||||
# "mixed" is a genuinely observed mixed reaction; "unknown" is nobody looked or
|
||||
# the extraction failed. They were the same value until 2026-08-13, so a failed
|
||||
# LLM parse silently became neutral *evidence* -- an observation of normality
|
||||
# manufactured out of a parse error. Same distinction the capex map already made
|
||||
# with its own "unknown", and the same one the whole channel is built on.
|
||||
GNSD_STATES = ("yes", "no", "mixed", "unknown")
|
||||
# v2 scored raising and holding identically at 0, so in a capex boom the reading
|
||||
# was pinned at 0 and could not express the raising -> holding deceleration that
|
||||
# is the actual early warning. Display-only in v3, but it should still describe.
|
||||
@@ -435,6 +479,104 @@ def score_warning_sensors(sensors: dict[str, float | None]) -> float | None:
|
||||
return sum(s * w for s, w in live) / sum(w for _, w in live)
|
||||
|
||||
|
||||
def _capex_signal(capex: dict[str, str] | None, names: list[str]) -> str:
|
||||
"""Categorical read of hyperscaler capex direction. Never an average.
|
||||
|
||||
Averaging is what this must not do: it would let two ``cutting`` reads and
|
||||
two ``unknown`` ones land on "neutral", presenting missing evidence as
|
||||
evidence of normality. Any cut is adverse on partial evidence; only a fully
|
||||
known, uniformly rising basket is supportive.
|
||||
"""
|
||||
states = [str((capex or {}).get(name, "unknown")).strip().lower() for name in names]
|
||||
known = [state for state in states if state in ("raising", "holding", "cutting")]
|
||||
if not known:
|
||||
return "unknown"
|
||||
if "cutting" in known:
|
||||
return "adverse"
|
||||
if "holding" in known:
|
||||
return "neutral"
|
||||
return "supportive"
|
||||
|
||||
|
||||
def _reaction_signal(good_news_stock_down: str | None) -> str:
|
||||
"""Good earnings being sold is a late-cycle tell; not being sold is healthy.
|
||||
|
||||
Anything that is not one of the three observed categories -- including the
|
||||
explicit ``"unknown"`` an extraction failure now writes -- falls through to
|
||||
``unknown`` rather than to ``mixed``. A parse error is not a reading.
|
||||
"""
|
||||
return {
|
||||
"yes": "adverse",
|
||||
"no": "supportive",
|
||||
"mixed": "neutral",
|
||||
}.get(str(good_news_stock_down or "").strip().lower(), "unknown")
|
||||
|
||||
|
||||
def combine_fundamental_signals(capex_signal: str, reaction_signal: str) -> str:
|
||||
"""Confluence, not arithmetic: precedence over the two categorical reads.
|
||||
|
||||
``unknown`` is deliberately unreachable by combination -- it survives only
|
||||
when *nothing* was observed. A single adverse read carries, because partial
|
||||
evidence of deterioration is still evidence of deterioration; supportive
|
||||
requires every observed signal to agree.
|
||||
"""
|
||||
signals = (capex_signal, reaction_signal)
|
||||
if "adverse" in signals:
|
||||
return "adverse"
|
||||
observed = [signal for signal in signals if signal != "unknown"]
|
||||
if not observed:
|
||||
return "unknown"
|
||||
return "supportive" if all(signal == "supportive" for signal in observed) else "neutral"
|
||||
|
||||
|
||||
def _usable_context(observed: bool, pending: bool, stale: bool, state: str) -> bool:
|
||||
"""Whether a fundamental reading may count as evidence.
|
||||
|
||||
One definition, called by both the point-in-time record and the live
|
||||
reading, because they publish the same field name to the same consumers and
|
||||
a second copy would drift. Distinct from `available`, which is about timing
|
||||
alone: an observation whose extraction failed on everything is effective and
|
||||
fresh, and still knows nothing.
|
||||
"""
|
||||
return observed and not pending and not stale and state != "unknown"
|
||||
|
||||
|
||||
def _evidence_quality(
|
||||
capex: dict[str, str] | None,
|
||||
good_news_stock_down: str | None,
|
||||
names: list[str],
|
||||
*,
|
||||
observed: bool,
|
||||
stale: bool,
|
||||
source: str | None,
|
||||
) -> str:
|
||||
"""How much to trust the state above, as one field the reader can act on.
|
||||
|
||||
Ordered by what an operator most needs to know: nothing collected beats
|
||||
everything else, then a reading too old to be current, then a hand override,
|
||||
then completeness.
|
||||
"""
|
||||
if not observed:
|
||||
return "unavailable"
|
||||
if stale:
|
||||
return "stale"
|
||||
if str(source or "").strip().lower() == "manual":
|
||||
return "manual"
|
||||
known = sum(
|
||||
1
|
||||
for name in names
|
||||
if str((capex or {}).get(name, "unknown")).strip().lower() != "unknown"
|
||||
)
|
||||
# `bool(names)` matters: with an empty basket `known == len(names)` is
|
||||
# vacuously true, so nothing observed would grade as complete.
|
||||
complete = (
|
||||
bool(names)
|
||||
and known == len(names)
|
||||
and _reaction_signal(good_news_stock_down) != "unknown"
|
||||
)
|
||||
return "complete" if complete else "partial"
|
||||
|
||||
|
||||
def _sensor(sensor_id: str, label: str, score: float | None, **details: object) -> dict:
|
||||
return {
|
||||
"id": sensor_id,
|
||||
@@ -572,26 +714,66 @@ def _overlay_timing(
|
||||
return effective, pending, age, stale
|
||||
|
||||
|
||||
def fundamental_overlay(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
"""Point-in-time qualitative overlay. Never feeds State or Warning since v3.
|
||||
def fundamental_context(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
"""Point-in-time fundamental channel. Never a term in State or Warning.
|
||||
|
||||
Called an "overlay" until 2026-08-12, which undersold it: it is the third
|
||||
channel of the model, read alongside the two scores by confluence rather than
|
||||
decorating them. The categorical ``state`` is what a reader and the chart
|
||||
consume; ``evidence_quality`` is how far to trust it.
|
||||
|
||||
Both are derived from the stored categorical facts by fixed rules, not from
|
||||
an LLM's numeric judgement. The LLM's job is extraction and explanation --
|
||||
find the capex guidance, classify it, cite it -- and the rules turn those
|
||||
facts into a state, so the same observation always yields the same category.
|
||||
|
||||
The effective-date gate stays even though nothing is scored from this: the
|
||||
400-session rebuild replays historical dates, and stamping today's LLM read
|
||||
onto 2024 snapshots would be plain lookahead in the stored record.
|
||||
rebuild replays historical dates, and stamping today's read onto 2024
|
||||
snapshots would be plain lookahead in the stored record.
|
||||
|
||||
This is the *record*. For "what do we know right now", use
|
||||
This is the *record*. For "what do we know right now", use
|
||||
``current_observation`` -- do not add a bypass flag here, because this runs
|
||||
for every replayed date during a rebuild.
|
||||
"""
|
||||
effective, pending, age, stale = _overlay_timing(overrides, config, as_of)
|
||||
names = list(config["tickers"]["hyperscalers"])
|
||||
capex = None if pending else overrides.get("capex")
|
||||
reaction = None if pending else overrides.get("good_news_stock_down")
|
||||
observed = not pending and bool(overrides.get("fetched_at"))
|
||||
|
||||
capex_signal = _capex_signal(capex, names) if observed else "unknown"
|
||||
reaction_signal = _reaction_signal(reaction) if observed else "unknown"
|
||||
state = combine_fundamental_signals(capex_signal, reaction_signal)
|
||||
return {
|
||||
"state": state,
|
||||
"evidence_quality": _evidence_quality(
|
||||
capex, reaction, names,
|
||||
observed=observed, stale=stale, source=overrides.get("source"),
|
||||
),
|
||||
"capex_signal": capex_signal,
|
||||
"reaction_signal": reaction_signal,
|
||||
# Two different questions, and conflating them is a trap:
|
||||
#
|
||||
# `available` is about *timing* -- there is an effective, non-stale record
|
||||
# to display. `usable` is about *content* -- it also actually says
|
||||
# something. A collected observation whose extraction failed on every
|
||||
# hyperscaler is available (show it, with its date) but not usable: it
|
||||
# knows nothing, so it must never count as evidence.
|
||||
#
|
||||
# The distinction is load-bearing for the event study. Coverage is
|
||||
# measured in sessions with usable context, and if repeated extraction
|
||||
# failures counted, they would slowly accumulate "exposure" until the
|
||||
# fundamental rows flipped to measurable 0/8 -- a failed result reported
|
||||
# for a channel that never knew anything, which is the exact confusion
|
||||
# coverage-matching exists to prevent.
|
||||
"available": not pending and not stale,
|
||||
"usable": _usable_context(observed, pending, stale, state),
|
||||
"pending": pending,
|
||||
"stale": stale,
|
||||
"effective_date": effective.isoformat() if effective else None,
|
||||
"age_days": age,
|
||||
"capex": None if pending else overrides.get("capex"),
|
||||
"good_news_stock_down": None if pending else overrides.get("good_news_stock_down"),
|
||||
"capex": capex,
|
||||
"good_news_stock_down": reaction,
|
||||
"capex_stress": None if pending else overrides.get("f1_score"),
|
||||
"earnings_stress": None if pending else overrides.get("f3_score"),
|
||||
"reasoning": None if pending else overrides.get("reasoning"),
|
||||
@@ -603,7 +785,7 @@ def fundamental_overlay(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
def current_observation(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
"""The observation as it stands now, for the live reading only.
|
||||
|
||||
Same shape as ``fundamental_overlay``, but the effective date is *reported*
|
||||
Same shape as ``fundamental_context``, but the effective date is *reported*
|
||||
rather than used to blank the content. A refresh stamps
|
||||
``_next_weekday(today)``, so gating the live card hid a just-collected read
|
||||
for one day -- three over a weekend -- and refreshing appeared to do
|
||||
@@ -611,14 +793,36 @@ def current_observation(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
published number; the stored snapshot keeps the gate.
|
||||
"""
|
||||
effective, pending, age, stale = _overlay_timing(overrides, config, as_of)
|
||||
# The default override carries "unknown"/"mixed" placeholders for every
|
||||
# The default override carries "unknown" placeholders for every
|
||||
# hyperscaler. Those are the absence of an observation, not an observation
|
||||
# of absence, and must never be presented as collected. ``fetched_at`` is
|
||||
# the collection timestamp and is the only field written on every path that
|
||||
# produces real content (LLM refresh and manual save both stamp it).
|
||||
observed = bool(overrides.get("fetched_at"))
|
||||
names = list(config["tickers"]["hyperscalers"])
|
||||
capex_signal = _capex_signal(overrides.get("capex"), names) if observed else "unknown"
|
||||
reaction_signal = (
|
||||
_reaction_signal(overrides.get("good_news_stock_down")) if observed else "unknown"
|
||||
)
|
||||
state = combine_fundamental_signals(capex_signal, reaction_signal)
|
||||
return {
|
||||
"observed": observed,
|
||||
"state": state,
|
||||
"evidence_quality": _evidence_quality(
|
||||
overrides.get("capex"), overrides.get("good_news_stock_down"), names,
|
||||
observed=observed, stale=stale, source=overrides.get("source"),
|
||||
),
|
||||
"capex_signal": capex_signal,
|
||||
"reaction_signal": reaction_signal,
|
||||
# Same shape as the record means the same *fields*, not just the same
|
||||
# ones this function happens to need: the frontend types both payloads
|
||||
# identically, so an omission here is an undefined at runtime that
|
||||
# TypeScript cannot catch across a trusted server boundary.
|
||||
#
|
||||
# Note this is stricter than the `available` directly below: a pending
|
||||
# observation is the freshest thing we have and worth showing, but it is
|
||||
# not yet in force, so it is not yet evidence.
|
||||
"usable": _usable_context(observed, pending, stale, state),
|
||||
# Live availability is about usefulness, not effectiveness: a pending
|
||||
# observation is the freshest thing we have -- but nothing collected is
|
||||
# never available.
|
||||
@@ -656,8 +860,16 @@ def _compute_index(
|
||||
breadth_series: Series | None = None,
|
||||
divergence_series: Series | None = None,
|
||||
breadth_counts: dict[date, int] | None = None,
|
||||
observations: list[dict] | None = None,
|
||||
) -> dict:
|
||||
"""Compute the complete State/Warning snapshot as of one trading date."""
|
||||
"""Compute the complete State/Warning snapshot as of one trading date.
|
||||
|
||||
``observations`` is the point-in-time fundamental series and is authoritative
|
||||
when supplied; ``overrides`` is the single-slot fallback for callers that
|
||||
predate the table (the calibration harness). Either way the reading is scored
|
||||
into the same ``fundamental_context`` -- only where it is read from differs,
|
||||
so the live monitor and the event study cannot report different states.
|
||||
"""
|
||||
tickers = config["tickers"]
|
||||
smh = _closes_asof(prices.get(tickers["leaders"][0], []), as_of)
|
||||
qqq = _closes_asof(prices.get(tickers["confirm"][0], []), as_of)
|
||||
@@ -682,7 +894,10 @@ def _compute_index(
|
||||
sensors = warning_sensor_scores(divergence, smh, spy, oas_window)
|
||||
relative_strength = sensors["relative_strength"]
|
||||
credit_impulse = sensors["credit_impulse"]
|
||||
overlay = fundamental_overlay(overrides, config, as_of)
|
||||
observation = (
|
||||
observation_asof(observations, as_of) if observations is not None else overrides
|
||||
) or {}
|
||||
context = fundamental_context(observation, config, as_of)
|
||||
|
||||
state_pillars = [
|
||||
{
|
||||
@@ -768,7 +983,7 @@ def _compute_index(
|
||||
"date": as_of.isoformat(),
|
||||
"state": state,
|
||||
"warning": warning,
|
||||
"fundamental_overlay": overlay,
|
||||
"fundamental_context": context,
|
||||
"quadrant_config": {
|
||||
"state_divider": QUADRANT_STATE_DIVIDER,
|
||||
"warning_divider": QUADRANT_WARNING_DIVIDER,
|
||||
@@ -790,8 +1005,8 @@ def _compute_index(
|
||||
"breadth_pct_above_200": round(breadth_pct, 1) if breadth_pct is not None else None,
|
||||
"breadth_date": breadth_item[0].isoformat() if breadth_item else None,
|
||||
"fundamentals_fetched_at": overrides.get("fetched_at"),
|
||||
"fundamentals_effective_date": overlay.get("effective_date"),
|
||||
"fundamentals_age_days": overlay.get("age_days"),
|
||||
"fundamentals_effective_date": context.get("effective_date"),
|
||||
"fundamentals_age_days": context.get("age_days"),
|
||||
},
|
||||
"data_quality": {
|
||||
"minimum_coverage": MIN_COVERAGE,
|
||||
@@ -859,7 +1074,7 @@ async def get_fundamental_overrides(db: AsyncSession) -> dict:
|
||||
"f1_score": None,
|
||||
"f3_score": None,
|
||||
"capex": {name: "unknown" for name in names},
|
||||
"good_news_stock_down": "mixed",
|
||||
"good_news_stock_down": "unknown",
|
||||
"locked": False,
|
||||
"reasoning": None,
|
||||
"fetched_at": None,
|
||||
@@ -880,9 +1095,9 @@ async def get_fundamental_overrides(db: AsyncSession) -> dict:
|
||||
if stored.get("methodology") not in CATEGORICAL_FUNDAMENTAL_METHODOLOGIES:
|
||||
return default
|
||||
capex = _normalise_capex_states(stored.get("capex"), names)
|
||||
reaction = str(stored.get("good_news_stock_down", "mixed")).strip().lower()
|
||||
reaction = str(stored.get("good_news_stock_down", "unknown")).strip().lower()
|
||||
if reaction not in GNSD_STATES:
|
||||
reaction = "mixed"
|
||||
reaction = "unknown"
|
||||
return {
|
||||
**default,
|
||||
**stored,
|
||||
@@ -922,6 +1137,100 @@ def _score_capex_states(capex: dict[str, str], names: list[str]) -> float | None
|
||||
return round(score, 1) if score is not None else None
|
||||
|
||||
|
||||
async def record_fundamental_observation(db: AsyncSession, observation: dict) -> None:
|
||||
"""Append the observation to the point-in-time series, keyed on effective date.
|
||||
|
||||
Upsert rather than insert: re-saving on the same effective date is a
|
||||
correction to that day's reading, not a second observation of it.
|
||||
|
||||
Silently does nothing without an effective date or a ``fetched_at``. Those
|
||||
are the default placeholder blob -- the absence of an observation, which must
|
||||
never enter the series as though someone had looked.
|
||||
|
||||
Deliberately does **not** commit. ``update_regime_monitor`` calls this inside
|
||||
a run that owns its transaction and commits once after the snapshot loop;
|
||||
committing here would take that boundary away from it. The two override
|
||||
writers commit for themselves.
|
||||
"""
|
||||
effective = _parse_date(observation.get("effective_date"))
|
||||
fetched_raw = observation.get("fetched_at")
|
||||
if effective is None or not fetched_raw:
|
||||
return
|
||||
try:
|
||||
fetched = datetime.fromisoformat(str(fetched_raw))
|
||||
except ValueError:
|
||||
fetched = datetime.now(timezone.utc)
|
||||
if fetched.tzinfo is None:
|
||||
fetched = fetched.replace(tzinfo=timezone.utc)
|
||||
|
||||
existing = await db.execute(
|
||||
select(RegimeFundamentalObservation).where(
|
||||
RegimeFundamentalObservation.effective_date == effective
|
||||
)
|
||||
)
|
||||
row = existing.scalar_one_or_none()
|
||||
payload = {
|
||||
"f1_score": observation.get("f1_score"),
|
||||
"f3_score": observation.get("f3_score"),
|
||||
"capex_json": json.dumps(observation.get("capex") or {}),
|
||||
"good_news_stock_down": str(observation.get("good_news_stock_down") or "unknown")[:10],
|
||||
"reasoning": observation.get("reasoning"),
|
||||
"source": str(observation.get("source") or "unknown")[:30],
|
||||
"fetched_at": fetched,
|
||||
}
|
||||
if row is None:
|
||||
db.add(RegimeFundamentalObservation(
|
||||
effective_date=effective,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
**payload,
|
||||
))
|
||||
else:
|
||||
for key, value in payload.items():
|
||||
setattr(row, key, value)
|
||||
|
||||
|
||||
async def get_fundamental_observations(db: AsyncSession) -> list[dict]:
|
||||
"""The whole observation series, oldest first, for point-in-time scoring."""
|
||||
result = await db.execute(
|
||||
select(RegimeFundamentalObservation).order_by(
|
||||
RegimeFundamentalObservation.effective_date.asc()
|
||||
)
|
||||
)
|
||||
out: list[dict] = []
|
||||
for row in result.scalars().all():
|
||||
try:
|
||||
capex = json.loads(row.capex_json)
|
||||
except (TypeError, ValueError):
|
||||
capex = {}
|
||||
out.append({
|
||||
"effective_date": row.effective_date,
|
||||
"f1_score": row.f1_score,
|
||||
"f3_score": row.f3_score,
|
||||
"capex": capex,
|
||||
"good_news_stock_down": row.good_news_stock_down,
|
||||
"reasoning": row.reasoning,
|
||||
"source": row.source,
|
||||
"fetched_at": row.fetched_at.isoformat() if row.fetched_at else None,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def observation_asof(observations: list[dict] | None, as_of: date) -> dict | None:
|
||||
"""Latest observation effective on or before ``as_of``.
|
||||
|
||||
This *is* the effective-date gate now. The settings-blob version had to
|
||||
recompute it per call because there was only ever one observation to gate;
|
||||
with a series, "which reading was live that day" is just a lookup.
|
||||
"""
|
||||
chosen: dict | None = None
|
||||
for observation in observations or []:
|
||||
if observation["effective_date"] <= as_of:
|
||||
chosen = observation
|
||||
else:
|
||||
break
|
||||
return chosen
|
||||
|
||||
|
||||
async def set_fundamental_overrides(
|
||||
db: AsyncSession,
|
||||
capex: dict[str, str] | None = None,
|
||||
@@ -954,7 +1263,15 @@ async def set_fundamental_overrides(
|
||||
"fetched_at": now.isoformat(),
|
||||
"effective_date": _next_weekday(now.date()).isoformat(),
|
||||
})
|
||||
await update_setting(db, KEY_FUNDAMENTALS, json.dumps(current))
|
||||
# The blob (what the live card reads) and the series row (what the
|
||||
# point-in-time replay reads) are the same observation. Committed together:
|
||||
# `update_setting` commits internally, so using it here would leave a window
|
||||
# where a failure publishes the reading to the card but not to the record,
|
||||
# and the two would disagree permanently with nothing to detect it.
|
||||
await settings_store.upsert_setting(db, KEY_FUNDAMENTALS, json.dumps(current))
|
||||
if observation_changed:
|
||||
await record_fundamental_observation(db, current)
|
||||
await db.commit()
|
||||
return current
|
||||
|
||||
|
||||
@@ -1058,12 +1375,59 @@ def _snapshot_revision(snapshot: dict) -> int:
|
||||
return 1
|
||||
|
||||
|
||||
def _context_from_legacy_overlay(overlay: dict) -> dict:
|
||||
"""Rebuild the categorical channel from a pre-rename snapshot's overlay.
|
||||
|
||||
The channel was called ``fundamental_overlay`` until 2026-08-12 and stored
|
||||
the same underlying facts -- the capex map, the earnings reaction, the
|
||||
effective date. The rename shipped without a methodology bump (no score
|
||||
changed), so those rows are still served and were never reseeded: reading
|
||||
only the new key would turn every one of them into ``unknown`` and silently
|
||||
discard real recorded evidence -- historical Path colours, and any exposure
|
||||
the event study could legitimately count.
|
||||
|
||||
Derived, not guessed. The hyperscaler list comes from the overlay's own
|
||||
capex keys, which is exactly the basket that was observed at the time rather
|
||||
than today's configured one.
|
||||
"""
|
||||
capex = overlay.get("capex") or {}
|
||||
reaction = overlay.get("good_news_stock_down")
|
||||
names = list(capex)
|
||||
pending = bool(overlay.get("pending"))
|
||||
stale = bool(overlay.get("stale"))
|
||||
observed = not pending and bool(overlay.get("fetched_at"))
|
||||
|
||||
capex_signal = _capex_signal(capex, names) if observed else "unknown"
|
||||
reaction_signal = _reaction_signal(reaction) if observed else "unknown"
|
||||
state = combine_fundamental_signals(capex_signal, reaction_signal)
|
||||
return {
|
||||
**overlay,
|
||||
"state": state,
|
||||
"evidence_quality": _evidence_quality(
|
||||
capex, reaction, names,
|
||||
observed=observed, stale=stale, source=overlay.get("source"),
|
||||
),
|
||||
"capex_signal": capex_signal,
|
||||
"reaction_signal": reaction_signal,
|
||||
"usable": _usable_context(observed, pending, stale, state),
|
||||
}
|
||||
|
||||
|
||||
def _parse_snapshot(raw: str) -> dict | None:
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return parsed if parsed.get("methodology") == METHODOLOGY else None
|
||||
if parsed.get("methodology") != METHODOLOGY:
|
||||
return None
|
||||
# Normalise here rather than at each call site: every reader of a stored
|
||||
# snapshot goes through this function, so a legacy row cannot reach one of
|
||||
# them un-adapted.
|
||||
if "fundamental_context" not in parsed and "fundamental_overlay" in parsed:
|
||||
parsed["fundamental_context"] = _context_from_legacy_overlay(
|
||||
parsed["fundamental_overlay"] or {}
|
||||
)
|
||||
return parsed
|
||||
|
||||
|
||||
async def _latest_snapshot_row(db: AsyncSession) -> tuple[RegimeSnapshot, dict] | None:
|
||||
@@ -1082,6 +1446,10 @@ async def update_regime_monitor(
|
||||
) -> dict:
|
||||
config = await get_regime_config(db)
|
||||
overrides = await get_fundamental_overrides(db)
|
||||
# Carries the pre-v5 single-slot observation into the series on first run, so
|
||||
# a deployment does not lose the live reading. A no-op once recorded, and a
|
||||
# no-op for the placeholder blob (no fetched_at).
|
||||
await record_fundamental_observation(db, overrides)
|
||||
if _fundamentals_stale(overrides, config) and not overrides.get("locked"):
|
||||
try:
|
||||
overrides = await refresh_fundamental_overrides(db, config=config)
|
||||
@@ -1131,6 +1499,9 @@ async def update_regime_monitor(
|
||||
|
||||
breadth_series = _mapping_series(breadth)
|
||||
divergence_series = _mapping_series(divergence)
|
||||
# Loaded once, after any refresh, so a reseed scores each replayed date with
|
||||
# the observation that was effective on it rather than with today's.
|
||||
observations = await get_fundamental_observations(db)
|
||||
latest_result: dict | None = None
|
||||
snapshots_written = 0
|
||||
for snapshot_date in dates:
|
||||
@@ -1144,6 +1515,7 @@ async def update_regime_monitor(
|
||||
breadth_series,
|
||||
divergence_series,
|
||||
breadth_counts,
|
||||
observations=observations,
|
||||
)
|
||||
written, latest_result = await _upsert_snapshot(
|
||||
db,
|
||||
@@ -1221,16 +1593,22 @@ async def get_regime_monitor(db: AsyncSession) -> dict:
|
||||
quality["is_fresh"] = bool(quality.get("inputs_fresh")) and snapshot_age <= 4
|
||||
result["data_quality"] = quality
|
||||
|
||||
# The snapshot's overlay is the point-in-time record; the reader also wants
|
||||
# the current observation even when it is not effective until the next
|
||||
# session, because otherwise refreshing it looks like it did nothing.
|
||||
# The snapshot's `fundamental_context` is the point-in-time record; the
|
||||
# reader also wants the current observation even when it is not effective
|
||||
# until the next session, or refreshing it looks like it did nothing.
|
||||
config = await get_regime_config(db)
|
||||
overrides = await get_fundamental_overrides(db)
|
||||
live = current_observation(overrides, config, date.today())
|
||||
# Deliberately reads the *snapshot's* overlay, not the live one: this is how
|
||||
# Deliberately reads the *snapshot's* record, not the live one: this is how
|
||||
# the reader tells "shown here" from "in the stored record".
|
||||
live["observed_in_snapshot"] = bool((result.get("fundamental_overlay") or {}).get("available"))
|
||||
result["fundamental_context"] = live
|
||||
live["observed_in_snapshot"] = bool(
|
||||
(result.get("fundamental_context") or {}).get("available")
|
||||
)
|
||||
# `fundamental_context` is the stored channel and stays the snapshot's;
|
||||
# `fundamental_live` is what we know right now. Collapsing the two under one
|
||||
# key is what made a just-collected observation look like it had been
|
||||
# backdated into history.
|
||||
result["fundamental_live"] = live
|
||||
result["available"] = True
|
||||
return result
|
||||
|
||||
@@ -1248,10 +1626,17 @@ async def get_regime_history(db: AsyncSession, days: int = 800) -> list[dict]:
|
||||
if data is None:
|
||||
continue
|
||||
state, warning = data.get("state") or {}, data.get("warning") or {}
|
||||
context = data.get("fundamental_context") or {}
|
||||
out.append({
|
||||
"date": row.date.isoformat(),
|
||||
"state": state.get("score") if state.get("band") is not None else None,
|
||||
"warning": warning.get("score") if warning.get("band") is not None else None,
|
||||
# The third channel, carried per point so the Path view can colour a
|
||||
# dot by the fundamental context that was on the record that day.
|
||||
# Rows written before the channel existed carry nothing, which reads
|
||||
# as "unknown" -- correct, since nothing was observed then either.
|
||||
"fundamental_state": context.get("state") or "unknown",
|
||||
"evidence_quality": context.get("evidence_quality") or "unavailable",
|
||||
"state_coverage": state.get("coverage"),
|
||||
"warning_coverage": warning.get("coverage"),
|
||||
"basket_hash": (data.get("basket") or {}).get("hash"),
|
||||
@@ -1383,7 +1768,7 @@ async def refresh_fundamental_overrides(
|
||||
f1 = _score_capex_states(capex, names)
|
||||
reaction = str(parsed.get("good_news_stock_down", "")).strip().lower()
|
||||
if reaction not in GNSD_STATES:
|
||||
reaction = "mixed"
|
||||
reaction = "unknown"
|
||||
f3 = _GNSD_SCORES.get(reaction)
|
||||
now = datetime.now(timezone.utc)
|
||||
result = {
|
||||
@@ -1398,7 +1783,11 @@ async def refresh_fundamental_overrides(
|
||||
"locked": False,
|
||||
"source": llm.get("provider"),
|
||||
}
|
||||
await update_setting(db, KEY_FUNDAMENTALS, json.dumps(result))
|
||||
# One transaction: see set_fundamental_overrides on why these two writes must
|
||||
# not be able to land separately.
|
||||
await settings_store.upsert_setting(db, KEY_FUNDAMENTALS, json.dumps(result))
|
||||
await record_fundamental_observation(db, result)
|
||||
await db.commit()
|
||||
logger.info(json.dumps({
|
||||
"event": "regime_fundamentals_refreshed",
|
||||
"f1": result["f1_score"],
|
||||
|
||||
Reference in New Issue
Block a user