Rewrite Regime Monitor as v3: fundamentals off the score, desaturate P3
The LLM-sourced capex/earnings observations carried 12+8 of 100 Warning points, so both pegged at 100 produced a Warning of 20.0 -- below the event study's 25.3 alarm threshold and still inside the "stable" band. The reading was arithmetically incapable of changing anything on screen, which is why refreshing it appeared to do nothing. They are now a qualitative overlay reported beside the scores rather than diluted into them. Calibrated against the 408 v2 sessions to 2026-07-24, reproduced offline from Alpaca + FRED; the harness matched the stored prod distribution exactly before any parameter was changed. State: - P3 used dd_pct * 5, reaching 100 at a 20% drawdown -- the 90th percentile of the observed distribution -- so 39/408 sessions sat at exactly 100 with no resolution left during the part of a selloff that matters most. Replaced with anchored breakpoints keeping headroom past the observed 36% maximum, blended 2:1 like P1/P2 instead of max(). P3's realized share of State falls from 65% to 40%, matching its nominal weight. - Credit level is now anchors-only. ICE capped FRED's BAMLH0A0HYM2 at a rolling 3-year window in April 2026, silently turning the 10-year percentile leg into a 3-year one that scored 20 points of stress at an OAS of 3.5 -- the level its own anchors call "mild". The anchors already encode the long-run distribution. Warning: - Added HY OAS 20-session widening (25%). The level is pinned at zero below the 3.5 anchor; its rate of change is not. - Divergence tapers to a 0.35 floor instead of a hard price_ret >= 0 gate, which zeroed the sensor through every decline: on 2026-07-24 the basket shed 10 points of participation in 20 sessions and Warning printed exactly 0. - The event study and the live monitor now share one sensor definition, so they cannot silently drift apart. Bands are per axis (State 20/50/80, Warning 20/40/60) with quadrant dividers at 50/40; v2 Warning never exceeded 64.9 against a shared 60, leaving that half of the quadrant unreachable. Realized shares: State 73/15/8/3%, Warning 69/20/8/3%. Snapshots now record credit_history_days and vix_history_days -- the percentile defect went unnoticed for months because nothing asserted the window the code claimed. Cutover: the first run rebuilds 400 sessions automatically; the Event Study job must be re-run, as its cached report self-invalidates on the methodology check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,16 +1,23 @@
|
||||
"""AI/Tech Regime Monitor v2.
|
||||
"""AI/Tech Regime Monitor v3.
|
||||
|
||||
The monitor is a risk thermometer, not a probability or trading rule. It keeps
|
||||
two deliberately separate outputs:
|
||||
|
||||
* State: current structural stress (price, breadth, credit, volatility).
|
||||
* Warning: deterioration/divergence that may precede State (breadth, relative
|
||||
strength, and sourced fundamental observations).
|
||||
* Warning: deterioration/divergence that may precede State (breadth divergence,
|
||||
relative strength, credit impulse).
|
||||
|
||||
Daily snapshots are the point-in-time record. The first v2 run rewrites the
|
||||
latest ``REBUILD_SESSIONS`` trading sessions once; ordinary runs thereafter only
|
||||
upsert the latest trading date. Fundamental observations are never replayed
|
||||
before their effective date.
|
||||
Both scores are quantitative and daily. The sourced hyperscaler capex and
|
||||
earnings-reaction observations are a qualitative *overlay* in v3 rather than
|
||||
weighted sensors: at a combined 20 points they could not reach the event
|
||||
study's alarm threshold even when both pegged, so refreshing them appeared to
|
||||
do nothing. They are reported next to the scores instead of inside them.
|
||||
|
||||
Daily snapshots are the point-in-time record. The first run under a new
|
||||
``METHODOLOGY`` rewrites the latest ``REBUILD_SESSIONS`` trading sessions once;
|
||||
ordinary runs thereafter only upsert the latest trading date. The overlay is
|
||||
still gated by its effective date so a rebuild cannot stamp today's observation
|
||||
onto historical snapshots.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -41,19 +48,51 @@ _CA_BUNDLE = os.environ.get("SSL_CERT_FILE", "")
|
||||
KEY_CONFIG = "regime_monitor_config"
|
||||
KEY_FUNDAMENTALS = "regime_fundamental_overrides"
|
||||
|
||||
METHODOLOGY = "v2"
|
||||
METHODOLOGY = "v3"
|
||||
# Snapshots are reseeded on a methodology bump, but fundamental observations are
|
||||
# collected by hand/LLM and carried across it when the format is compatible.
|
||||
CATEGORICAL_FUNDAMENTAL_METHODOLOGIES = frozenset({"v2", "v3"})
|
||||
REBUILD_SESSIONS = 400
|
||||
MIN_COVERAGE = 75.0
|
||||
SOURCE_MAX_LAG_DAYS = 7
|
||||
|
||||
QUADRANT_STATE_DIVIDER = 60.0
|
||||
QUADRANT_WARNING_DIVIDER = 60.0
|
||||
# Bands are per axis: the two scores have genuinely different realized ranges,
|
||||
# so one shared set made Warning's top bands unreachable (v2 Warning never
|
||||
# exceeded 64.9 in 408 sessions while State reached 91.2). Thresholds are round
|
||||
# numbers chosen so each band covers a sane share of history, not percentile
|
||||
# fits -- percentile-derived bands would drift on every rebuild and silently
|
||||
# rewrite what past snapshots meant. Realized shares over the 408 sessions to
|
||||
# 2026-07-24: State 73/15/8/3%, Warning 69/20/8/3%.
|
||||
STATE_BANDS = (20.0, 50.0, 80.0)
|
||||
WARNING_BANDS = (20.0, 40.0, 60.0)
|
||||
|
||||
QUADRANT_STATE_DIVIDER = 50.0
|
||||
QUADRANT_WARNING_DIVIDER = 40.0
|
||||
QUADRANT_MARGIN = 5.0
|
||||
|
||||
HY_OAS_MILD = 3.5
|
||||
HY_OAS_ELEVATED = 5.0
|
||||
HY_OAS_STRESSED = 7.0
|
||||
HY_OAS_REFERENCE_YEARS = 10.0
|
||||
# ICE restricted FRED to a rolling 3-year window for BAMLH0A0HYM2 in April 2026
|
||||
# ("Starting in April 2026, this series will only include 3 years of
|
||||
# observations"), so v2's 10-year reference window silently became 3. Against 3
|
||||
# years of uniformly tight spreads (2.59-4.61 over the calibration window) the
|
||||
# blended upper-tail percentile saturated at an OAS of ~4.5 and scored 20 points
|
||||
# of stress at 3.5 -- the level these anchors call "mild". The anchors already
|
||||
# encode the long-run distribution, so the credit *level* is now purely anchored
|
||||
# and credit *dynamics* live in W3 on the Warning axis where they belong.
|
||||
HY_OAS_WINDOW_DAYS = 400 # only W3's lookback plus slack is needed now
|
||||
W3_OAS_LOOKBACK = 20
|
||||
W3_OAS_FULL_SCALE_PCT = 35.0
|
||||
|
||||
# Drawdown anchors (drawdown %, stress score). v2 used a bare ``dd_pct * 5``,
|
||||
# which pegged at a 20% drawdown -- the 90th percentile of the observed
|
||||
# distribution -- so 39 of 408 sessions sat at exactly 100 with no resolution
|
||||
# left during the part of a selloff that matters most. These anchors keep
|
||||
# headroom past the observed 36% maximum.
|
||||
P3_DRAWDOWN_ANCHORS = (
|
||||
(0.0, 0.0), (4.0, 10.0), (8.0, 25.0), (16.0, 50.0), (28.0, 78.0), (40.0, 100.0),
|
||||
)
|
||||
|
||||
STATE_WEIGHTS = {
|
||||
"price": 40.0,
|
||||
@@ -61,11 +100,15 @@ STATE_WEIGHTS = {
|
||||
"credit": 20.0,
|
||||
"volatility": 15.0,
|
||||
}
|
||||
# Fundamentals left the score in v3. At 12 + 8 points they could not reach the
|
||||
# event study's alarm threshold even when both pegged at 100, so the LLM read was
|
||||
# decorative; it is now a separate qualitative overlay. Credit *impulse* takes
|
||||
# their place because the OAS level is pinned at zero below the 3.5 anchor while
|
||||
# its rate of change is not.
|
||||
WARNING_WEIGHTS = {
|
||||
"breadth_divergence": 50.0,
|
||||
"breadth_divergence": 45.0,
|
||||
"relative_strength": 30.0,
|
||||
"capex": 12.0,
|
||||
"earnings_reaction": 8.0,
|
||||
"credit_impulse": 25.0,
|
||||
}
|
||||
|
||||
# Fixed at the v2 launch. These are liquid S&P 500/Nasdaq AI, semiconductor,
|
||||
@@ -92,7 +135,10 @@ DEFAULT_CONFIG: dict = {
|
||||
|
||||
CAPEX_STATES = ("raising", "holding", "cutting", "unknown")
|
||||
GNSD_STATES = ("yes", "no", "mixed")
|
||||
_CAPEX_STATE_SCORES = {"raising": 0.0, "holding": 0.0, "cutting": 100.0}
|
||||
# v2 scored raising and holding identically at 0, so in a capex boom the reading
|
||||
# was pinned at 0 and could not express the raising -> holding deceleration that
|
||||
# is the actual early warning. Display-only in v3, but it should still describe.
|
||||
_CAPEX_STATE_SCORES = {"raising": 0.0, "holding": 50.0, "cutting": 100.0}
|
||||
_GNSD_SCORES = {"yes": 100.0, "no": 0.0}
|
||||
|
||||
Series = list[tuple[date, float]]
|
||||
@@ -127,12 +173,23 @@ def _blend(leader: float | None, confirm: float | None, leader_weight: float = 2
|
||||
return sum(v * w for v, w in parts) / sum(w for _, w in parts)
|
||||
|
||||
|
||||
def band_for(score: float) -> str:
|
||||
if score < 30:
|
||||
def _interpolate(x: float, anchors: tuple[tuple[float, float], ...]) -> float:
|
||||
"""Piecewise-linear lookup, flat outside the first and last anchor."""
|
||||
if x <= anchors[0][0]:
|
||||
return anchors[0][1]
|
||||
for (x0, y0), (x1, y1) in zip(anchors, anchors[1:]):
|
||||
if x <= x1:
|
||||
return y0 + (y1 - y0) * (x - x0) / (x1 - x0)
|
||||
return anchors[-1][1]
|
||||
|
||||
|
||||
def band_for(score: float, bands: tuple[float, float, float] = STATE_BANDS) -> str:
|
||||
watch, elevated, breaking = bands
|
||||
if score < watch:
|
||||
return "stable"
|
||||
if score < 60:
|
||||
if score < elevated:
|
||||
return "watch"
|
||||
if score < 80:
|
||||
if score < breaking:
|
||||
return "elevated"
|
||||
return "breaking"
|
||||
|
||||
@@ -167,19 +224,29 @@ def p2_death_cross(smh: list[float], qqq: list[float], leader_weight: float = 2.
|
||||
return _blend(_death_cross(smh), _death_cross(qqq), leader_weight)
|
||||
|
||||
|
||||
def _drawdown(closes: list[float]) -> float | None:
|
||||
def drawdown_pct(closes: list[float]) -> float | None:
|
||||
"""Percentage below the trailing 52-week closing high."""
|
||||
if len(closes) < 30:
|
||||
return None
|
||||
peak = max(closes[-252:])
|
||||
if peak <= 0:
|
||||
return None
|
||||
dd_pct = (peak - closes[-1]) / peak * 100.0
|
||||
return _clamp(dd_pct * 5.0)
|
||||
return (peak - closes[-1]) / peak * 100.0
|
||||
|
||||
|
||||
def p3_drawdown(smh: list[float], qqq: list[float]) -> float | None:
|
||||
vals = [v for v in (_drawdown(smh), _drawdown(qqq)) if v is not None]
|
||||
return max(vals) if vals else None
|
||||
def _drawdown(closes: list[float]) -> float | None:
|
||||
dd_pct = drawdown_pct(closes)
|
||||
return None if dd_pct is None else _clamp(_interpolate(dd_pct, P3_DRAWDOWN_ANCHORS))
|
||||
|
||||
|
||||
def p3_drawdown(smh: list[float], qqq: list[float], leader_weight: float = 2.0) -> float | None:
|
||||
"""Anchored drawdown stress on the same 2:1 leader/confirm blend P1 and P2 use.
|
||||
|
||||
v2 took ``max()`` here, which meant the more volatile leader always won and
|
||||
the price pillar reduced to this one sensor: its realized share of State was
|
||||
65% against a nominal 40% weight. Blending brings that back to 40%.
|
||||
"""
|
||||
return _blend(_drawdown(smh), _drawdown(qqq), leader_weight)
|
||||
|
||||
|
||||
def p4_relative_strength(smh: list[float], spy: list[float], lookback: int = 60) -> float | None:
|
||||
@@ -220,18 +287,68 @@ def _oas_absolute_score(value: float) -> float:
|
||||
|
||||
|
||||
def f2_credit_spreads(oas_values: list[float]) -> float | None:
|
||||
"""HY OAS stress: 70% named absolute anchors + 30% upper-tail percentile."""
|
||||
"""HY OAS level against named absolute anchors (3.5 mild / 5.0 / 7.0).
|
||||
|
||||
v2 blended 70% of this with a 30% upper-tail percentile over the available
|
||||
history. That leg was always a second, noisier estimate of what the anchors
|
||||
already encode -- and once the usable window shrank to 3 uniformly tight
|
||||
years it saturated far below any real stress level. Removed rather than
|
||||
repaired: see ``HY_OAS_WINDOW_DAYS``.
|
||||
"""
|
||||
if not oas_values:
|
||||
return None
|
||||
latest = oas_values[-1]
|
||||
absolute = _oas_absolute_score(latest)
|
||||
if len(oas_values) < 30:
|
||||
return round(absolute, 2)
|
||||
less = sum(1 for v in oas_values if v < latest)
|
||||
equal = sum(1 for v in oas_values if v == latest)
|
||||
percentile = (less + 0.5 * equal) / len(oas_values) * 100.0
|
||||
relative = _clamp((percentile - 50.0) / 45.0 * 100.0)
|
||||
return round(absolute * 0.7 + relative * 0.3, 2)
|
||||
return round(_oas_absolute_score(oas_values[-1]), 2)
|
||||
|
||||
|
||||
def w3_credit_impulse(
|
||||
oas_values: list[float], lookback: int = W3_OAS_LOOKBACK
|
||||
) -> float | None:
|
||||
"""HY OAS rate of change: widening only, relative so it works at any level.
|
||||
|
||||
The credit *level* (C1) sits at zero for as long as spreads stay under the
|
||||
3.5 mild anchor -- 2.77 as of the v3 cutover -- so it contributes nothing to
|
||||
State in a calm tape. The rate of change still does, and spread widening is
|
||||
a classic lead, which is what Warning is for. Relative rather than absolute
|
||||
because +0.5pp means something very different at 2.7 than at 8.0.
|
||||
"""
|
||||
if len(oas_values) < lookback + 1:
|
||||
return None
|
||||
past = oas_values[-lookback - 1]
|
||||
if past <= 0:
|
||||
return None
|
||||
change_pct = (oas_values[-1] / past - 1.0) * 100.0
|
||||
return _clamp(change_pct / W3_OAS_FULL_SCALE_PCT * 100.0)
|
||||
|
||||
|
||||
def warning_sensor_scores(
|
||||
divergence: float | None,
|
||||
leader_closes: list[float],
|
||||
market_closes: list[float],
|
||||
oas_window: list[float],
|
||||
) -> dict[str, float | None]:
|
||||
"""The three Warning sensors, by pillar id.
|
||||
|
||||
Single definition so the live monitor and the event study cannot drift apart
|
||||
-- in v2 the study re-derived the score from ``WARNING_WEIGHTS`` by hand and
|
||||
would have silently kept measuring the old construct through this change.
|
||||
"""
|
||||
return {
|
||||
"breadth_divergence": divergence,
|
||||
"relative_strength": p4_relative_strength(leader_closes, market_closes),
|
||||
"credit_impulse": w3_credit_impulse(oas_window),
|
||||
}
|
||||
|
||||
|
||||
def score_warning_sensors(sensors: dict[str, float | None]) -> float | None:
|
||||
"""Weighted Warning score, renormalised over the sensors that are available."""
|
||||
live = [
|
||||
(float(score), float(WARNING_WEIGHTS[key]))
|
||||
for key, score in sensors.items()
|
||||
if score is not None and key in WARNING_WEIGHTS
|
||||
]
|
||||
if not live:
|
||||
return None
|
||||
return sum(s * w for s, w in live) / sum(w for _, w in live)
|
||||
|
||||
|
||||
def _sensor(sensor_id: str, label: str, score: float | None, **details: object) -> dict:
|
||||
@@ -244,7 +361,11 @@ def _sensor(sensor_id: str, label: str, score: float | None, **details: object)
|
||||
}
|
||||
|
||||
|
||||
def _score_pillars(pillars: list[dict], weights: dict[str, float]) -> dict:
|
||||
def _score_pillars(
|
||||
pillars: list[dict],
|
||||
weights: dict[str, float],
|
||||
bands: tuple[float, float, float] = STATE_BANDS,
|
||||
) -> dict:
|
||||
expected = sum(max(0.0, float(w)) for w in weights.values())
|
||||
available_weight = sum(
|
||||
max(0.0, float(weights.get(p["id"], 0.0)))
|
||||
@@ -276,7 +397,12 @@ def _score_pillars(pillars: list[dict], weights: dict[str, float]) -> dict:
|
||||
rounded = round(score, 1) if score is not None else None
|
||||
return {
|
||||
"score": rounded,
|
||||
"band": band_for(rounded) if rounded is not None and coverage >= MIN_COVERAGE else None,
|
||||
"band": (
|
||||
band_for(rounded, bands)
|
||||
if rounded is not None and coverage >= MIN_COVERAGE
|
||||
else None
|
||||
),
|
||||
"bands": {"watch": bands[0], "elevated": bands[1], "breaking": bands[2]},
|
||||
"coverage": round(coverage, 1),
|
||||
"minimum_coverage": MIN_COVERAGE,
|
||||
"available_pillars": [p["id"] for p in rows if p["available"]],
|
||||
@@ -309,13 +435,24 @@ def _value_asof(series: Series | None, as_of: date) -> float | None:
|
||||
return item[1] if item else None
|
||||
|
||||
|
||||
def _window_asof(series: Series | None, as_of: date, years: float) -> list[float]:
|
||||
def _window_asof(series: Series | None, as_of: date, days: int) -> list[float]:
|
||||
if not series:
|
||||
return []
|
||||
start = as_of - timedelta(days=int(365.25 * years))
|
||||
start = as_of - timedelta(days=days)
|
||||
return [v for d, v in series if start <= d <= as_of]
|
||||
|
||||
|
||||
def _coverage_days(series: Series | None, as_of: date) -> int | None:
|
||||
"""Span of history actually available at ``as_of``.
|
||||
|
||||
Recorded in every snapshot because the v2 credit percentile degraded from a
|
||||
10-year to a 3-year reference silently when the upstream licence changed --
|
||||
nothing asserted the window it claimed, so nothing noticed for months.
|
||||
"""
|
||||
dates = [d for d, _ in series or [] if d <= as_of]
|
||||
return (as_of - dates[0]).days if dates else None
|
||||
|
||||
|
||||
def _next_weekday(d: date) -> date:
|
||||
candidate = d + timedelta(days=1)
|
||||
while candidate.weekday() >= 5:
|
||||
@@ -340,19 +477,31 @@ def _fundamental_effective_date(overrides: dict) -> date | None:
|
||||
return _next_weekday(fetched) if fetched else None
|
||||
|
||||
|
||||
def _fundamental_scores_asof(overrides: dict, config: dict, as_of: date) -> tuple[float | None, float | None, dict]:
|
||||
def fundamental_overlay(overrides: dict, config: dict, as_of: date) -> dict:
|
||||
"""Point-in-time qualitative overlay. Never feeds State or Warning in v3.
|
||||
|
||||
The effective-date gate stays even though nothing is scored from this: the
|
||||
400-session rebuild replays historical dates, and stamping today's LLM read
|
||||
onto 2024 snapshots would be plain lookahead in the stored record.
|
||||
"""
|
||||
effective = _fundamental_effective_date(overrides)
|
||||
if effective is None or as_of < effective:
|
||||
return None, None, {"effective_date": effective.isoformat() if effective else None, "age_days": None}
|
||||
age = (as_of - effective).days
|
||||
stale = age > int(config.get("fundamental_staleness_days", 80))
|
||||
f1 = overrides.get("f1_score")
|
||||
f3 = overrides.get("f3_score")
|
||||
return (
|
||||
None if stale or f1 is None else _clamp(float(f1)),
|
||||
None if stale or f3 is None else _clamp(float(f3)),
|
||||
{"effective_date": effective.isoformat(), "age_days": age, "stale": stale},
|
||||
)
|
||||
pending = effective is None or as_of < effective
|
||||
age = None if pending else (as_of - effective).days
|
||||
stale = bool(age is not None and age > int(config.get("fundamental_staleness_days", 80)))
|
||||
return {
|
||||
"available": not pending and not stale,
|
||||
"pending": pending,
|
||||
"stale": stale,
|
||||
"effective_date": effective.isoformat() if effective else None,
|
||||
"age_days": age,
|
||||
"capex": None if pending else overrides.get("capex"),
|
||||
"good_news_stock_down": None if pending else overrides.get("good_news_stock_down"),
|
||||
"capex_stress": None if pending else overrides.get("f1_score"),
|
||||
"earnings_stress": None if pending else overrides.get("f3_score"),
|
||||
"reasoning": None if pending else overrides.get("reasoning"),
|
||||
"source": overrides.get("source"),
|
||||
"fetched_at": overrides.get("fetched_at"),
|
||||
}
|
||||
|
||||
|
||||
def _basket_hash(symbols: list[str]) -> str:
|
||||
@@ -393,12 +542,14 @@ def _compute_index(
|
||||
vix_item = _item_asof(vix_series, as_of)
|
||||
vix_score = p5_volatility(vix_item[1] if vix_item else None)
|
||||
oas_item = _item_asof(oas_series, as_of)
|
||||
oas_window = _window_asof(oas_series, as_of, HY_OAS_REFERENCE_YEARS)
|
||||
oas_window = _window_asof(oas_series, as_of, HY_OAS_WINDOW_DAYS)
|
||||
credit_score = f2_credit_spreads(oas_window)
|
||||
|
||||
divergence = _value_asof(divergence_series, as_of)
|
||||
relative_strength = p4_relative_strength(smh, spy)
|
||||
f1, f3, fundamental_meta = _fundamental_scores_asof(overrides, config, as_of)
|
||||
sensors = warning_sensor_scores(divergence, smh, spy, oas_window)
|
||||
relative_strength = sensors["relative_strength"]
|
||||
credit_impulse = sensors["credit_impulse"]
|
||||
overlay = fundamental_overlay(overrides, config, as_of)
|
||||
|
||||
state_pillars = [
|
||||
{
|
||||
@@ -444,21 +595,22 @@ def _compute_index(
|
||||
"sensors": [_sensor("W2", "60-session relative-strength deterioration", relative_strength)],
|
||||
},
|
||||
{
|
||||
"id": "capex",
|
||||
"label": "Hyperscaler capex revisions",
|
||||
"score": round(f1, 1) if f1 is not None else None,
|
||||
"sensors": [_sensor("F1", "Capex guidance cuts", f1)],
|
||||
},
|
||||
{
|
||||
"id": "earnings_reaction",
|
||||
"label": "Good news, stock down",
|
||||
"score": round(f3, 1) if f3 is not None else None,
|
||||
"sensors": [_sensor("F3", "Abnormal earnings reaction", f3)],
|
||||
"id": "credit_impulse",
|
||||
"label": "Credit impulse",
|
||||
"score": round(credit_impulse, 1) if credit_impulse is not None else None,
|
||||
"sensors": [
|
||||
_sensor(
|
||||
"W3",
|
||||
f"HY OAS {W3_OAS_LOOKBACK}-session widening",
|
||||
credit_impulse,
|
||||
oas=oas_item[1] if oas_item else None,
|
||||
)
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
state = _score_pillars(state_pillars, STATE_WEIGHTS)
|
||||
warning = _score_pillars(warning_pillars, WARNING_WEIGHTS)
|
||||
state = _score_pillars(state_pillars, STATE_WEIGHTS, STATE_BANDS)
|
||||
warning = _score_pillars(warning_pillars, WARNING_WEIGHTS, WARNING_BANDS)
|
||||
|
||||
price_item = _item_asof(prices.get(tickers["leaders"][0]), as_of)
|
||||
dated_sources = {
|
||||
@@ -481,6 +633,7 @@ def _compute_index(
|
||||
"date": as_of.isoformat(),
|
||||
"state": state,
|
||||
"warning": warning,
|
||||
"fundamental_overlay": overlay,
|
||||
"quadrant_config": {
|
||||
"state_divider": QUADRANT_STATE_DIVIDER,
|
||||
"warning_divider": QUADRANT_WARNING_DIVIDER,
|
||||
@@ -502,14 +655,18 @@ def _compute_index(
|
||||
"breadth_pct_above_200": round(breadth_pct, 1) if breadth_pct is not None else None,
|
||||
"breadth_date": breadth_item[0].isoformat() if breadth_item else None,
|
||||
"fundamentals_fetched_at": overrides.get("fetched_at"),
|
||||
"fundamentals_effective_date": fundamental_meta.get("effective_date"),
|
||||
"fundamentals_age_days": fundamental_meta.get("age_days"),
|
||||
"fundamentals_effective_date": overlay.get("effective_date"),
|
||||
"fundamentals_age_days": overlay.get("age_days"),
|
||||
},
|
||||
"data_quality": {
|
||||
"minimum_coverage": MIN_COVERAGE,
|
||||
"oldest_market_input_age_days": max(source_ages.values()) if source_ages else None,
|
||||
"stale_inputs": stale_inputs,
|
||||
"inputs_fresh": not stale_inputs,
|
||||
# Upstream history spans, so a provider silently truncating a series
|
||||
# shows up in the record instead of quietly reshaping a sensor.
|
||||
"credit_history_days": _coverage_days(oas_series, as_of),
|
||||
"vix_history_days": _coverage_days(vix_series, as_of),
|
||||
},
|
||||
}
|
||||
|
||||
@@ -581,7 +738,11 @@ async def get_fundamental_overrides(db: AsyncSession) -> dict:
|
||||
stored = json.loads(raw)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
if stored.get("methodology") != METHODOLOGY:
|
||||
# The guard rejects pre-v2 blobs, where f1/f3 were arbitrary numbers with no
|
||||
# categorical source. v2 and v3 share the categorical format and both derive
|
||||
# f1/f3 from it below, so a methodology bump must not discard a live
|
||||
# observation -- only the capex *scale* changed, and that is recomputed.
|
||||
if stored.get("methodology") not in CATEGORICAL_FUNDAMENTAL_METHODOLOGIES:
|
||||
return default
|
||||
capex = _normalise_capex_states(stored.get("capex"), names)
|
||||
reaction = str(stored.get("good_news_stock_down", "mixed")).strip().lower()
|
||||
@@ -745,7 +906,7 @@ async def _upsert_snapshot(
|
||||
created_at=datetime.now(timezone.utc),
|
||||
))
|
||||
else:
|
||||
existing_v2 = _parse_v2(row.breakdown_json)
|
||||
existing_v2 = _parse_snapshot(row.breakdown_json)
|
||||
if existing_v2 is not None and not rewrite_existing_v2:
|
||||
return False, existing_v2
|
||||
row.total_score = float(state_score or 0.0)
|
||||
@@ -754,7 +915,7 @@ async def _upsert_snapshot(
|
||||
return True, result
|
||||
|
||||
|
||||
def _parse_v2(raw: str) -> dict | None:
|
||||
def _parse_snapshot(raw: str) -> dict | None:
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except (TypeError, ValueError):
|
||||
@@ -762,12 +923,12 @@ def _parse_v2(raw: str) -> dict | None:
|
||||
return parsed if parsed.get("methodology") == METHODOLOGY else None
|
||||
|
||||
|
||||
async def _latest_v2_row(db: AsyncSession) -> tuple[RegimeSnapshot, dict] | None:
|
||||
async def _latest_snapshot_row(db: AsyncSession) -> tuple[RegimeSnapshot, dict] | None:
|
||||
result = await db.execute(
|
||||
select(RegimeSnapshot).order_by(RegimeSnapshot.date.desc()).limit(1000)
|
||||
)
|
||||
for row in result.scalars().all():
|
||||
parsed = _parse_v2(row.breakdown_json)
|
||||
parsed = _parse_snapshot(row.breakdown_json)
|
||||
if parsed is not None:
|
||||
return row, parsed
|
||||
return None
|
||||
@@ -791,8 +952,10 @@ async def update_regime_monitor(db: AsyncSession, rebuild_sessions: int = REBUIL
|
||||
latest_date = leader_series[-1][0]
|
||||
|
||||
vix_series = await _fetch_fred_series("VIXCLS", end - timedelta(days=1200), end)
|
||||
# Asking for 13 years was misleading once the licence capped the series at 3;
|
||||
# the level needs the latest point and W3 needs its lookback, nothing more.
|
||||
oas_series = await _fetch_fred_series(
|
||||
"BAMLH0A0HYM2", end - timedelta(days=int(365.25 * 13)), end
|
||||
"BAMLH0A0HYM2", end - timedelta(days=HY_OAS_WINDOW_DAYS), end
|
||||
)
|
||||
|
||||
basket = config["breadth_basket"]
|
||||
@@ -805,7 +968,7 @@ async def update_regime_monitor(db: AsyncSession, rebuild_sessions: int = REBUIL
|
||||
logger.warning("Regime monitor: fixed-basket breadth skipped: %s", exc)
|
||||
breadth, breadth_counts, divergence = {}, {}, {}
|
||||
|
||||
latest_v2 = await _latest_v2_row(db)
|
||||
latest_v2 = await _latest_snapshot_row(db)
|
||||
rebuilding = latest_v2 is None and bool(leader_series)
|
||||
if rebuilding:
|
||||
dates = [d for d, _ in leader_series[-max(1, rebuild_sessions):]]
|
||||
@@ -860,7 +1023,7 @@ async def _result_at_or_before(
|
||||
.limit(1000)
|
||||
)
|
||||
for raw in result.scalars().all():
|
||||
parsed = _parse_v2(raw)
|
||||
parsed = _parse_snapshot(raw)
|
||||
parsed_hash = ((parsed or {}).get("basket") or {}).get("hash")
|
||||
if parsed is not None and (basket_hash is None or parsed_hash == basket_hash):
|
||||
return parsed
|
||||
@@ -877,7 +1040,7 @@ def _delta(current: dict, previous: dict | None) -> float | None:
|
||||
|
||||
|
||||
async def get_regime_monitor(db: AsyncSession) -> dict:
|
||||
latest = await _latest_v2_row(db)
|
||||
latest = await _latest_snapshot_row(db)
|
||||
if latest is None:
|
||||
return {"available": False, "reason": "v2 not computed yet"}
|
||||
row, result = latest
|
||||
@@ -902,6 +1065,15 @@ async def get_regime_monitor(db: AsyncSession) -> dict:
|
||||
quality["snapshot_age_days"] = snapshot_age
|
||||
quality["is_fresh"] = bool(quality.get("inputs_fresh")) and snapshot_age <= 4
|
||||
result["data_quality"] = quality
|
||||
|
||||
# The snapshot's overlay is the point-in-time record; the reader also wants
|
||||
# the current observation even when it is not effective until the next
|
||||
# session, because otherwise refreshing it looks like it did nothing.
|
||||
config = await get_regime_config(db)
|
||||
overrides = await get_fundamental_overrides(db)
|
||||
live = fundamental_overlay(overrides, config, date.today())
|
||||
live["observed_in_snapshot"] = bool((result.get("fundamental_overlay") or {}).get("available"))
|
||||
result["fundamental_context"] = live
|
||||
result["available"] = True
|
||||
return result
|
||||
|
||||
@@ -915,7 +1087,7 @@ async def get_regime_history(db: AsyncSession, days: int = 800) -> list[dict]:
|
||||
)
|
||||
out: list[dict] = []
|
||||
for row in result.scalars().all():
|
||||
data = _parse_v2(row.breakdown_json)
|
||||
data = _parse_snapshot(row.breakdown_json)
|
||||
if data is None:
|
||||
continue
|
||||
state, warning = data.get("state") or {}, data.get("warning") or {}
|
||||
|
||||
Reference in New Issue
Block a user