Two sensors saturated in exactly the range where resolution matters, and the top
State band had no headroom. Calibrated with scripts/run_regime_monitor_calibration.py
over the 408 sessions ending 2026-07-24; the shipped code reproduces that run's
band shares exactly (78.9 / 13.0 / 4.7 / 3.4).
V1 read VIX 30, 50 and 82 as an identical 100 — the same defect v3 had just
removed from P3, left in place one sensor over. In the window it flattened five
distinct April-2025 prints (52.33, 46.98, 45.31, 40.72, 38.57) into one value.
Now an anchor table reaching full scale at 55, not at 2020's ~82: anchoring the
top at a once-in-a-generation print would make VIX 50 read only ~70. Pegged on
14 of 408 sessions before; none now.
_under_200 returned a bare 0/100, so P1 printed 100 the moment SMH and QQQ were
both under their average — and since the price pillar takes max(P1, P2, P3),
that pinned the pillar and stopped P3's ladder resolving for the whole of a
selloff. Now graded by depth below the 200-DMA, with a deliberate floor of 20 at
the crossing: the break is a genuine binary event, only its depth is graded.
Pegged on 46 of 408 sessions before; none now. A 2% break reads ~30, not 100.
max() was KEPT — the defect was the step function feeding it, not the vote, and
v3's "one capped vote for correlated reads" rationale still holds. P1 is the sole
price argmax on 17 of 408 sessions (4.2%), so the P1_SCORE_CAP fallback drafted
during design was measured as unnecessary and not shipped.
STATE_BANDS breaking 80 -> 65, and only that threshold. Credit returns 0.0 (not
None) when calm, so it holds its 20 points pinned at zero and price + breadth +
volatility at literal maximum summed to exactly 80.0 — v3's threshold to the
decimal, with nothing above it. The sensor is deliberately unchanged: a
calm-credit selloff genuinely is less stressed. What was stale is the band, fit
on v2 while credit's since-removed percentile leg still contributed. A
2022-style AI/tech drawdown with calm credit computes to 70.3 (no death cross) or
74.0 (with one); 70 would have left 0.33 points of headroom, reproducing the
defect. Chosen by scenario arithmetic, and the realized breaking share then lands
on 3.4% — the same as v3's, arrived at independently.
"v4" added to CATEGORICAL_FUNDAMENTAL_METHODOLOGIES in this same commit, which is
load-bearing: that set is checked against the STORED blob, so bumping without it
discards the collected observation on first write, leaving fetched_at null and
locked false — and update_regime_monitor then fires a paid LLM refresh on every
run, forever. Now guarded by a test parametrised over v2 and v3 stored blobs.
SENSOR_REVISION deliberately stays 2: a METHODOLOGY change already forces a full
reseed via _parse_snapshot, and bumping both would imply the reseed was
revision-driven.
QUADRANT_STATE_DIVIDER stays 50 because only breaking moved, so alert_service,
RegimeChart and the quadrant tests need no change. A new test enforces
divider == band boundary on both axes, which nothing did before.
Doc renamed to regime-monitor-v4.md with a tombstone at the old path (commit
messages cite it), the three open questions converted to resolved with the
reasoning that closed them, and indexed in docs/research/README.md for the first
time. The P2 limit is stated honestly: _death_cross pegs at a -5% MA gap, so a
deep selloff still reaches 100 via P2 — v4 repairs the shallow-to-moderate break,
not "the price pillar no longer pegs".
DEPLOY: the first run reseeds ~464 sessions. Expect one phantom quadrant alert
(the dedup key carries basket_hash, not methodology) and re-run the Event Study
manually — its cached report self-invalidates but does not self-regenerate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follows 5ea0785, which renamed the user-visible labels. This finishes the pass
so code, docs and operator output use one vocabulary: README (pipeline list,
route table, FRED row), the methodology doc title, .env.example and config
comments, the snapshot model / event-study / service / test docstrings, the
scheduler section headers and morning-pipeline docstring, the TopBar status
text ("bullish regime" -> "bullish trend"), and the four "Regime monitor:" log
prefixes.
Deliberately NOT changed, because "market regime" is also a standard finance
term and most occurrences are not this job: the backtest caveat "~6 months is
roughly one market regime" in backtest_service, README, BacktestPanel and every
generated reports/*.json; "a regime shift" in TrackRecordPanel; and the
capacity-bracket findings doc. Renaming those would have made the text wrong.
Also unchanged, being persisted or externally linked rather than wording: the
regime_monitor / market_regime job ids, the regime_quadrant_enabled setting key,
the /regime route, METHODOLOGY and the snapshot fields, the service/test module
filenames, and docs/research/regime-monitor-v3.md's path (referenced from commit
messages). The doc now carries a one-line note recording the old name and why
those identifiers still use it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two review findings on 46ace50.
[P1] Raising HY_OAS_WINDOW_DAYS to 700 only reached newly computed rows. A
routine run recomputes the latest trading date alone, and `rebuilding` was keyed
on "no v3 snapshot exists at all", which is false once the cutover has run --
so every row already written kept the credit gap the wider window exists to
close, indefinitely.
Adds SENSOR_REVISION: stamped into each snapshot, absent on pre-marker rows
(read as 1), and a stored revision below the current one triggers exactly one
reseed. Deliberately not METHODOLOGY, which would partition the history API and
discard the cached event study -- neither warranted, since the study recomputes
its Warning series from source rather than reading snapshots and so cannot be
staled by a reseed.
The reseed is bounded by REBUILD_LOOKBACK_DAYS in calendar days rather than a
session count, because the binding constraint is the OAS fetch: each replayed
row needs W3's 20-business-day lookback inside HY_OAS_WINDOW_DAYS. Replaying by
session count would have left the oldest stored rows unrepaired -- the exact
rows the fix targets. At 672 days the replay covers ~464 sessions, W3's oldest
requirement lands on the first fetched OAS day, and the ~400-session series the
cutover wrote is fully covered. A test asserts that relationship so the two
constants cannot drift back into recreating the gap.
[P2] With nothing ever collected, current_observation returned available=true
and the default placeholders -- "unknown" for every hyperscaler, "mixed" for the
reaction -- so the card announced a reading that never happened. Those are the
absence of an observation, not an observation of absence. Gated on `observed`
(non-null fetched_at, the one field every path writing real content stamps),
which blanks the content and drives a proper empty state naming where an admin
collects one. This was a regression from 46ace50; fundamental_overlay never had
it, since no observation means no effective date means pending.
Also renames the leftover v2 identifiers in the touched paths
(rewrite_existing_v2, latest_v2).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The page had twelve stacked blocks, several of them different views of the
same numbers. The quadrant plot and the score-history chart drew the same two
series from the same query key, which read as two datasets; they are now one
card with a Time | Path toggle. The two pillar disclosures become one grouped
table, and three prose blocks (data quality, basket, coverage) become one
provenance chip strip. Page text is now limited to what changes how the reader
interprets today's number; the rest moved to the methodology doc.
Removes three stale-threshold bugs of one class. The quadrant fell back to v2's
60/60 dividers when quadrant_config was absent -- the real values are 50/40 and
they feed alert_service, so the chart could disagree with what actually fires.
The gauge fell back to v2's 30/60/80 band ticks, and drew a divider line that
always landed on its own "elevated" tick. The time series' reference lines were
at 30/60/80, which correspond to nothing in v3; they are now per-axis dashed
lines read from the same quadrant_config. Rendering also surfaced a live
clipping bug inherited from the old chart: margin.left -18 against YAxis
width 28 left ~10px for a 3-digit label, so every Y tick was cut off.
HY_OAS_WINDOW_DAYS was 400 *calendar* days while a rebuild replays
REBUILD_SESSIONS = 400 *trading* sessions (~579 calendar days), so the oldest
~180 days of any rebuild got no OAS at all and both credit sensors returned
None. State then lands at 80% coverage and Warning at exactly MIN_COVERAGE, so
both still publish bands -- a series that looks homogeneous while its oldest
rows were scored without credit. Widened to 700. This needs no methodology
bump: C1 reads [-1] and W3 reads [-21], both from the end, so widening only
prepends and every live score is bit-identical. Sequenced deliberately, since
acting on the open findings below bumps METHODOLOGY and fires the rebuild.
A just-collected fundamental observation was hidden until its effective date --
one day, three over a weekend -- because the live reading called the
point-in-time function, so refreshing appeared to do nothing. That was the
opposite of what the doc claimed. fundamental_overlay stays the gated record
(it runs for every replayed date during a rebuild); current_observation is the
live reading and reports the effective date instead of blanking the content.
Nothing in the overlay is scored, so showing it early cannot reach a published
number.
Documents four calculation findings. Three are not implemented, since each
changes a published score and so requires a v4 cut: State's top band is a
credit-event band (credit returns 0.0 rather than None below the 3.5 anchor, so
it is pinned at zero at weight 20 -- with everything else pegged State computes
to exactly 80.0, the breaking threshold); V1 saturates at VIX 30; and the
deliberate max(P1,P2,P3) defeats P3's anchoring because P1 is binary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings the durable artifacts of research/portfolio-capacity-rebalancing onto
main so the rationale for raising the count cap lives with the code that cites
it. The matrix runner, the research simulator hooks and the study's unit tests
are deliberately left behind; they remain at tag research/portfolio-capacity-final.
Corrects conclusions that were reached on EV per trade and are now superseded:
the findings doc's decisions 1 (keep cap 10) and 4 (run the risk-floor A/B) are
struck through and answered in a new correction section, and the research README
and phase-A matrix entries are updated to match. The frozen specification itself
is untouched -- its recorded SHA-256 f1e37783 still verifies.
effective-risk-floor-ab.md is retained but marked CLOSED/NEGATIVE: the study it
proposes is already answered by cap15 vs cash_unbounded (-0.753pp CAGR while
EV/trade rises), and its EV-based pass rule would have shipped it.
scripts/research_rankings.py replaces a fourth copy of the historical rank-map
helper; run_research_matrix, run_execution_recovery_matrix and
run_daily_reentry_matrix now share it. The shared version adds a duplicate
observation guard and a deterministic symbol tie-break the copies lacked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The v3 cutover run scored 2/4 corrections warned against v2's 3/4, which reads
like a regression and is not one. Only 4 of the 11 detected corrections fall in
the holdout, so recall is one event from a different headline -- and the event
that flips is decided by threshold placement, not by what the score saw. "v3
without the credit sensor" catches 2025-02-21 at a *higher* threshold (35.5)
than shipped v3 misses it at (32.3), because the alarm rule needs a rising edge
and a lower threshold can fire outside the horizon then never reset below.
Two caveats are now computed and surfaced rather than left for the reader to
infer:
- Holdout event count against MIN_EVENTS_FOR_CONFIDENCE. The summary sentence
states how many of the detected corrections actually fall in the test period.
- Warning-sensor coverage across the split. The score renormalises over what is
available, so a training window predating a sensor's history freezes the
threshold on a different construct than the holdout is measured against. At
the cutover that is 39% of training sessions with all three sensors versus
100% of the test period, credit history beginning 2023-07-25.
Restricting the threshold to sensor-matched training sessions was tested and
rejected: those sessions are a calm recent stretch, so the threshold falls from
32.3 to 22.5 and false alarms rise from 3.3 to 8.6/yr. It swaps a coverage bias
for a regime-selection bias. The report states its limits instead.
_warning_series now returns per-session sensor counts alongside the scores.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The LLM-sourced capex/earnings observations carried 12+8 of 100 Warning points,
so both pegged at 100 produced a Warning of 20.0 -- below the event study's 25.3
alarm threshold and still inside the "stable" band. The reading was
arithmetically incapable of changing anything on screen, which is why refreshing
it appeared to do nothing. They are now a qualitative overlay reported beside
the scores rather than diluted into them.
Calibrated against the 408 v2 sessions to 2026-07-24, reproduced offline from
Alpaca + FRED; the harness matched the stored prod distribution exactly before
any parameter was changed.
State:
- P3 used dd_pct * 5, reaching 100 at a 20% drawdown -- the 90th percentile of
the observed distribution -- so 39/408 sessions sat at exactly 100 with no
resolution left during the part of a selloff that matters most. Replaced with
anchored breakpoints keeping headroom past the observed 36% maximum, blended
2:1 like P1/P2 instead of max(). P3's realized share of State falls from 65%
to 40%, matching its nominal weight.
- Credit level is now anchors-only. ICE capped FRED's BAMLH0A0HYM2 at a rolling
3-year window in April 2026, silently turning the 10-year percentile leg into
a 3-year one that scored 20 points of stress at an OAS of 3.5 -- the level its
own anchors call "mild". The anchors already encode the long-run distribution.
Warning:
- Added HY OAS 20-session widening (25%). The level is pinned at zero below the
3.5 anchor; its rate of change is not.
- Divergence tapers to a 0.35 floor instead of a hard price_ret >= 0 gate, which
zeroed the sensor through every decline: on 2026-07-24 the basket shed 10
points of participation in 20 sessions and Warning printed exactly 0.
- The event study and the live monitor now share one sensor definition, so they
cannot silently drift apart.
Bands are per axis (State 20/50/80, Warning 20/40/60) with quadrant dividers at
50/40; v2 Warning never exceeded 64.9 against a shared 60, leaving that half of
the quadrant unreachable. Realized shares: State 73/15/8/3%, Warning 69/20/8/3%.
Snapshots now record credit_history_days and vix_history_days -- the percentile
defect went unnoticed for months because nothing asserted the window the code
claimed.
Cutover: the first run rebuilds 400 sessions automatically; the Event Study job
must be re-run, as its cached report self-invalidates on the methodology check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Earnings backfill sourced from the public DoltHub earnings repo at a
pinned commit rather than the FMP API: reproducible for anyone re-running
the study, and it burns no request quota. 12,414 events, 98.6% of symbols
with >=8 announcements, 99.2% paired actual/estimate, no keyed duplicates.
2a earnings-gap diagnostic: INFORMATIONAL, no filter shipped. The
pre-earnings cohort's right tail was better, so the registered
avoid-earnings condition failed. Note the raw 23/266 vs 115/574 incidence
gap is largely a duration confound -- severe losses stop out fast and have
less time to span an announcement -- so it is not evidence that holding
through earnings is safe.
2b SUE: FAIL against the pre-registered +0.03 bar (unconditional IC
+0.0151 over 56 reliable windows, momentum-conditional +0.0213). Signs
stable across eras, so this is a clean null rather than an ambiguous one,
consistent with post-earnings drift having decayed in large caps.
Closes the Tier-1 arc: Task 1 dead on deep evidence, Task 2 dead here,
Task 3 complete as diagnostic. No in-sample research thread remains open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop intermediate history-depth reports, sector-residual runners/map/code hooks
(evidence stays in final reports + docs), and slim MacBook helper to ssl/earnings/
prod-book-matrix only. SSL bootstrap and archived research conclusions retained.
Four-arm results: 505 stays positive (softer on deep history); liquid breadth
destroys book under current knobs. Stop tracking 1.3GB pkl cache under reports/.cache.
Pre-register A-D (4y/2016 x 505/505+liquid) with unchanged production knobs.
Runner caches full GTL candidates then re-ranks per arm; MacBook entry via
run_tier1_macbook.sh --prod-book-matrix.
Deep masked retest failed the iron IC bar (0.027 < 0.03). Log as rejected #13 in
research README; close sector-residual and history-depth docs. Production market
residual unchanged.
Authoritative report history-depth-20260719-103315: race guard pass on deep
research.sqlite. Sector residual still short-window only (no pre-2021); fip sign
flips on broad deep sample; no production retune.
Tier-1 alpha research (local only, no production deploy):
Sector residual momentum: two-factor SPY+sector residual and sector demean signals, IC harness + A/B. Sector resid clears pre-registered bars narrowly (PROMOTE for human wire design only). Sector demean fails t vs market resid.
Earnings: earnings_events backfill (FMP bulk paid; FMP/AV per-symbol), 2a gap diagnostic report-only, 2b SUE IC (PARK; incomplete 48/506 coverage).
History-depth: pre-registered doc + runner for MacBook deep rebuild/harness.
Do not ship production residual or filters from this branch.
Log the 21:14 orphan as a snapshot-build race, rewrite the context table to
authoritative ICs only, and soften the vol-tilt warning. Add extender completion
manifest + breadth refuse guard; strip intermediate/orphaned reports; park the
thread (no book sim, no deploy).
Add lagged/tier/prod-subset/mom-conditional checks on research.sqlite.
Log: unconditional sign is a winner/bleeder tug-of-war; mom-conditional
fip stays negative and reliable; warn on high-vol tilt if universe broadens.
Add research-only snapshot extender, PIT dollar-volume mask for signal IC,
rank-only harness path, fingerprint+breadth runner, and docs. Fingerprint
reproduced IC -0.045 / t -2.91 on prod.sqlite. No production gate/schedule changes.
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts,
put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day
requalify semantics, stamp paper trades fill_mode=near_close, and migrate
stored schedule_* keys to America/New_York.
Record monotone fill-timing gradient, live [1.57,1.77] bracket, recover-flag
interpretation, gap-cap as third tail-trim, and pre-scheduler ops checklist.
Document Phase A (max-hold/vol/corr closed; next-open as decision baseline).
Add stale_close and next_open gap-cap fill modes plus a small matrix to test
whether near-close scheduling recovers overnight momentum drift.
Honor custom S/R tolerance as a transient detect, refresh levels after OHLCV
mutations without failing committed price writes, report per-ticker S/R
rebuild failures from admin cleanup, and warn in the admin UI when refresh is partial.
dennisthiessenandClaude Opus 5