The v3 cutover run scored 2/4 corrections warned against v2's 3/4, which reads
like a regression and is not one. Only 4 of the 11 detected corrections fall in
the holdout, so recall is one event from a different headline -- and the event
that flips is decided by threshold placement, not by what the score saw. "v3
without the credit sensor" catches 2025-02-21 at a *higher* threshold (35.5)
than shipped v3 misses it at (32.3), because the alarm rule needs a rising edge
and a lower threshold can fire outside the horizon then never reset below.
Two caveats are now computed and surfaced rather than left for the reader to
infer:
- Holdout event count against MIN_EVENTS_FOR_CONFIDENCE. The summary sentence
states how many of the detected corrections actually fall in the test period.
- Warning-sensor coverage across the split. The score renormalises over what is
available, so a training window predating a sensor's history freezes the
threshold on a different construct than the holdout is measured against. At
the cutover that is 39% of training sessions with all three sensors versus
100% of the test period, credit history beginning 2023-07-25.
Restricting the threshold to sensor-matched training sessions was tested and
rejected: those sessions are a calm recent stretch, so the threshold falls from
32.3 to 22.5 and false alarms rise from 3.3 to 8.6/yr. It swaps a coverage bias
for a regime-selection bias. The report states its limits instead.
_warning_series now returns per-session sensor counts alongside the scores.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The LLM-sourced capex/earnings observations carried 12+8 of 100 Warning points,
so both pegged at 100 produced a Warning of 20.0 -- below the event study's 25.3
alarm threshold and still inside the "stable" band. The reading was
arithmetically incapable of changing anything on screen, which is why refreshing
it appeared to do nothing. They are now a qualitative overlay reported beside
the scores rather than diluted into them.
Calibrated against the 408 v2 sessions to 2026-07-24, reproduced offline from
Alpaca + FRED; the harness matched the stored prod distribution exactly before
any parameter was changed.
State:
- P3 used dd_pct * 5, reaching 100 at a 20% drawdown -- the 90th percentile of
the observed distribution -- so 39/408 sessions sat at exactly 100 with no
resolution left during the part of a selloff that matters most. Replaced with
anchored breakpoints keeping headroom past the observed 36% maximum, blended
2:1 like P1/P2 instead of max(). P3's realized share of State falls from 65%
to 40%, matching its nominal weight.
- Credit level is now anchors-only. ICE capped FRED's BAMLH0A0HYM2 at a rolling
3-year window in April 2026, silently turning the 10-year percentile leg into
a 3-year one that scored 20 points of stress at an OAS of 3.5 -- the level its
own anchors call "mild". The anchors already encode the long-run distribution.
Warning:
- Added HY OAS 20-session widening (25%). The level is pinned at zero below the
3.5 anchor; its rate of change is not.
- Divergence tapers to a 0.35 floor instead of a hard price_ret >= 0 gate, which
zeroed the sensor through every decline: on 2026-07-24 the basket shed 10
points of participation in 20 sessions and Warning printed exactly 0.
- The event study and the live monitor now share one sensor definition, so they
cannot silently drift apart.
Bands are per axis (State 20/50/80, Warning 20/40/60) with quadrant dividers at
50/40; v2 Warning never exceeded 64.9 against a shared 60, leaving that half of
the quadrant unreachable. Realized shares: State 73/15/8/3%, Warning 69/20/8/3%.
Snapshots now record credit_history_days and vix_history_days -- the percentile
defect went unnoticed for months because nothing asserted the window the code
claimed.
Cutover: the first run rebuilds 400 sessions automatically; the Event Study job
must be re-run, as its cached report self-invalidates on the methodology check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parity gate has been exercised: nine-pass investigation, fixes, two prod
reparses, and an after-report at 504/511 candidate coverage with agreement
unchanged. What remains of workstream A is the activation itself (step c, the
post-approval fundamental_data refresh — not yet implemented) and A6
decommissioning, both now specified in a handoff section with the caveats the
next implementer must carry (KLAC splits, BRK-B, FITB, the 25% guard, CIK
overrides, reparse-after-parser-changes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. net_debt requires BOTH cash and total_debt; a missing side is null, not
treated as zero (which would be a partial, misleading value).
2. net_debt_to_ebitda is null when TTM EBITDA <= 0 — a negative denominator
would otherwise rank a distressed issuer as favorably low-leverage.
3. The quarter tape is the CONSECUTIVE run ending at the latest period (stops at
a gap), so trend text never compares non-adjacent quarters as if consecutive.
4. YoY growth is null when the prior-year TTM is <= 0 (e.g. loss->profit), which
is not a meaningful percentage.
Also corrected the plan's net-debt formula to total debt − (cash + ST) matching
the positive-means-net-debt implementation. +4 tests. 10 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. Removed the 45-day index-walk cap: it discarded the older part of a long
outage while still advancing source_max_date, permanently losing filings.
The walk now covers every unprocessed date (a large gap is one-time cost).
2. Discrepancy detection meets the immutability contract: it compares ALL source
snapshot fields (not five), read-only during stage/validate, reports the
differing accessions + fields in validation_json, and promote emits a warning
system event (in-transaction) — never mutating the stored row.
3. Malformed companyfacts (missing facts/units structure) are recorded separately
and FAIL validation, instead of silently degrading to skipped rows that the
50% backfill coverage floor could still pass.
Also corrected the stale "sum share classes" / DEI-only wording in the snapshot
model docstring and the A3 design doc to describe the us-gaap fallback.
Tests: +4 regressions (>45-day gap loses nothing, newly-added issuer backfills
without filing, malformed payload fails, shares discrepancy detected + evented).
23 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. Multi-class shares: prefer the single dei:EntityCommonStockSharesOutstanding
cover-page fact; else fall back to us-gaap:CommonStockSharesOutstanding at
period end (Alphabet has no dei fact). Never sum class facts (companyfacts is
non-dimensional) and never use weighted-average/diluted; conflicting values ->
null, counted as an "ambiguous shares outstanding" note in validation. Plan's
"sum class-specific" wording corrected. Verified live: Alphabet shares now
populate (12.1B), Apple still uses its dei cover date.
2. Fiscal context is the majority (fy, fp) among facts ending at reportDate, with
ties rejected — no longer the arbitrary first fact.
3. Hardening: catalog selectors require taxonomy == "us-gaap"; indexing drops
malformed facts (missing accession/end, non-finite value) so a custom concept
or bad date can't be selected.
Tests: +8 (dei precedence, us-gaap fallback, conflict->null, no weighted-average,
tie-context skip, foreign-taxonomy/malformed ignored, ambiguous-shares note).
14 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold in the A3 design review:
1. Composite revision = latest-index-date + index-content-hash + tracked
symbol->CIK fingerprint, so a newly added ticker forces a run instead of
being no_op'd/starved. No backfill sentinel — absence of a prior promoted
run triggers backfill; source_max_date records the processed index date.
2. Full history needs the paginated submissions shards: filings.recent caps at
1000; older accessions (reportDate/acceptanceDateTime/isXBRL) live in
filings.files[] shards (verified on Apple: recent=1000, one 1994-2015 shard).
3. Index<->Company-Facts consistency gate: they are separate SEC products that
can lag; for every tracked isXBRL index accession, confirm it exists in
Company Facts before promotion, else fail+retry (never record a null/partial
snapshot). Non-XBRL amendments skipped with a recorded reason.
4. Immutable = insert-only (ON CONFLICT DO NOTHING); a differing re-fetch is a
reported discrepancy, never a silent mutation / import_run_id replacement.
Plus deterministic, mutually-exclusive cash/debt composition (aggregate-first;
each source tag counted at most once).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All three decisions approved: fetch via EDGAR daily-index (not bulk zip),
one snapshot per accession for its primary period (comparative-only
restatements out of scope), full-history backfill on first run. Doc status
flipped to approved / ready to implement.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Design (not implementation) for phase A3, grounded in live SEC data probes.
Key findings: fp has no Q4 (derive it); fy/fp are the filing's context not each
fact's period (select by end==reportDate); SEC provides both discrete and YTD
facts (confirms stored-YTD schema); companyfacts endpoint has no ETag/
Last-Modified (conditional GET impossible); tickers are dash-form and GOOG/GOOGL
share one CIK.
Two plan deviations flagged for sign-off:
1. Fetch via the EDGAR daily-index (fetch companyfacts only for tracked issuers
that filed) rather than the multi-GB bulk zip — lighter and restores the
revision/no_op model.
2. One snapshot row per accession for its primary period (YTD-cumulative);
comparative-only restatements out of scope (only real 10-K/A updates a period).
Plus a metric tag catalog, read-time derivation rules (missing period -> null),
CIK resolution, validation gates, and SEC fair-access handling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
post-no-preference/earnings approved for private/internal ingestion under
CC BY-SA 4.0. Conditions the A2 importer must honor: preserve upstream license /
attribution / transformation notes; no public API, bulk export, or
redistribution; re-review before any public or commercial access. The stocks
repo (workstream B) is not covered and will be reviewed separately if B begins.
A0 rollout item marked done (dolt binary pin + DOLT_DATA_DIR still pending at
deploy time).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review items 3-4 on the plan doc:
- Replace remaining "diluted shares" (the share *count*) with point-in-time
shares_outstanding (dei:EntityCommonStockSharesOutstanding) across schema,
metrics catalog and market-cap note. "diluted EPS" is left as-is (correctly a
duration fact). Adds the multi-class rule: derive the issuer-wide count from
the consolidated cover-page figure OR by summing class-specific facts (GOOG +
GOOGL) — never both, to avoid double counting.
- The framework stages into a representation *outside the live tables* (in-memory
for workstream A; a file/table handle is fine if B needs it), not physical
"staging tables" — wording now matches the implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the Dolt bulk-data integration hand-off plan (authored in prior
session) and applies clarifications found in a pre-handoff review:
- step (c): fields refresh from three distinct sources, not "snapshots +
close" — earnings_surprise/next_earnings_date come from earnings_events,
not SEC facts. pe_ratio = close / TTM diluted EPS and market_cap =
issuer-wide diluted shares × close stated as separate formulas.
- fundamental_snapshots made implementable: add period_start, fiscal_year,
fiscal_period; duration facts store the filing's cumulative YTD/FY values,
balance-sheet facts store period-end values. Discrete quarters, Q4, TTM
and YoY are derived at read time (correct for non-calendar fiscal years;
amendments never freeze a stale derived quarter).
- flag Q4 derivation / fiscal-period alignment as the primary A3 risk.
- anchor "tracked universe" to ticker_universe_service + per-run CIK
resolution.
All code anchors in the doc verified accurate against the current tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Earnings backfill sourced from the public DoltHub earnings repo at a
pinned commit rather than the FMP API: reproducible for anyone re-running
the study, and it burns no request quota. 12,414 events, 98.6% of symbols
with >=8 announcements, 99.2% paired actual/estimate, no keyed duplicates.
2a earnings-gap diagnostic: INFORMATIONAL, no filter shipped. The
pre-earnings cohort's right tail was better, so the registered
avoid-earnings condition failed. Note the raw 23/266 vs 115/574 incidence
gap is largely a duration confound -- severe losses stop out fast and have
less time to span an announcement -- so it is not evidence that holding
through earnings is safe.
2b SUE: FAIL against the pre-registered +0.03 bar (unconditional IC
+0.0151 over 56 reliable windows, momentum-conditional +0.0213). Signs
stable across eras, so this is a clean null rather than an ambiguous one,
consistent with post-earnings drift having decayed in large caps.
Closes the Tier-1 arc: Task 1 dead on deep evidence, Task 2 dead here,
Task 3 complete as diagnostic. No in-sample research thread remains open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop intermediate history-depth reports, sector-residual runners/map/code hooks
(evidence stays in final reports + docs), and slim MacBook helper to ssl/earnings/
prod-book-matrix only. SSL bootstrap and archived research conclusions retained.
Four-arm results: 505 stays positive (softer on deep history); liquid breadth
destroys book under current knobs. Stop tracking 1.3GB pkl cache under reports/.cache.
Pre-register A-D (4y/2016 x 505/505+liquid) with unchanged production knobs.
Runner caches full GTL candidates then re-ranks per arm; MacBook entry via
run_tier1_macbook.sh --prod-book-matrix.
Deep masked retest failed the iron IC bar (0.027 < 0.03). Log as rejected #13 in
research README; close sector-residual and history-depth docs. Production market
residual unchanged.
Authoritative report history-depth-20260719-103315: race guard pass on deep
research.sqlite. Sector residual still short-window only (no pre-2021); fip sign
flips on broad deep sample; no production retune.
Tier-1 alpha research (local only, no production deploy):
Sector residual momentum: two-factor SPY+sector residual and sector demean signals, IC harness + A/B. Sector resid clears pre-registered bars narrowly (PROMOTE for human wire design only). Sector demean fails t vs market resid.
Earnings: earnings_events backfill (FMP bulk paid; FMP/AV per-symbol), 2a gap diagnostic report-only, 2b SUE IC (PARK; incomplete 48/506 coverage).
History-depth: pre-registered doc + runner for MacBook deep rebuild/harness.
Do not ship production residual or filters from this branch.
Log the 21:14 orphan as a snapshot-build race, rewrite the context table to
authoritative ICs only, and soften the vol-tilt warning. Add extender completion
manifest + breadth refuse guard; strip intermediate/orphaned reports; park the
thread (no book sim, no deploy).
Add lagged/tier/prod-subset/mom-conditional checks on research.sqlite.
Log: unconditional sign is a winner/bleeder tug-of-war; mom-conditional
fip stays negative and reliable; warn on high-vol tilt if universe broadens.
Add research-only snapshot extender, PIT dollar-volume mask for signal IC,
rank-only harness path, fingerprint+breadth runner, and docs. Fingerprint
reproduced IC -0.045 / t -2.91 on prod.sqlite. No production gate/schedule changes.
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts,
put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day
requalify semantics, stamp paper trades fill_mode=near_close, and migrate
stored schedule_* keys to America/New_York.
Record monotone fill-timing gradient, live [1.57,1.77] bracket, recover-flag
interpretation, gap-cap as third tail-trim, and pre-scheduler ops checklist.
Document Phase A (max-hold/vol/corr closed; next-open as decision baseline).
Add stale_close and next_open gap-cap fill modes plus a small matrix to test
whether near-close scheduling recovers overnight momentum drift.
Honor custom S/R tolerance as a transient detect, refresh levels after OHLCV
mutations without failing committed price writes, report per-ticker S/R
rebuild failures from admin cleanup, and warn in the admin UI when refresh is partial.
dennisthiessenandClaude Opus 5