TradeChart shows a fixed 21-bar window. Once a trade is older than that,
its entry bar precedes the window and entryIdx goes negative, so the price
and trail paths index past the start of series/stopPath and emit NaN
coordinates -- the browser then drops both paths entirely, leaving only the
horizontal level lines. Clamp the index to the left edge and drop the entry
marker when the entry bar is outside the window; the full-width entry line
already carries it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test_rr_scanner_bug_exploration.py and test_rr_scanner_fix_check.py both
assert one invariant: the headline target is the probability-based near
level, not the far max-R:R lottery. That is already covered directly by
test_recommendation_service.py's _select_primary_target tests, which also
reach cases these never did (empty list, probability floor, activation vs
scanner floor), and end to end by test_rr_scanner_integration.py's
full-flow test -- a strict superset of their deterministic cases: three
resistance and three support levels, both directions, plus persistence
and rr_ratio consistency.
The two files also duplicated each other, and their docstrings had gone
stale: test_deterministic_long_three_levels documented a hand-computed
_compute_quality_score winner even though the assertion is about the
probability primary that supersedes it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The autouse _setup_db fixture ran create_all + drop_all for every test in
the suite, including the many that never open a session. That cycle costs
~49ms against these 22 tables; truncating them instead costs ~6ms for the
same guarantee of an empty database per test.
Build the schema once, then delete every row before each subsequent test.
No model sets sqlite_autoincrement, so SQLite reuses rowids after a full
delete and generated ids still restart at 1.
Measured over 874 tests, deterministic order: 138.6s -> 74.5s (~46%).
Verified green under pytest-randomly's default random ordering as well.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A weekday with no published index is usually just a market holiday
(~10/yr), not a fault. The WARNING still earns its place as the guard
against inferring missing from an error code, but the comment should
not claim more than it can.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fundamentals import has been dead since 2026-07-25, alerting
SecForbiddenError on form.20260725.idx with "set a real sec_user_agent
contact email". The User-Agent was never the problem.
www.sec.gov/Archives is served from an S3 bucket with no ListBucket
grant, so a MISSING key cannot answer 404 — it returns 403 with S3's
AccessDenied XML. SEC publishes a daily index for business days only, so
2026-07-25 (a Saturday) is simply absent. _get mapped every 403 to the
fatal SecForbiddenError, which made daily_index's `except
SecNotFoundError` unreachable for the exact case it was written for:
the first weekend an incremental walk crossed killed the run, and
last_processed never advanced past Friday.
Latent until activation, not a change at SEC: with no promoted run the
importer takes the backfill path and makes zero daily_index calls, so
the walk was first exercised by the first incremental run.
Verified live 2026-07-30: Sat/Sun 403 with AccessDenied XML while Fri
(51 rows) and Mon (26 rows) return 200 on the same UA; a genuine
rejection is instead the WAF's text/html "Undeclared Automated Tool"
page, served even for files that exist. So the downgrade to "missing" is
gated on all three: the /Archives/ prefix, an XML content type, and
S3's own error code. Every other 403 still alerts and stops.
Missing weekday indexes now log at WARNING — if a rejection page were
ever misread as absent, the importer must not advance past real filings
quietly.
No state to reset: _last_processed_index_date reads promoted runs only,
so the next run walks 2026-07-25..29, skipping the weekend.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fundamentals import had been failing for three days on two tracked
filings the index listed but Company Facts appeared not to have. They were
not lagging: SEC filed the XBRL of NEE's and DOW's 2026-07-24 combined
parent/subsidiary 10-Qs under the co-registrant's CIK (Florida Power &
Light, Dow Chemical), so the ticker-carrying filer's own facts file never
receives that accession. This does not self-correct - an NEE filing
misattributed the same way in 2014 is still misfiled.
Because source_max_date advances only on a promoted run, the failure was
self-perpetuating: every later run re-walked the same index day and re-hit
the same two filings.
- Recover from the co-registrant file. The daily index lists every
co-registrant of an accession, which is the only pointer to where the
facts actually landed. Rows are re-stamped to the real filer, since
parse_snapshots stamps the CIK of the payload it read.
- Guard the recovery with a share-count continuity check against the
issuer's own history, so a subsidiary's standalone facts can never be
stored as the parent's. No history, no recovery.
- Bound the blocking: a filing still unresolvable after
MISSING_XBRL_RETRY_DAYS promotes with a named unresolved_filing warning
instead of wedging every later import.
- Name the offending filings in the alert and record them in
validation_json, separating not_in_companyfacts from not_in_submissions.
The gate previously reported a count and discarded the accessions.
Verified against live SEC data: both filings recover with the correct CIK
and the guard rejects a mismatched reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A conditional clause in the summary sentence carried an f prefix with no
placeholders, failing `ruff check app/` and blocking the deploy. Literal only;
the rendered text is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The v3 cutover run scored 2/4 corrections warned against v2's 3/4, which reads
like a regression and is not one. Only 4 of the 11 detected corrections fall in
the holdout, so recall is one event from a different headline -- and the event
that flips is decided by threshold placement, not by what the score saw. "v3
without the credit sensor" catches 2025-02-21 at a *higher* threshold (35.5)
than shipped v3 misses it at (32.3), because the alarm rule needs a rising edge
and a lower threshold can fire outside the horizon then never reset below.
Two caveats are now computed and surfaced rather than left for the reader to
infer:
- Holdout event count against MIN_EVENTS_FOR_CONFIDENCE. The summary sentence
states how many of the detected corrections actually fall in the test period.
- Warning-sensor coverage across the split. The score renormalises over what is
available, so a training window predating a sensor's history freezes the
threshold on a different construct than the holdout is measured against. At
the cutover that is 39% of training sessions with all three sensors versus
100% of the test period, credit history beginning 2023-07-25.
Restricting the threshold to sensor-matched training sessions was tested and
rejected: those sessions are a calm recent stretch, so the threshold falls from
32.3 to 22.5 and false alarms rise from 3.3 to 8.6/yr. It swaps a coverage bias
for a regime-selection bias. The report states its limits instead.
_warning_series now returns per-session sensor counts alongside the scores.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The LLM-sourced capex/earnings observations carried 12+8 of 100 Warning points,
so both pegged at 100 produced a Warning of 20.0 -- below the event study's 25.3
alarm threshold and still inside the "stable" band. The reading was
arithmetically incapable of changing anything on screen, which is why refreshing
it appeared to do nothing. They are now a qualitative overlay reported beside
the scores rather than diluted into them.
Calibrated against the 408 v2 sessions to 2026-07-24, reproduced offline from
Alpaca + FRED; the harness matched the stored prod distribution exactly before
any parameter was changed.
State:
- P3 used dd_pct * 5, reaching 100 at a 20% drawdown -- the 90th percentile of
the observed distribution -- so 39/408 sessions sat at exactly 100 with no
resolution left during the part of a selloff that matters most. Replaced with
anchored breakpoints keeping headroom past the observed 36% maximum, blended
2:1 like P1/P2 instead of max(). P3's realized share of State falls from 65%
to 40%, matching its nominal weight.
- Credit level is now anchors-only. ICE capped FRED's BAMLH0A0HYM2 at a rolling
3-year window in April 2026, silently turning the 10-year percentile leg into
a 3-year one that scored 20 points of stress at an OAS of 3.5 -- the level its
own anchors call "mild". The anchors already encode the long-run distribution.
Warning:
- Added HY OAS 20-session widening (25%). The level is pinned at zero below the
3.5 anchor; its rate of change is not.
- Divergence tapers to a 0.35 floor instead of a hard price_ret >= 0 gate, which
zeroed the sensor through every decline: on 2026-07-24 the basket shed 10
points of participation in 20 sessions and Warning printed exactly 0.
- The event study and the live monitor now share one sensor definition, so they
cannot silently drift apart.
Bands are per axis (State 20/50/80, Warning 20/40/60) with quadrant dividers at
50/40; v2 Warning never exceeded 64.9 against a shared 60, leaving that half of
the quadrant unreachable. Realized shares: State 73/15/8/3%, Warning 69/20/8/3%.
Snapshots now record credit_history_days and vix_history_days -- the percentile
defect went unnoticed for months because nothing asserted the window the code
claimed.
Cutover: the first run rebuilds 400 sessions automatically; the Event Study job
must be re-run, as its cached report self-invalidates on the methodology check.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parity gate has been exercised: nine-pass investigation, fixes, two prod
reparses, and an after-report at 504/511 candidate coverage with agreement
unchanged. What remains of workstream A is the activation itself (step c, the
post-approval fundamental_data refresh — not yet implemented) and A6
decommissioning, both now specified in a handoff section with the caveats the
next implementer must carry (KLAC splits, BRK-B, FITB, the 25% guard, CIK
overrides, reparse-after-parser-changes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the parity investigation: prod reparse runs 6+7 reconciled against the
dry run, the collision check that caught the BEN regression (and its residue --
36 historical 53-week-drift rows, deliberately left), and the 2026-07-24 parity
report diffed against the 2026-07-23 baseline. Candidate coverage 482 -> 504 of
511 with the gap fully explained (PSKY/Q new registrants, FITB guard-tripped
with no revenue), agreement unchanged where both sides exist, and the remaining
deltas are documented definition differences. Includes the after-report and a
correction to the seventh pass (FITB loses its score, not just one input).
Recommends approving the A5 cutover with KLAC as the one carried caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Regression found by the post-reparse collision check. _period_identity trusted
submissions.fiscalYearEnd, which is not reliable: Franklin Resources (BEN)
declares 1231 while every one of its 10-Ks ends 09-30.
The effect was data loss, not just a bad label. BEN's real fiscal Q1 (Dec 31)
sat 0 days from the claimed year end, matching no quarter band, so it fell back
to SEC's fy/fp; its fiscal Q2 (Mar 31) computed 275 days out and was labelled
Q1. Both landed on the same key, the collision discarded one, and BEN lost TTM
EPS and revenue growth entirely — values it had before this branch.
A 10-K's reportDate IS the fiscal year end by definition, so resolve_fiscal_
year_end() now prefers the issuer's most recent annual filing and treats the
declared value as a fallback for issuers with no 10-K in the set.
Scanned the full tracked universe: 2 of 506 issuers declare a year end more
than 21 days from their own 10-K — BEN (91d, broken) and DELL (29d, mislabelled
but functionally correct). Both now derive correctly and match the legacy
provider: BEN revenue growth 3.8243 vs 3.82, DELL 38.5735 vs 38.57. Controls
(AAPL, COST, PEP, DPZ, IRM, JPM, CRM, STX, AVY) byte-identical.
826 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found in review. _merge_amendments rebuilds a period from _MERGED_FIELDS +
_CARRIED_FIELDS alone, so a column in neither list is absent from the merged
row, not just stale — and callers read it with getattr(..., None), which
silently yields None. weighted_avg_diluted_shares was never added when the
market-cap fallback landed (_SNAPSHOT_COLS in the importer was updated, its
counterpart in the derivation was not).
The failure needed both of this branch's fixes at once: a multi-class issuer
with a partial amendment on its latest period (META with a Part-III-only
10-K/A) would silently lose market cap and FCF yield again.
Adds the field, a regression test for that case, and a guard test asserting
the merge/carry lists cover every SnapshotRow field, so the next column added
fails loudly rather than losing data quietly. Confirmed the guard catches the
original bug.
Also from review:
- Expose pe_caveat in the valuation payload, so a P/E suppressed by split
contamination says why instead of looking like missing data (the caveat was
set but never read).
- no_xbrl_filings now names both causes; the old text advised pinning a CIK
override, which is wrong for a genuine new registrant that simply has not
filed yet and clears itself.
- Document that fiscalYearEnd is the issuer's current calendar, so a fiscal-
year-end change degrades old periods (fallback/newest-wins), not current ones.
- Parser-level tests for _select_weighted_avg_shares (shortest-span-wins and
concept priority), which only had derivation-level coverage.
823 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Nine-pass investigation of the 2026-07-23 A5 parity report: for each coverage
gap and wrong value, the root cause traced against live SEC company facts, the
fix, and its live-data validation. Also records the decisions taken (weighted-
average share fallback, keep ASC-606 revenue basis, basic-EPS fallback, keep the
25% split-guard threshold) and what remains genuinely unfixable from this data
(KLAC post-filing split, BRK-B dimensional share count). Includes the source
parity report the findings analyse.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Operational plumbing to land the parser fixes and to make silent resolution
failures visible.
- Reparse: SecFundamentalsImporter(reparse=True) restages every accession with
the current parser and rewrites the ones that now reconstruct differently,
writing the full column set so a row is never half old-parse. Snapshots stay
immutable with respect to SEC; the stored row is our reconstruction, and after
a parser fix keeping it is a stale cache, not history. run_import(force=True)
bypasses the unchanged-revision no-op, since the staleness is on our side, not
the source's. Exposed as scripts/reparse_fundamentals.py, dry-run by default.
- CIK overrides: sec_universe reads a {symbol: cik} pin from
SystemSetting['sec_cik_overrides'], applied ahead of company_tickers.json, for
when SEC maps a ticker to a successor shell with no filings (XOM -> a zero-
filing "ExxonMobil Holdings Corp" while every 10-K/Q is under CIK 34088).
- Resolution validation: a tracked issuer resolving to a registrant with no XBRL
filings now records no_xbrl_filings and raises a warning naming the CIKs and
the override setting, instead of silently yielding nothing on every run.
- _diff_fields compares datetime instants, not representations: accepted_at
round-trips naive from SQLite but tz-aware from Postgres, which otherwise made
a reparse of identical data report every row as changed (and false-positived
the pre-existing discrepancy warning).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The A5 parity report surfaced coverage gaps and wrong values that all traced
to the SEC facts parser and read-time derivation rather than to bad source
data. Fixes, each validated by replaying the production parser + derivation
against live company facts:
- Period identity is derived from period_end against the issuer's fiscal
calendar, not SEC's fy/fp fields, which collide (two period ends on one key,
one silently discarded) and invert (a period sorting before one that precedes
it) often enough to break the quarter chain. Recovers BXP, CRM, CRWD, FRT,
MTD, NTAP, PPL, STX, WDAY. Fixed labels are internal ordering keys only (not
in any API schema), so a filer whose year ends in early January shifting by
one is harmless.
- Revenue concept list gains RevenuesNetOfInterestExpense (banks) and the
IncludingAssessedTax variant (REITs/consumer); EPS gains the continuing-ops
variant (REG/FCX) and, last, basic EPS for a period tagging no diluted
variant at all (PPL). All appended, so any issuer that already resolved keeps
its concept.
- YTD span tolerance 20 -> 25 days, covering 4-4-5 retail calendars whose
36-week YTD-Q3 (251-252d) previously missed by ~2 (COST, PEP, DPZ).
- Amendment resolution is per field: a partial 10-K/A (Part III only, no
financial facts) no longer blanks the period (DVN).
- TTM diluted EPS is suppressed when a split contaminates the trailing window
(BKNG's mixed-unit sum produced a P/E of 1.10 that clamped to a perfect
fundamental sub-score). A post-filing split with no share-count evidence
(KLAC) remains undetectable from this data.
- Multi-class share fallback: weighted_avg_diluted_shares is captured and used
for market cap when the cover-page count is absent (dimensional, so missing
from company facts for META/CMCSA/CHTR/FOXA/NWSA/LEN). Within ~0.6% of the
true count on controls; flagged shares_estimated in the API. BRK-B has no
weighted-average fact either and stays unavailable.
820 unit tests pass; new tests confirmed to fail against the pre-fix code.
Effect is inert until existing rows are reparsed (see reparse path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dennisthiessen
Dennis Thiessen