fix(sec): correct fundamentals derivation from SEC company facts

The A5 parity report surfaced coverage gaps and wrong values that all traced
to the SEC facts parser and read-time derivation rather than to bad source
data. Fixes, each validated by replaying the production parser + derivation
against live company facts:

- Period identity is derived from period_end against the issuer's fiscal
  calendar, not SEC's fy/fp fields, which collide (two period ends on one key,
  one silently discarded) and invert (a period sorting before one that precedes
  it) often enough to break the quarter chain. Recovers BXP, CRM, CRWD, FRT,
  MTD, NTAP, PPL, STX, WDAY. Fixed labels are internal ordering keys only (not
  in any API schema), so a filer whose year ends in early January shifting by
  one is harmless.
- Revenue concept list gains RevenuesNetOfInterestExpense (banks) and the
  IncludingAssessedTax variant (REITs/consumer); EPS gains the continuing-ops
  variant (REG/FCX) and, last, basic EPS for a period tagging no diluted
  variant at all (PPL). All appended, so any issuer that already resolved keeps
  its concept.
- YTD span tolerance 20 -> 25 days, covering 4-4-5 retail calendars whose
  36-week YTD-Q3 (251-252d) previously missed by ~2 (COST, PEP, DPZ).
- Amendment resolution is per field: a partial 10-K/A (Part III only, no
  financial facts) no longer blanks the period (DVN).
- TTM diluted EPS is suppressed when a split contaminates the trailing window
  (BKNG's mixed-unit sum produced a P/E of 1.10 that clamped to a perfect
  fundamental sub-score). A post-filing split with no share-count evidence
  (KLAC) remains undetectable from this data.
- Multi-class share fallback: weighted_avg_diluted_shares is captured and used
  for market cap when the cover-page count is absent (dimensional, so missing
  from company facts for META/CMCSA/CHTR/FOXA/NWSA/LEN). Within ~0.6% of the
  true count on controls; flagged shares_estimated in the API. BRK-B has no
  weighted-average fact either and stays unavailable.

820 unit tests pass; new tests confirmed to fail against the pre-fix code.
Effect is inert until existing rows are reparsed (see reparse path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 10:23:51 +02:00
co-authored by Claude Opus 4.8
parent 259001e419
commit 921f3d06fb
7 changed files with 626 additions and 21 deletions
+10 -2
View File
@@ -12,8 +12,10 @@ class FundamentalSnapshot(Base):
Keyed by issuer (CIK), not ticker — multi-class issuers (GOOG/GOOGL) share
one CIK and one set of fundamentals; the ``tickers.cik`` column is the only
join point. Amendments are retained: every accession is a distinct immutable
row, and readers pick the newest valid ``accepted_at`` per
(cik, fiscal_year, fiscal_period) at read time — no flags, no mutation.
row, and readers resolve (cik, fiscal_year, fiscal_period) at read time by
taking the newest ``accepted_at`` **per field**, falling back to the newest
accession that actually reports one — a partial amendment (a 10-K/A adding
Part III reports no financial facts) must not blank the period — no flags, no mutation.
**Facts are stored as the filing reports them, never as derived quarters.**
Duration facts (revenue, net_income, operating_income, diluted_eps, cfo,
@@ -70,6 +72,12 @@ class FundamentalSnapshot(Base):
# reported "as of" its own date, which can differ from period_end — store it
# so market cap uses the right point-in-time count.
shares_outstanding_date: Mapped[date | None] = mapped_column(Date, nullable=True)
# Weighted-average diluted count for the filing's most recent quarter — the
# market-cap fallback when the cover-page count is absent, which it always is
# for multi-class issuers (per-class facts are dimensional, and companyfacts
# is not). An average is not cumulative, so unlike the duration facts above
# this is NOT a YTD value: it is the shortest-span fact ending at period_end.
weighted_avg_diluted_shares: Mapped[float | None] = mapped_column(Float, nullable=True)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("data_import_runs.id", ondelete="SET NULL"), nullable=True