Fix/sec fundamentals parity gaps #2

2026-07-24T10:57:50+02:00

dennisthiessen commented

2026-07-24 10:57:50 +02:00

No description provided.

dennisthiessen added 4 commits 2026-07-24 10:57:51 +02:00

fix(sec): correct fundamentals derivation from SEC company facts 921f3d06fb

The A5 parity report surfaced coverage gaps and wrong values that all traced
to the SEC facts parser and read-time derivation rather than to bad source
data. Fixes, each validated by replaying the production parser + derivation
against live company facts:

- Period identity is derived from period_end against the issuer's fiscal
  calendar, not SEC's fy/fp fields, which collide (two period ends on one key,
  one silently discarded) and invert (a period sorting before one that precedes
  it) often enough to break the quarter chain. Recovers BXP, CRM, CRWD, FRT,
  MTD, NTAP, PPL, STX, WDAY. Fixed labels are internal ordering keys only (not
  in any API schema), so a filer whose year ends in early January shifting by
  one is harmless.
- Revenue concept list gains RevenuesNetOfInterestExpense (banks) and the
  IncludingAssessedTax variant (REITs/consumer); EPS gains the continuing-ops
  variant (REG/FCX) and, last, basic EPS for a period tagging no diluted
  variant at all (PPL). All appended, so any issuer that already resolved keeps
  its concept.
- YTD span tolerance 20 -> 25 days, covering 4-4-5 retail calendars whose
  36-week YTD-Q3 (251-252d) previously missed by ~2 (COST, PEP, DPZ).
- Amendment resolution is per field: a partial 10-K/A (Part III only, no
  financial facts) no longer blanks the period (DVN).
- TTM diluted EPS is suppressed when a split contaminates the trailing window
  (BKNG's mixed-unit sum produced a P/E of 1.10 that clamped to a perfect
  fundamental sub-score). A post-filing split with no share-count evidence
  (KLAC) remains undetectable from this data.
- Multi-class share fallback: weighted_avg_diluted_shares is captured and used
  for market cap when the cover-page count is absent (dimensional, so missing
  from company facts for META/CMCSA/CHTR/FOXA/NWSA/LEN). Within ~0.6% of the
  true count on controls; flagged shares_estimated in the API. BRK-B has no
  weighted-average fact either and stays unavailable.

820 unit tests pass; new tests confirmed to fail against the pre-fix code.
Effect is inert until existing rows are reparsed (see reparse path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(sec): reparse path, CIK overrides, resolution validation e54f03cba6

Operational plumbing to land the parser fixes and to make silent resolution
failures visible.

- Reparse: SecFundamentalsImporter(reparse=True) restages every accession with
  the current parser and rewrites the ones that now reconstruct differently,
  writing the full column set so a row is never half old-parse. Snapshots stay
  immutable with respect to SEC; the stored row is our reconstruction, and after
  a parser fix keeping it is a stale cache, not history. run_import(force=True)
  bypasses the unchanged-revision no-op, since the staleness is on our side, not
  the source's. Exposed as scripts/reparse_fundamentals.py, dry-run by default.
- CIK overrides: sec_universe reads a {symbol: cik} pin from
  SystemSetting['sec_cik_overrides'], applied ahead of company_tickers.json, for
  when SEC maps a ticker to a successor shell with no filings (XOM -> a zero-
  filing "ExxonMobil Holdings Corp" while every 10-K/Q is under CIK 34088).
- Resolution validation: a tracked issuer resolving to a registrant with no XBRL
  filings now records no_xbrl_filings and raises a warning naming the CIKs and
  the override setting, instead of silently yielding nothing on every run.
- _diff_fields compares datetime instants, not representations: accepted_at
  round-trips naive from SQLite but tz-aware from Postgres, which otherwise made
  a reparse of identical data report every row as changed (and false-positived
  the pre-existing discrepancy warning).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs(sec): fundamentals parity root-cause findings fae621475b

Nine-pass investigation of the 2026-07-23 A5 parity report: for each coverage
gap and wrong value, the root cause traced against live SEC company facts, the
fix, and its live-data validation. Also records the decisions taken (weighted-
average share fallback, keep ASC-606 revenue basis, basic-EPS fallback, keep the
25% split-guard threshold) and what remains genuinely unfixable from this data
(KLAC post-filing split, BRK-B dimensional share count). Includes the source
parity report the findings analyse.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(sec): keep the market-cap fallback through an amendment merge 0e556d8a43

Found in review. _merge_amendments rebuilds a period from _MERGED_FIELDS +
_CARRIED_FIELDS alone, so a column in neither list is absent from the merged
row, not just stale — and callers read it with getattr(..., None), which
silently yields None. weighted_avg_diluted_shares was never added when the
market-cap fallback landed (_SNAPSHOT_COLS in the importer was updated, its
counterpart in the derivation was not).

The failure needed both of this branch's fixes at once: a multi-class issuer
with a partial amendment on its latest period (META with a Part-III-only
10-K/A) would silently lose market cap and FCF yield again.

Adds the field, a regression test for that case, and a guard test asserting
the merge/carry lists cover every SnapshotRow field, so the next column added
fails loudly rather than losing data quietly. Confirmed the guard catches the
original bug.

Also from review:
- Expose pe_caveat in the valuation payload, so a P/E suppressed by split
  contamination says why instead of looking like missing data (the caveat was
  set but never read).
- no_xbrl_filings now names both causes; the old text advised pinning a CIK
  override, which is wrong for a genuine new registrant that simply has not
  filed yet and clears itself.
- Document that fiscalYearEnd is the issuer's current calendar, so a fiscal-
  year-end change degrades old periods (fallback/newest-wins), not current ones.
- Parser-level tests for _select_weighted_avg_shares (shortest-span-wins and
  concept priority), which only had derivation-level coverage.

823 unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

dennisthiessen merged commit 77fa8b8c65 into main

2026-07-24 10:58:17 +02:00

dennisthiessen deleted branch fix/sec-fundamentals-parity-gaps

2026-07-24 10:58:17 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dennisthiessen/signal-platform#2