signal-platform/reports/fundamentals-parity-20260723-findings.md

A5 parity report — root-cause findings

Investigation of fundamentals-parity-20260723T210658161480Z.json (511 tickers, generated 2026-07-23). Method: replayed the production parser (sec_facts_parser.parse_snapshots) and derivation (fundamentals_derivation.derive) against live SEC companyfacts, using the importer's own _filing_meta and backfill accession set, then cross-checked prices against IBKR. No database was available locally, so every conclusion below is reproduced from source data rather than read out of prod.

Repro script: scratchpad/diag.py (--history replays the full backfill path). Every claim below was verified on the named issuer. Names that were not individually inspected are listed as unclassified — an earlier draft of this document guessed their cause from fiscal-year-end dates and was wrong for most of them, so the guessing is not repeated here.

Verdict

Where both sides have a value the candidate data is good: P/E spearman 0.968, revenue growth agreeing to 4 decimals for most names, score spearman 0.825. Every defect found is a parser/derivation bug or an identity problem — not a data quality problem with SEC or Dolt. The largest cluster is period identity, which is exactly the risk A3 flagged as primary.

1. P/E outliers — splits corrupt TTM EPS, and the split guard doesn't cover it

derive() sets result.ttm_diluted_eps at fundamentals_derivation.py:88 and only calls _guard_split_sensitive_metrics() at line 105, which annotates result.metrics (the MetricSeries objects). ttm_diluted_eps is a bare scalar and is never guarded. fundamentals_parity_service._pe() consumes it directly.

The cleanest evidence that the candidate side is the broken one: reconcile each P/E against the report's own price. Legacy comes out sane in both cases, candidate does not.

BKNG — guard fired, nobody listened. Share count jumps 31.7M → 774.9M between the FY2025 10-K and the 2026 Q1 10-Q (≈25:1 split). TTM EPS therefore sums three pre-split quarters (27.31 + 84.01 + 44.18 = 155.50) plus one post-split quarter (1.36) = 156.86 — mixed units. Live price $172.83 matches the price the report implies exactly (1.1018 × 156.86 = 172.83), so the price is correct and current. Against that price, legacy's P/E of 22.44 implies EPS ≈ 7.70 — a coherent post-split number, versus the candidate's 156.86. The derivation did raise "Not comparable: share count changed at least 25%; possible split or corporate action." on eps_growth_yoy and share_count_change_yoy — P/E never sees it.

KLAC — the guard cannot fire. The split post-dates the most recent 10-Q (period end 2026-03-31), so no snapshot shows any share-count change (share_count_change_yoy = −1.2%). TTM EPS 35.31 is internally consistent and entirely pre-split; the price ($223.30 live, ≈218.7 in the report) is post-split. Reconciling: legacy P/E 60.21 against the report price implies EPS ≈ 3.63 ≈ 35.31/9.7 — i.e. legacy is consistent with a ~10:1 split and correct, and the candidate is off by exactly the split factor. (IBKR's split-adjusted open_52w of 89.36 corroborates 10:1.)

This is the important case: a split after the latest filing is undetectable from snapshots alone. No share-count test can catch it. Reconciliation needs a corporate actions source or a price-vs-EPS plausibility check.

COF — not a bug, a definition difference. Shares 383M → 639M in 2025 Q2 is the Discover acquisition. TTM GAAP EPS is genuinely $3.92 because the merger-charge quarter (−10.19) sits in the window. Candidate P/E 51.01 is arithmetically correct on a GAAP TTM basis; legacy's 11.61 is an adjusted/forward convention. Disclose, don't fix. Note this single row drives the report's largest change (rank 1 → 456).

2. Bank revenue growth — concept-mapping gap (confirmed)

sec_facts_parser._DURATION_USD["revenue"] is:

RevenueFromContractWithCustomerExcludingAssessedTax, Revenues, SalesRevenueNet

Banks tag RevenuesNetOfInterestExpense in their 10-Qs:

filer	2026 Q1 10-Q tags present	parsed revenue
JPM	RevenuesNetOfInterestExpense 49,836M, NoninterestIncome, InterestIncomeExpenseNet	null
GS	RevenuesNetOfInterestExpense 17,227M, InterestAndDividendIncomeOperating, …	null
WFC	RevenuesNetOfInterestExpense 21,436M, …	null

JPM's FY2025 10-K also tags Revenues (182,447M — identical value), so only the annual row populates; GS never tags Revenues at all. Revenue growth needs five consecutive quarterly values, so it is null for the whole cluster (JPM, GS, MS, WFC, TFC, MTB, FITB, RF, SYF, BNY, BX, BLK, SPGI, ACGL, CBOE).

A second variant of the same gap: ARE and KHC tag RevenueFromContractWithCustomer**Including**AssessedTax — also absent from the list — so revenue is null on every row while EPS parses fine.

Fix: add RevenuesNetOfInterestExpense and the IncludingAssessedTax variant.

Latent risk while you're in there: RevenueFromContractWithCustomerExcludingAssessedTax is first and "first present wins". For a bank that tags it, it captures only ASC-606 fee revenue, not total revenue — a silently understated number rather than a null, which is worse. DVN shows the same hazard from the other side: its 2026 Q1 tags both RevenueFromContractWithCustomerExcludingAssessedTax (4,508M) and Revenues (3,807M), an 18% difference decided purely by list order.

3. Period identity — the largest cluster, three confirmed mechanisms

3a. Fiscal-year label collisions (CRM, FRT, STX)

_fiscal_context majority-votes SEC's fy/fp fields, and _select_latest_per_period keys on (fiscal_year, fiscal_period). When SEC's labels disagree with the calendar, two distinct periods collide on one key and one is silently discarded:

• CRM — two rows keyed 2025 FY, ending 2025-01-31 and 2026-01-31.
• FRT — two rows keyed 2024 FY, ending 2024-12-31 and 2025-12-31.
• STX — the year ending 2025-06-27 is labelled 2027 FY, so it sorts after 2026 Q3 (period end 2026-04-03) and is taken as the latest quarter.

The survivor's period_end then contradicts the fiscal ordering, Q4 derivation and the consecutive-quarter chain break, and TTM EPS + YoY both go null.

FRT is a calendar-year (Dec) filer, so this is not limited to non-calendar fiscal years — the earlier assumption that it was is wrong. Any filer SEC labels inconsistently is exposed.

3b. Amendment selection blanks a period (DVN)

DVN has two rows for 2025 FY (both ending 2025-12-31): the 10-K with complete financials, and a 10-K/A carrying no financial facts at the report date (rev=None eps=None). _select_latest_per_period takes the newest accepted_at, so the empty amendment wins and the FY2025 row becomes all-null, breaking the chain.

This is the most dangerous of the three: it is not exotic. Any issuer filing a 10-K/A — including routine Part III amendments that restate nothing — silently loses that period. The rule needs to prefer the newest accession that actually carries the fact, per field, rather than the newest accession outright.

3c. 4-4-5 retail calendar — Q3 only, misses by ~2 days (COST, PEP)

_EXPECTED_YTD_DAYS["Q3"] = 273 with _YTD_TOLERANCE_DAYS = 20 accepts 253–293 days. A 12/12/12/16-week filer's YTD-Q3 is 36 weeks ≈ 251–252 days — just under the floor.

Confirmed, facts present and rejected:

• COST 2026 Q3: RevenueFromContractWithCustomerExcludingAssessedTax span=251d val=207,431M, EarningsPerShareDiluted span=251d val=14.01 → row stored with rev=None eps=None start=None. Same for 2025 Q3 and 2024 Q3.
• PEP: every Q3 row is rev=None eps=None; Q1/Q2/FY all populate.

Q1 (83d vs 91±20), Q2 (167d vs 182±20) and FY (363–364d vs 365±20) all pass — only Q3 fails, every year. The code comment claims the tolerance "covers 52/53-week fiscal calendars"; it does not cover 4-4-5 ones.

Note this does not apply to ordinary 13-week 52/53-week filers (STX's Q3 YTD is 279d and passes) — their failures are 3a, not this.

Fix: widen the Q3 tolerance to ~25 days, or derive the expected span from the filer's own fiscal calendar rather than a fixed 91/182/273.

4. CIK identity (XOM)

SEC's company_tickers.json now maps XOM → CIK 2115436 "ExxonMobil Holdings Corp", which has 0 filings. All 26 XBRL 10-K/10-Qs sit under the old CIK 34088 "EXXON MOBIL CORP". XOM therefore has no snapshots at all, and nothing in the pipeline notices that a tracked issuer resolved to a CIK with zero filings.

PSKY (5 filings) and Q (3 filings) are genuinely new registrants — expected, not a bug.

Status of the 25 names that lose their fundamental score

Production requires ≥2 metrics (scoring_service.py:502), the same rule the parity harness uses, so these genuinely drop the fundamental dimension and the composite renormalises over the remaining four.

cause (confirmed on the named issuer)	names
FY label collision (3a)	CRM, FRT, STX
4-4-5 Q3 span (3c)	COST, PEP
revenue concept gap (§2)	ARE, KHC
amendment blanks period (3b)	DVN
CIK identity (§4)	XOM
new registrant — expected	PSKY, Q
not yet classified	AZO, BXP, CRWD, FCX, HAL, MOS, MTD, NTAP, PPL, REG, SJM, SWKS, WDAY

13 of 25 confirmed. The unclassified 13 have not been inspected and should not be assumed to share a cause — the confirmed set already spans five distinct mechanisms.

Recommended order of work

1. Amendment selection (3b) — highest blast radius, affects any 10-K/A filer, and the current rule is wrong in principle rather than at the margin.
2. Revenue concept list (§2) — add RevenuesNetOfInterestExpense and IncludingAssessedTax; audit the ASC-606-first priority, which can understate rather than null.
3. Q3 YTD span tolerance (3c) — effectively one line.
4. XOM CIK remap (§4) — plus a validation that flags any tracked ticker resolving to a CIK with zero XBRL filings.
5. Split safety for ttm_diluted_eps (§1) — propagate the existing guard to the scalar, and add a price-vs-EPS plausibility check for splits that post-date the last filing.
6. Fiscal-period identity (3a) — the deepest fix; consider keying period identity on period_end rather than SEC's fy/fp.

Re-run the parity report after these and re-classify the remaining 13 before making a cutover decision. The current report should not be approved as-is: its coverage gaps are artifacts of the above, not real absences in the source data.

---

Fixes applied (items 1–3)

#	change	file	effective
1	amendment resolution is now per field — newest accession that actually reports a fact wins; only rows sharing the newest period_end are merged, so a mislabelled filing is never blended in	fundamentals_derivation.py	read time — immediately
2	appended RevenueFromContractWithCustomerIncludingAssessedTax and RevenuesNetOfInterestExpense to the revenue concept list	sec_facts_parser.py	parse time — needs reparse
3	YTD span tolerance 20 → 25 days, covering 4-4-5 retail calendars	sec_facts_parser.py	parse time — needs reparse

Fix 2 is deliberately additive: the new tags go at the end of the priority list, so every issuer that already resolved keeps the same concept and only issuers that resolved to nothing gain a value. A regression test pins that ordering.

Tests: 7 added across test_sec_facts_parser.py and test_fundamentals_derivation.py. The 5 behaviour-changing ones were confirmed to fail against the pre-fix code; the other 2 are invariance guards that pass both ways. Full unit suite: 795 passed.

Validation against live SEC data

Re-ran the parser + derivation on live companyfacts. Every targeted name recovers, and the recovered values independently agree with the legacy provider:

name	cause	revenue growth before → after	legacy	TTM EPS after
COST	4-4-5 Q3	null → 9.2311	9.23	19.88
PEP	4-4-5 Q3	null → 5.6197	5.62	7.63
KHC	concept (Including)	null → −1.7457	−1.75	−4.85
DVN	partial 10-K/A	null → 0.0956	−1.51	3.59
ARE	concept (Including)	null → −5.3462	−9.53	−6.27
JPM	concept (bank)	null → 3.3388	108.98	20.89
GS	concept (bank)	null → 11.1974	6.67	54.75
WFC	concept (bank)	null → 4.1847	72.75	6.47

COST/PEP/KHC matching legacy to two decimals is strong evidence the parse is now correct. The banks are the opposite case and worth noting for the cutover argument: legacy's JPM 109% and WFC 73% "revenue growth" are not plausible for a bank, while the SEC-derived 3.3% and 4.2% are — here the candidate is better than what it would replace. DVN and ARE still differ from legacy; DVN is the Revenues vs ASC-606 ambiguity noted in §2 and is the one open definition question.

Regression check on names that were already correct — IRM, KLAC, BKNG — reproduces their previous values exactly (IRM 15.6375, KLAC 13.3895, BKNG 14.9506; TTM EPS unchanged). Nothing that worked before moved.

Concept consistency across the bank chains (checked, clean)

Because Revenues still outranks RevenuesNetOfInterestExpense, a filer could resolve the FY row to one concept and its quarters to the other — which would make Q4 = YTD(FY) − YTD(Q3) a subtraction across two definitions, and poison every TTM window containing it. Checked all 15 recovered banks (scratchpad/concept_check.py):

• 14 resolve a single concept across the whole chain (GS, WFC, MS, TFC, MTB, FITB, RF, SYF, BNY, BX, BLK, SPGI, ACGL, CBOE).
• JPM is mixed but benign: its FY2025 row tags both, at an identical 182,447M, so Q4 subtracts like for like. No filer showed the two tags disagreeing where both appear.

So the "candidate beats legacy for banks" claim above is safe as stated. Residual risk: a future filer whose two tags differ would fail silently. Cheapest hardening is to treat the two as one logical revenue concept rather than separate priority entries; the detector script above turns this into a one-command check.

Operational note — the parser fixes need a deliberate reparse

sec_fundamentals_importer.promote() treats snapshots as immutable per accession: a re-run skips any accession already stored and records a snapshot_discrepancy SystemEvent instead. So fixes 2 and 3 change nothing for rows already in the database — recovering COST/PEP/JPM/etc. requires deleting the affected snapshot rows and re-importing, or adding an explicit reparse path. Usefully, the discrepancy warning names exactly which stored accessions now reconstruct differently, so a dry run over existing data will enumerate the blast radius before anything is rewritten.

---

Second pass — all 25 lost names now classified

Re-ran diag.py --history over every previously unclassified name, with fixes 1–3 in place. (One name, DPZ, had been dropped from the unclassified list when this document was rewritten; it is included here.)

11 of 25 recover

COST, PEP, KHC, DVN, ARE, AZO, MOS, SJM, SWKS, HAL, DPZ — and again the recovered revenue growth matches the legacy provider to two decimals on every one:

name	candidate	legacy		name	candidate	legacy
AZO	5.7405	5.74		SWKS	2.3303	2.33
MOS	12.3388	12.34		HAL	−1.7201	−1.72
SJM	3.7222	3.72		DPZ	5.1573	5.16

Precisely: all 11 clear the ≥2-metric floor and regain a fundamental score. P/E returns for AZO, MOS, SWKS, DPZ, COST, PEP and DVN. ARE, KHC and SJM have genuinely negative TTM EPS, so their P/E stays null correctly. HAL's TTM EPS is still null and the cause is not yet established — it scores on revenue growth + surprise. Loose end.

14 remain, in four causes

cause	names	count
fiscal-year label collisions (§3a)	CRM, FRT, STX, BXP, CRWD, MTD, NTAP, WDAY, PPL	9
EPS concept gap (new — §5 below)	FCX, REG	2
CIK identity (§4)	XOM	1
new registrant — expected, not a bug	PSKY, Q	2

The label bug is now the dominant cause by a wide margin, and it is more varied than first described — it is not only colliding fiscal_year values:

• BXP — a 10-Q for period end 2026-03-31 is labelled 2026 **FY**. The fiscal period is wrong, not just the year, so _select_ytd then measures the 90-day fact against the 365-day FY expectation and rejects it too.
• NTAP, WDAY, MTD, CRWD — two different period-ends colliding on one key (the pattern first seen on CRM/FRT).
• PPL — the worst observed: four rows keyed 2022 Q3, with period ends 2022-09-30, 2023-03-31, 2023-06-30 and 2023-09-30.

5. New cause — EPS concept coverage

_EPS_CONCEPTS = ["EarningsPerShareDiluted"] is the only tag read. Confirmed by listing every USD/shares duration concept in the relevant filings:

• REG tags only IncomeLossFromContinuingOperationsPerDilutedShare, on every filing — EPS is null everywhere, so no TTM EPS and no P/E, ever.
• FCX is the nastier shape: its 10-Qs tag EarningsPerShareDiluted, but its 10-K tags only IncomeLossFromContinuingOperationsPerDilutedShare. The FY row loses EPS, so Q4 = YTD(FY) − YTD(Q3) is undefined and TTM dies — an issuer that switches concept by form type looks like partial data rather than a mapping gap.

Fix: append IncomeLossFromContinuingOperationsPerDilutedShare to _EPS_CONCEPTS. Same additive shape as the revenue fix; recovers REG outright and FCX's FY row.

Related decision, not a fix: PPL's 2026 Q1 tags no diluted variant at all — only EarningsPerShareBasic and IncomeLossFromContinuingOperationsPerBasicShare. Adding the diluted continuing-ops tag does not help it. Falling back to basic EPS is a definition change (basic ≠ diluted) and should be an explicit call, not a silent one.

---

Third pass — fixes #2 and #3 applied

#	change	file	effective
2a	_guard_split_sensitive_metrics() now returns whether the latest period is split-suspect, and derive() nulls ttm_diluted_eps (setting ttm_diluted_eps_caveat) when it is	fundamentals_derivation.py	read time — immediately
3	appended IncomeLossFromContinuingOperationsPerDilutedShare to _EPS_CONCEPTS	sec_facts_parser.py	parse time — needs reparse

5 tests added; the 3 behaviour-changing ones confirmed to fail against pre-fix code, 2 are invariance guards. Full unit suite: 800 passed.

Validated on live data

name	before	after
FCX	TTM EPS null	1.89	recovered
REG	TTM EPS null	2.92	recovered
BKNG	TTM EPS 156.86 → P/E 1.10	null + caveat	false perfect score removed
COF	TTM EPS 3.92 → P/E 51.01	null + caveat	see side effect below
KLAC	TTM EPS 35.31 → P/E 6.19	unchanged	still wrong — 2b not fixed
IRM, COST	—	unchanged	no regression

FCX and REG regain a fundamental score (EPS + surprise clears the ≥2 floor). Their revenue growth is still null — both are also blocked by the label bug (REG has a mislabelled duplicate 2024 Q2; FCX is missing its 2024 FY row entirely).

Threshold decision — RESOLVED: keep 25%

Measured against the database (scratchpad/share_change_check.sql): 15 of 467 comparable issuers (3.2%) trip the ≥25% guard on their latest period.

band	names	cause
≥200%	BKNG 23.8×, ORLY 14.5×, NFLX 9.8×, NOW 5.0×, TPL 3.0×	forward splits
50–142%	CHTR (query artifact), AMCR −68% (1-for-5 reverse split), WAT, COF	split + stock-funded M&A
25–47%	OMC, BG, HBAN, FITB, COHR, RKLB	stock-funded M&A, ordinary dilution

Keep the threshold at 25%, for three reasons — the first of which is empirical and came out of checking AMCR:

1. A real split trips at only 68%. AMCR's 1-for-5 reverse consolidation (2,308,359,941 → 462,045,690 shares, ratio 4.996, between the Nov 2025 and Feb 2026 10-Qs) shows up as −68%. Raising the bar to 100% to spare the M&A cases would have let a genuine split straight through. Split magnitude and M&A magnitude overlap in practice, not just in theory.
2. The cost is milder than first described. Losing P/E leaves revenue growth + earnings surprise = 2 metrics, which still clears the ≥2 floor. Affected issuers keep a fundamental score; they lose one of three inputs.
3. The severities are asymmetric. A missed split yields a P/E off by 10–25×, clamping to a perfect 100 sub-score. Over-nulling yields a missing input the scorer already handles by renormalising.

Honest caveat: the guard is blunt — it detects that a share base moved, not how much damage resulted. AMCR's pre-fix P/E was 28.61 against legacy's 29.47, i.e. only ~10-15% off, because most of its YTD figures had already been restated on the post-split basis. So the guard sometimes removes a roughly-usable number. That is the accepted price of a rule that cannot measure the split factor.

Two data notes from the same check:

• CHTR is a query artifact, not a guard trip. The SQL picks the newest period with a share count, while derive() picks the newest period and then reads shares off it. CHTR's recent snapshots have a null shares_outstanding, so the query fell back to the 2016 Time Warner merger. In the real path its change is None and the guard never fires — so the true count is ~14. But it also means CHTR has no recent share count, which breaks its market cap in the API — a separate small bug.
• AMCR was suspected of being a shares_outstanding parsing bug and is not. It is a real corporate action, correctly detected. abs() in the guard already handles reverse splits.

Side effect — COF

The guard fires on any ≥25% YoY share-count move, not only splits. COF's 383M → 639M jump is the Discover acquisition, so it now nulls too and loses the P/E of 51.01 that this document previously called "arithmetically correct on a GAAP TTM basis".

I think nulling is right: TTM EPS sums four quarters whose per-share figures use different weighted-average denominators, and across a 67% share change that sum is not a meaningful per-share number regardless of whether the cause was a split or an acquisition. It follows the formula without being a valid result.

But the cost is real and worth stating plainly: any issuer doing a large stock-funded acquisition loses its P/E for four quarters. That frequency has not been measured — it needs a count of |share_count_change_yoy| ≥ 25% across the universe, which needs the database. If it turns out to be common, the alternative is a higher or split-shaped threshold, at the cost of letting more BKNG-class errors through.

2b is genuinely unfixed

KLAC's split post-dates its most recent 10-Q, so no snapshot carries any share-count evidence and no guard built on share counts can fire. Its P/E is still 6.19 — the true P/E divided by the split factor. I did not ship a heuristic for this: the obvious one, flagging implausibly low P/Es, would misfire on genuinely cheap names — CHTR (3.42) and CMCSA (4.30) sit below KLAC's corrupted 6.19 in this very report. Detecting it needs an actual corporate-actions source, or a price-vs-share-count reconciliation against an external market-cap reference.

---

Fourth pass — the reparse path

Snapshots are immutable per accession, so the parser fixes never reached stored rows. promote() skipped them and logged a discrepancy. Reparse is the deliberate exception: immutability protects SEC's record, but the stored row is our reconstruction — after a parser fix, keeping it is preserving a stale cache, not preserving history.

change	file
run_import(..., force=True) bypasses the unchanged-revision no-op. The revision tracks the source; a fix on our side leaves it unchanged, so the gate would skip the run	data_import.py
SecFundamentalsImporter(reparse=True) — forces full-history staging, and promote() rewrites the accessions whose reconstruction changed, stamping import_run_id	sec_fundamentals_importer.py
scripts/reparse_fundamentals.py — dry run by default, --apply to write	new

Unchanged rows are never touched; only accessions appearing in staged.discrepancies are rewritten. The update writes the full _SNAPSHOT_COLS set via the same _row_values() the insert uses, so a rewritten row can never be half old-parse and half new-parse. created_at keeps its original value. Nothing is wired into the scheduler.

A real bug the tests caught: false-positive discrepancies

test_reparse_leaves_unchanged_rows_untouched failed on first run — reparsing identical data reported a change. Cause: accepted_at is written tz-aware UTC but DateTime(timezone=True) only preserves tzinfo on Postgres; SQLite returns it naive, so _diff_fields compared representations and saw a difference.

Left alone this would have made the dry-run report claim every row needs rewriting — exactly the misleading signal that makes a blast-radius report worthless. _diff_fields now compares datetime instants via _same_value(). This also fixes a latent false positive in the pre-existing snapshot_discrepancy warning, which shares the same code path.

Verification

4 reparse tests added, driven through the real import framework with the fake SEC client. The key one seeds the database through the pre-fix parser (monkeypatching _YTD_TOLERANCE_DAYS back to 20 so a 4-4-5 Q3 is rejected and stored as null), then reparses with the fixed parser and asserts the row is rewritten in place with new provenance — the production scenario end to end. Also covered: unchanged rows keep their original import_run_id; reparse=False still reports and refuses to mutate; force bypasses the no-op. Full suite: 804 passed.

Not verifiable here: this reads and writes production Postgres, which is unreachable from this machine, so the SQLite harness is the limit of what could be self-tested. The UPDATE is plain SQLAlchemy Core with no dialect-specific constructs.

Running it

python scripts/reparse_fundamentals.py            # dry run, writes nothing
python scripts/reparse_fundamentals.py --apply    # rewrite changed rows

Two cautions for whoever runs it:

• Read the dry run for kinds of change, not just the count. The tolerance 20→25 change newly accepts facts for arbitrary filers, not only the names investigated here. Sample changed rows for issuers that were never on the list and confirm they are recovered nulls and corrected values — not something unexpected.
• It refetches Company Facts for every tracked issuer under the SEC throttle, because the facts a fixed parser now accepts were never stored. Expect a long run; the dry run pays that cost too, so budget for two passes.

Scope: this rewrites fundamental_snapshots only. Those rows currently feed the fundamentals API/UI and the parity report — scoring still reads the legacy fundamental_data table, and nothing in the backtest path touches FundamentalSnapshot. So a reparse cannot move composite scores or backtests until the A5 cutover happens. The plan's "changed history changes backtests" caution applies to workstream B's OHLCV rewrites, not to this.

---

Fifth pass — period identity

The parser's own stated rule was "period identity comes from end == reportDate, never fy/fp" — but _fiscal_context() derived the stored fiscal_year/fiscal_period by majority-voting exactly those fy/fp fields. The labelling contradicted the module's own principle, and SEC's labels are unreliable enough to break the quarter chain.

_period_identity() now derives both from period_end against the issuer's submissions.fiscalYearEnd: the form decides FY vs quarter (a 10-Q can no longer be labelled FY), and distance to the fiscal-year end decides which quarter. The MMDD is threaded through parse_snapshots(..., fiscal_year_end=...); without it the old fy/fp path is used unchanged, so nothing regresses for issuers lacking a calendar.

Rejected approach: classifying the period by fact spans. Every 10-Q carries both a YTD and a discrete fact ending at reportDate, so "best span match" reads COST's Q2 (167d) as a Q1; and taking the longest span mislabelled IRM's Q3 2020 10-Q as FY because that filing carries a 12-month fact. The prototype caught this as a regression on a working name before any code was written. Distance-to-year-end needs no facts at all and is unambiguous — the quarter bands sit 91 days apart, so ±35 absorbs even a 4-4-5 filer's 16-week Q4.

Labels no longer match issuer naming in one case, deliberately. A filer whose year ends in early January (DPZ, fiscalYearEnd 0102) shifts by one. That is harmless: fiscal_year and fiscal_period appear nowhere in the API schemas or routers — they are internal keys the derivation uses for ordering, YTD differencing and YoY pairing, and the API surfaces period_end. The requirement is uniqueness, monotonicity and YoY alignment, not nomenclature. DPZ's derived values are byte-identical before and after the shift, which is the proof.

Prototype evidence (before implementing)

Collisions = two period ends on one key, one silently discarded. Inversions = a period sorting before one that precedes it.

	CRM	FRT	STX	BXP	PPL	MTD	NTAP	WDAY	CRWD	COST	PEP	IRM	DPZ	AMCR	AAPL
collisions before	1	1	0	0	4	5	2	3	2	0	0	0	1	0	0
collisions after	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
inversions before	1	1	1	0	2	10	2	3	5	1	0	0	2	0	0
inversions after	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Validated on live data

8 of the 9 recover fully, every one matching the legacy provider to two decimals:

name	TTM EPS	revenue growth	legacy
CRM	8.63	10.9818	10.98
FRT	5.77	7.4263	7.43
STX	10.54	28.9227	28.92
BXP	1.99	1.6227	1.62
MTD	42.57	6.7785	6.78
NTAP	6.35	5.3713	5.37
WDAY	3.21	13.3165	13.32
CRWD	−0.10	23.1667	23.17

PPL is partial: revenue growth recovers (8.3353) but TTM EPS is still null — its 2026 Q1 tags no diluted EPS variant at all, which is the open basic-vs-diluted decision, not this bug. Note legacy claims −58.81% revenue growth for a utility; 8.34% is far more plausible.

Two bonus recoveries: FCX and REG had recovered EPS in the fourth pass but their revenue growth was still blocked by label collisions. REG now reads 7.7569 against legacy's 7.76. FCX reads 5.4378 against legacy's −24.23 — a genuine disagreement, likely the same Revenues vs ASC-606 ambiguity flagged for DVN in §2, and worth resolving with that decision.

Regression check — all byte-identical: IRM 0.92/15.637543, COST 19.88/9.231107, PEP 7.63/5.619741, DPZ 17.64/5.157289, AMCR null/64.834349, JPM 20.89/3.338823, DVN 3.59/0.095648, AZO 145.39/5.740494. Nothing that worked before moved.

7 tests added at _period_identity covering each production shape (10-Q-labelled-FY, December collision, January and mid-year ends, 4-4-5 quarters, the January-crossing shift, and the no-calendar fallback). Full suite: 811 passed.

Reparse note

This changes fiscal_year/fiscal_period for a large share of rows — every non-December filer, not only the broken ones. The dry-run count will be much larger than for the earlier fixes, and that is expected. Read it by field: fiscal_year/fiscal_period churn is the intended relabelling; changes to value columns are the recoveries.

Where the 25 stand now

22 of 25 have a fundamental score again. Remaining: XOM (CIK identity, still unfixed) and PSKY / Q, which are new registrants without enough filing history — correct behaviour, not a bug.

---

Sixth pass — CIK identity, and a much larger finding about share counts

XOM: pinned, plus the validation that should have caught it

company_tickers.json maps XOM to CIK 2115436 "ExxonMobil Holdings Corp", which has zero XBRL filings, while every 10-K/10-Q — including one filed 2026-05-04 — is still under CIK 34088. Which registrant is the real filer is a judgement about a corporate event, so it is pinned explicitly rather than guessed:

• sec_universe.cik_overrides() reads a {symbol: cik} JSON map from SystemSetting['sec_cik_overrides'] and applies it ahead of company_tickers.json. A malformed setting is logged and ignored, never fatal.
• To fix XOM, set: sec_cik_overrides = {"XOM": 34088}.

The more valuable half is that nothing noticed. A tracked issuer resolving to a registrant with no XBRL filings can never produce a snapshot, and is restaged on every run forever. The importer now records those in staged.no_xbrl_filings, reports them in the validation summary (no_xbrl_filings_count), and raises a no_xbrl_filings SystemEvent naming the CIKs and pointing at the override setting. It warns rather than fails — one misresolved ticker must not block the whole import.

3 tests added. Full suite: 814 passed.

CHTR was not a bug, and the real problem is much bigger

I previously called this "a separate small bug". Both halves were wrong.

CHTR's dei:EntityCommonStockSharesOutstanding facts stop at 2016-06-30 — exactly when the Time Warner Cable / Bright House deal closed and Charter became a multi-class issuer. Since then the cover page reports the count per share class, which is dimensional, and companyfacts is non-dimensional — so the facts are simply not in the API. Its recent filings tag no consolidated common-share concept at all, only preferred and treasury.

This is not specific to CHTR. Of 12 issuers checked, 7 have no share count at all:

issuer	latest shares_outstanding	dei fact history
META	null (4/4 recent)	never tagged (n=0)
CMCSA	null (4/4 recent)	stops 2009-12-31
BRK-B	null (4/4 recent)	stops 2011-04-29
CHTR	null (4/4 recent)	stops 2016-06-30
FOXA, NWSA, LEN	null (4/4 recent)	—
GOOGL / GOOG	12,230,000,000	works via the us-gaap fallback

So market cap is silently unavailable for a meaningful slice of the large-cap universe, and it is a source limitation rather than a parser defect: the two obvious workarounds are both already-rejected design decisions — class sums are impossible (the per-class facts are not in companyfacts at all), and the weighted-average diluted count is explicitly excluded because market cap needs a point-in-time value.

No code change made. Substituting weighted-average diluted shares would silently overturn a deliberate design decision and produce a subtly wrong market cap for exactly the biggest, most-watched names. That is a call to make explicitly, so it is listed as a decision below rather than quietly implemented.

---

Seventh pass — multi-class share counts (decision taken: weighted-average fallback)

Why this fallback, and why not the alternatives

Two candidates existed. The one not taken: derive the count as net_income ÷ diluted_eps from columns already stored — no migration at all, and measured accurate (GOOGL +0.48%, MRNA −0.45%, AAPL +0.19%, MSFT +0.18%). Rejected because it depends on the derived quarter chain — the very thing these fixes have been repairing, and FOXA already fails it — and because the two-class EPS method makes net_income differ from the EPS numerator for exactly the multi-class issuers this targets.

Taken instead: store the reported WeightedAverageNumberOfDilutedSharesOutstanding. It is the number the filer computed, needs no chain, and covers one issuer more.

control	point-in-time	wavg diluted (latest qtr)	ratio
GOOGL	12,230,000,000	12,309,000,000	0.9936
MRNA	396,786,259	395,000,000	1.0045
AAPL	14,687,356,000	14,725,873,000	0.9974
MSFT	7,428,434,704	7,445,000,000	0.9978

Shape of the change

• Migration 027 adds fundamental_snapshots.weighted_avg_diluted_shares. A separate column, never backfilled into shares_outstanding, so the point-in-time column keeps its strict meaning and the fallback stays a read-time decision.
• Parser stores the shortest-span fact ending at period_end (the most recent quarter's average, closest to the current count) — deliberately not the YTD one, since an average is not cumulative and the YTD convention does not apply.
• Derivation falls back only when the cover-page count is absent, and sets shares_outstanding_estimated.
• API exposes shares_estimated, so market_cap_est and fcf_yield are never presented as exact when they rest on a period average.

Validated on live data

issuer	shares_outstanding	estimated
GOOGL, AAPL, MSFT, MRNA	unchanged point-in-time values	False
META	2,564,000,000	True
CMCSA	3,570,000,000	True
CHTR	126,849,271	True
FOXA	432,000,000	True
NWSA	555,700,000	True
LEN	240,776,000	True
BRK-B	still null	False

6 of 7 recovered, no regression on the controls. BRK-B remains unavailable and honestly so: Berkshire reports per equivalent Class A share, dimensionally, so it has no consolidated weighted-average fact either. Nothing in companyfacts can give it a share count.

Known caveat, accepted: for issuers using the two-class method the count is the EPS denominator. For CHTR that is Class A only — which is also the basis on which Charter's equity market cap is normally quoted, so it is the right number for this purpose, but it is not "all shares of all classes".

3 tests added. Full suite: 817 passed. Alembic single head at 027.

Needs the reparse to land: existing rows have weighted_avg_diluted_shares = NULL until scripts/reparse_fundamentals.py --apply runs, so market cap stays missing for these issuers until then.

---

Eighth pass — revenue basis (decision: keep ASC-606, no change)

The two concepts measure different things: RevenueFromContractWithCustomerExcludingAssessedTax is customer-contract revenue (an E&P's oil/gas/NGL sales), while Revenues is the total income-statement line, which for commodity producers folds in mark-to-market derivative gains/losses. That is why DVN's ASC-606 figure is larger: 4,508M of sales minus ~701M of hedging losses gives the 3,807M Revenues line.

Measured across 21 issuers (deliberately energy-weighted, where the gap concentrates):

• Both tags present and differing >1%: 5 of 21 — DVN +18.4%, COP −14.3%, OXY +6.5%, FCX −2.8%, PPL +1.6%. Everyone else tags one, or they are identical (COST +0.0%).
• Concept choice flips within an issuer's chain: 0 of 21. Whichever tag wins, the series is internally consistent, so YoY never compares two definitions.

Decision: keep ASC-606 first, change nothing. Derivative gains/losses are mean-reverting and sign-flipping; folding them into "revenue growth" turns the sub-score into a partial hedging-P&L read for exactly the affected names. The consistency argument for switching is empirically absent (zero flips), and changing would churn every dual-tagging issuer's stored value — widening the reparse diff — to make ~5 names noisier.

Correction to the fourth/fifth-pass note: FCX's disagreement with legacy (+5.44% vs −24.23%) is not this ambiguity. Its two tags differ by only 2.8%, and FCX's own revenue rose 22,703M → 25,186M YoY, so −24% is not credible — legacy is simply wrong there, and this decision does not touch it. So the basis choice moves only DVN, COP, OXY.

The mirror hazard — an issuer where ASC-606 is only a fragment of revenue (a bank's fee income) — was checked (all 15 recovered banks resolve total revenue, not a fragment). A fragment-detection warning was prototyped and then removed: with no UI surface it would only have lived in the run summary, and the case it guards against is not currently present. Documented and closed rather than shipped as dead plumbing. If a fragment case ever appears, it shows up as an implausibly low revenue in the next parity report.

---

Ninth pass — basic-EPS fallback (PPL) and HAL resolved

PPL: basic-EPS fallback (decision taken)

PPL's 2026 Q1 tags no diluted EPS variant at all, only basic — a single-filing omission (its other quarters tag diluted), but that one missing period broke the quarter chain and nulled TTM. EarningsPerShareBasic / IncomeLossFromContinuingOperationsPerBasicShare are now appended to _EPS_CONCEPTS, last, so they only fire when no diluted variant exists.

Evidence (19-name scan): a basic fallback helps exactly 1 name (PPL). Basic-vs-diluted is ~0.5–1.2% for most, +1.2% for PPL. The one name where it genuinely diverges (TSLA +13.3%) already tags diluted, so it never reaches the fallback. Basic is always ≥ diluted, so the result slightly overstates EPS / understates P/E — accepted, since it fires only on an otherwise-null period.

Validated: PPL TTM EPS null → 1.63 (≈$36 / 1.63 = 22.1 vs legacy P/E 22.43). AAPL, MSFT, DUK, HAL unchanged — diluted still wins wherever present. 2 tests added. Full suite: 819 passed.

HAL: resolved, and it was never our bug

HAL's TTM EPS is now 1.81 (≈$33 / 1.81 = 18.2 vs legacy P/E 18.01) — the period-identity and EPS-concept work already fixed it. The "unexplained null" is closed.

Its 2024 EPS values are garbage (680000, 1480000, …) because Halliburton's own 2024 XBRL tags EarningsPerShareDiluted = 680000 in unit USD/shares — a filer scale error in the source, faithfully stored. It only poisons TTM windows that include 2024, which the current point-in-time report does not use, so no code change: clamping EPS to "plausible" values would risk masking real ones. Documented as a known source-data quirk.

This does surface a latent robustness point (not acted on): a single fat-fingered per-share value poisons any TTM window it lands in. It is invisible in the current report and out of scope here, but worth a note if historical TTM series are ever surfaced.

All 25 lost names accounted for

status	names
recovered (22)	ARE, AZO, BXP, COST, CRM, CRWD, DPZ, DVN, FCX, FRT, HAL, KHC, MOS, MTD, NTAP, PEP, PPL, REG, SJM, STX, SWKS, WDAY
XOM	fixed by the sec_cik_overrides pin (needs the setting applied)
PSKY, Q	new registrants without enough filing history — correct behaviour, not a bug

Still outstanding

Revised after the second pass, in the order I would take them:

Everything actionable without a live database is now done. What remains is one hard data limitation and two operational steps that only run against production.

1. §3a fiscal-period identity — done, fifth pass.
2. §1 split contamination, part (a) — done, third pass.
3. §5 EPS concept gap — done, third pass.
4. §4 XOM CIK remap + zero-filings validation — done, sixth pass.
5. Reparse path — done, fourth pass.
6. Multi-class share counts — done, seventh pass (weighted-average fallback).
7. DVN/FCX revenue basis — decided, eighth pass (keep ASC-606, no change).
8. PPL basic-EPS fallback — done, ninth pass.
9. HAL null TTM EPS — resolved, ninth pass (already fixed; 2024 is a filer error).

Review finding — two fixes on this branch silently interacted

Caught in review, not by me. _merge_amendments (the per-field amendment fix, first pass) builds the merged period from _MERGED_FIELDS + _CARRIED_FIELDS alone, so a column in neither list is absent from the merged row, not merely stale — and every caller reads it with getattr(row, name, None), which quietly returns None.

weighted_avg_diluted_shares (the market-cap fallback, seventh pass) was never added to _MERGED_FIELDS. The failure needed both fixes to be present at once: a multi-class issuer and a partial amendment on its latest period — META with a Part-III-only 10-K/A — would silently lose market cap and FCF yield again, i.e. the seventh pass's fix undone by the first pass's mechanism. I updated _SNAPSHOT_COLS in the importer when adding the column but not _MERGED_FIELDS in the derivation.

Fixed, with a regression test for the specific case. The more useful addition is a guard — test_merge_lists_cover_every_parser_field asserts the two lists cover every SnapshotRow field, so the next column added fails loudly instead of losing data quietly. Verified it would have caught this one.

Lesson worth keeping: a hand-maintained field list that reconstructs an object is a silent data-loss footgun. _SNAPSHOT_COLS (importer) and _MERGED_FIELDS (derivation) must both track the parser's SnapshotRow, and only one of them is now enforced by a test.

Genuinely unfixable from this data

• §1 part (b) — a split post-dating the last filing (KLAC). No snapshot carries share-count evidence, so no guard built on share counts can fire. Needs a corporate-actions source or an external market-cap reconciliation.
• BRK-B market cap — Berkshire reports per equivalent Class A share, dimensionally, so it has neither a cover-page count nor a weighted-average one. Nothing in companyfacts can give it a share count.

Operational steps (production only — cannot run from here)

• Apply the setting sec_cik_overrides = {"XOM": 34088}.
• Run scripts/reparse_fundamentals.py — dry run first, then --apply. This is what lands every parser-side fix (revenue/EPS concepts, Q3 span, period identity, weighted-average shares via migration 027) onto existing rows. Until it runs, those fixes are inert in prod.

Standing decision, revisit only if it bites

• COF-class share-change threshold — kept at 25%. Revisit only if the 3.2% universe hit-rate proves painful.

Known source-data quirk, not acted on

• A single fat-fingered per-share value in a filer's XBRL (HAL 2024) poisons any TTM window it lands in. Invisible in the current point-in-time report; relevant only if historical TTM series are ever surfaced.

PSKY and Q need nothing — they are new registrants without enough filing history, which is correct behaviour.

---

Closing — post-reparse verification (2026-07-24)

Production reparse

Two apply runs against prod (scripts/reparse_fundamentals.py --apply):

• Run 6 (all fixes through the seventh pass): 262 inserted, 28,664 rewritten — 99.4% of which was backfilling the new weighted_avg_diluted_shares column; the behavioural churn matched the dry run exactly. The five duration facts clustering at 190–234 changed rows each is the 4-4-5 Q3 recovery signature. accepted_at changed on only 76 rows (0.25%), confirming the tz-comparison fix works against real Postgres.
• Run 7 (after the fiscal-year-end fix below): 3 inserted, 322 rewritten — BEN, DELL, and boundary-year relabels for 53-week filers whose derived MMDD shifted a few days.

cik_updates: 1 on run 6 was the XOM pin taking effect; XOM now has 68 snapshot rows, latest period end 2026-03-31.

Regression caught by the collision check — and its fix

The before/after key-collision query (~130 rows max 6 → 44 rows all 2) surfaced one real regression: BEN. submissions.fiscalYearEnd declares 1231 while every Franklin Resources 10-K ends 09-30, so _period_identity — which trusted the declared value — put BEN's real fiscal Q1 zero days from the claimed year end (no band matched) and labelled its fiscal Q2 as Q1. The collision discarded a period and BEN lost TTM EPS and revenue growth it had before the branch. Fixed in 3d42ca7: resolve_fiscal_year_end() prefers the issuer's own most recent 10-K reportDate (which is the fiscal year end by definition) and treats the declared field as fallback. Full-universe scan: 2 of 506 issuers mis-declare (BEN 91d, DELL 29d); both now derive correctly (BEN rg 3.8243 vs legacy 3.82; DELL 38.5735 vs 38.57).

Residual collisions after run 7: 36 rows, all count-2, latest year 2023 — the 53-week drift class (AVY/CDNS/RVTY/JNJ/TDY/DPZ at 5–6-year intervals). Newest-wins degrades one historical FY row; no current period is affected. Left alone deliberately: eliminating them means modelling each filer's actual 52/53-week calendar per year, for rows feeding no current metric.

The verdict: 2026-07-24 parity report vs the 2026-07-23 baseline

metric	baseline	after
candidate scored	482	504	legacy scores 507; gap = PSKY, Q (new registrants) + FITB
revenue_growth candidate available	442	489	banks, REITs, 4-4-5 recovered
pe_ratio candidate available	432	452	net of the split-guard nulls
revenue_growth median abs delta	0.0038	0.0038	47 names added at unchanged agreement
pe_ratio median / p95 abs delta	0.5883 / 7.73	0.5576 / 6.03	corrupted outliers gone

Revenue material_differences rose 84 → 96: the newly compared names include the cases where legacy is the wrong side (JPM 108.98% vs 3.34%, PPL −58.81% vs 8.34%, FCX −24.23% vs +5.44%). Material is symmetric; these flag the provider being corrected.

The split guard is visible in the report: BKNG (1.10), COF, TPL, AMCR, WAT all null P/E now. KLAC (6.19) is the one known-wrong value left — the post-filing split documented as unfixable without a corporate-actions source.

Correction to the seventh pass: the claim that guard-tripped issuers "keep their fundamental score, losing one of three inputs" fails for FITB, the one name that also lacks revenue growth (its recent filings tag only ASC-606 fee-income fragments, then nothing) — nulling its contaminated P/E (Comerica merger, 661M → 902M shares) drops it to one metric and no score. Accepted: the composite renormalises, and legacy's 58% "revenue growth" for a bank was itself junk.

Recommendation

The A5 gate evidence now supports approving the cutover: coverage within 3 of legacy with every gap explained, agreement essentially exact where both sides exist, every corrupted value either fixed or deliberately nulled with a caveat, and the remaining score deltas are documented definition differences — called out, not averaged away, as the plan requires. Carry KLAC as the one known caveat in the approval note.