signal-platform/reports/fundamentals-parity-20260723-findings.md

# A5 parity report — root-cause findings

Investigation of `fundamentals-parity-20260723T210658161480Z.json` (511 tickers,
generated 2026-07-23). Method: replayed the production parser
(`sec_facts_parser.parse_snapshots`) and derivation (`fundamentals_derivation.derive`)
against **live SEC companyfacts**, using the importer's own `_filing_meta` and
backfill accession set, then cross-checked prices against IBKR. No database was
available locally, so every conclusion below is reproduced from source data rather
than read out of prod.

Repro script: `scratchpad/diag.py` (`--history` replays the full backfill path).
Every claim below was verified on the named issuer. Names that were *not*
individually inspected are listed as unclassified — an earlier draft of this
document guessed their cause from fiscal-year-end dates and was wrong for most of
them, so the guessing is not repeated here.

## Verdict

Where both sides have a value the candidate data is good: P/E spearman 0.968,
revenue growth agreeing to 4 decimals for most names, score spearman 0.825. Every
defect found is a **parser/derivation bug or an identity problem** — not a data
quality problem with SEC or Dolt. The largest cluster is period identity, which is
exactly the risk A3 flagged as primary.

## 1. P/E outliers — splits corrupt TTM EPS, and the split guard doesn't cover it

`derive()` sets `result.ttm_diluted_eps` at `fundamentals_derivation.py:88` and only
calls `_guard_split_sensitive_metrics()` at line 105, which annotates `result.metrics`
(the `MetricSeries` objects). `ttm_diluted_eps` is a bare scalar and is never guarded.
`fundamentals_parity_service._pe()` consumes it directly.

The cleanest evidence that the *candidate* side is the broken one: reconcile each
P/E against the report's own price. Legacy comes out sane in both cases, candidate
does not.

**BKNG — guard fired, nobody listened.** Share count jumps 31.7M → 774.9M between the
FY2025 10-K and the 2026 Q1 10-Q (≈25:1 split). TTM EPS therefore sums three pre-split
quarters (27.31 + 84.01 + 44.18 = 155.50) plus one post-split quarter (1.36) =
**156.86** — mixed units. Live price $172.83 matches the price the report implies
exactly (1.1018 × 156.86 = 172.83), so the price is correct and current. Against that
price, legacy's P/E of 22.44 implies EPS ≈ 7.70 — a coherent post-split number, versus
the candidate's 156.86. The derivation *did* raise `"Not comparable: share count
changed at least 25%; possible split or corporate action."` on `eps_growth_yoy` and
`share_count_change_yoy` — P/E never sees it.

**KLAC — the guard cannot fire.** The split post-dates the most recent 10-Q (period end
2026-03-31), so no snapshot shows any share-count change (`share_count_change_yoy` =
−1.2%). TTM EPS **35.31** is internally consistent and entirely pre-split; the price
($223.30 live, ≈218.7 in the report) is post-split. Reconciling: legacy P/E 60.21
against the report price implies EPS ≈ 3.63 ≈ 35.31/9.7 — i.e. legacy is consistent
with a ~10:1 split and correct, and the candidate is off by exactly the split factor.
(IBKR's split-adjusted `open_52w` of 89.36 corroborates 10:1.)

This is the important case: **a split after the latest filing is undetectable from
snapshots alone.** No share-count test can catch it. Reconciliation needs a corporate
actions source or a price-vs-EPS plausibility check.

**COF — not a bug, a definition difference.** Shares 383M → 639M in 2025 Q2 is the
Discover acquisition. TTM GAAP EPS is genuinely $3.92 because the merger-charge quarter
(−10.19) sits in the window. Candidate P/E 51.01 is arithmetically correct on a GAAP TTM
basis; legacy's 11.61 is an adjusted/forward convention. Disclose, don't fix. Note this
single row drives the report's largest change (rank 1 → 456).

## 2. Bank revenue growth — concept-mapping gap (confirmed)

`sec_facts_parser._DURATION_USD["revenue"]` is:

```
RevenueFromContractWithCustomerExcludingAssessedTax, Revenues, SalesRevenueNet
```

Banks tag **`RevenuesNetOfInterestExpense`** in their 10-Qs:

| filer | 2026 Q1 10-Q tags present | parsed `revenue` |
|---|---|---|
| JPM | `RevenuesNetOfInterestExpense` 49,836M, `NoninterestIncome`, `InterestIncomeExpenseNet` | **null** |
| GS | `RevenuesNetOfInterestExpense` 17,227M, `InterestAndDividendIncomeOperating`, … | **null** |
| WFC | `RevenuesNetOfInterestExpense` 21,436M, … | **null** |

JPM's FY2025 10-K *also* tags `Revenues` (182,447M — identical value), so only the annual
row populates; GS never tags `Revenues` at all. Revenue growth needs five consecutive
quarterly values, so it is null for the whole cluster (JPM, GS, MS, WFC, TFC, MTB, FITB,
RF, SYF, BNY, BX, BLK, SPGI, ACGL, CBOE).

A second variant of the same gap: **ARE** and **KHC** tag
`RevenueFromContractWithCustomer**Including**AssessedTax` — also absent from the list —
so revenue is null on every row while EPS parses fine.

**Fix:** add `RevenuesNetOfInterestExpense` and the `IncludingAssessedTax` variant.

**Latent risk while you're in there:** `RevenueFromContractWithCustomerExcludingAssessedTax`
is *first* and "first present wins". For a bank that tags it, it captures only ASC-606 fee
revenue, not total revenue — a silently **understated** number rather than a null, which is
worse. DVN shows the same hazard from the other side: its 2026 Q1 tags both
`RevenueFromContractWithCustomerExcludingAssessedTax` (4,508M) and `Revenues` (3,807M),
an 18% difference decided purely by list order.

## 3. Period identity — the largest cluster, three confirmed mechanisms

### 3a. Fiscal-year label collisions (CRM, FRT, STX)

`_fiscal_context` majority-votes SEC's `fy`/`fp` fields, and `_select_latest_per_period`
keys on `(fiscal_year, fiscal_period)`. When SEC's labels disagree with the calendar, two
distinct periods collide on one key and **one is silently discarded**:

- **CRM** — two rows keyed `2025 FY`, ending 2025-01-31 and 2026-01-31.
- **FRT** — two rows keyed `2024 FY`, ending 2024-12-31 and 2025-12-31.
- **STX** — the year ending 2025-06-27 is labelled **`2027 FY`**, so it sorts *after*
  `2026 Q3` (period end 2026-04-03) and is taken as the latest quarter.

The survivor's `period_end` then contradicts the fiscal ordering, Q4 derivation and the
consecutive-quarter chain break, and TTM EPS + YoY both go null.

**FRT is a calendar-year (Dec) filer**, so this is *not* limited to non-calendar fiscal
years — the earlier assumption that it was is wrong. Any filer SEC labels inconsistently
is exposed.

### 3b. Amendment selection blanks a period (DVN)

DVN has two rows for `2025 FY` (both ending 2025-12-31): the 10-K with complete financials,
and a **10-K/A carrying no financial facts at the report date** (`rev=None eps=None`).
`_select_latest_per_period` takes the newest `accepted_at`, so **the empty amendment wins**
and the FY2025 row becomes all-null, breaking the chain.

This is the most dangerous of the three: it is not exotic. Any issuer filing a 10-K/A —
including routine Part III amendments that restate nothing — silently loses that period.
The rule needs to prefer the newest accession *that actually carries the fact*, per field,
rather than the newest accession outright.

### 3c. 4-4-5 retail calendar — Q3 only, misses by ~2 days (COST, PEP)

`_EXPECTED_YTD_DAYS["Q3"] = 273` with `_YTD_TOLERANCE_DAYS = 20` accepts 253–293 days. A
12/12/12/16-week filer's YTD-Q3 is 36 weeks ≈ **251–252 days** — just under the floor.

Confirmed, facts present and rejected:

- COST 2026 Q3: `RevenueFromContractWithCustomerExcludingAssessedTax` span=**251d**
  val=207,431M, `EarningsPerShareDiluted` span=251d val=14.01 → row stored with
  `rev=None eps=None start=None`. Same for 2025 Q3 and 2024 Q3.
- PEP: every Q3 row is `rev=None eps=None`; Q1/Q2/FY all populate.

Q1 (83d vs 91±20), Q2 (167d vs 182±20) and FY (363–364d vs 365±20) all pass — only Q3
fails, every year. The code comment claims the tolerance "covers 52/53-week fiscal
calendars"; it does not cover 4-4-5 ones.

Note this does **not** apply to ordinary 13-week 52/53-week filers (STX's Q3 YTD is 279d and
passes) — their failures are 3a, not this.

**Fix:** widen the Q3 tolerance to ~25 days, or derive the expected span from the filer's own
fiscal calendar rather than a fixed 91/182/273.

## 4. CIK identity (XOM)

SEC's `company_tickers.json` now maps **XOM → CIK 2115436 "ExxonMobil Holdings Corp", which
has 0 filings**. All 26 XBRL 10-K/10-Qs sit under the old CIK **34088 "EXXON MOBIL CORP"**.
XOM therefore has no snapshots at all, and nothing in the pipeline notices that a tracked
issuer resolved to a CIK with zero filings.

PSKY (5 filings) and Q (3 filings) are genuinely new registrants — expected, not a bug.

## Status of the 25 names that lose their fundamental score

Production requires ≥2 metrics (`scoring_service.py:502`), the same rule the parity harness
uses, so these genuinely drop the fundamental dimension and the composite renormalises over
the remaining four.

| cause (confirmed on the named issuer) | names |
|---|---|
| FY label collision (3a) | CRM, FRT, STX |
| 4-4-5 Q3 span (3c) | COST, PEP |
| revenue concept gap (§2) | ARE, KHC |
| amendment blanks period (3b) | DVN |
| CIK identity (§4) | XOM |
| new registrant — expected | PSKY, Q |
| **not yet classified** | AZO, BXP, CRWD, FCX, HAL, MOS, MTD, NTAP, PPL, REG, SJM, SWKS, WDAY |

13 of 25 confirmed. The unclassified 13 have not been inspected and should not be assumed to
share a cause — the confirmed set already spans five distinct mechanisms.

## Recommended order of work

1. **Amendment selection (3b)** — highest blast radius, affects any 10-K/A filer, and the
   current rule is wrong in principle rather than at the margin.
2. **Revenue concept list (§2)** — add `RevenuesNetOfInterestExpense` and
   `IncludingAssessedTax`; audit the ASC-606-first priority, which can understate rather
   than null.
3. **Q3 YTD span tolerance (3c)** — effectively one line.
4. **XOM CIK remap (§4)** — plus a validation that flags any tracked ticker resolving to a
   CIK with zero XBRL filings.
5. **Split safety for `ttm_diluted_eps` (§1)** — propagate the existing guard to the scalar,
   and add a price-vs-EPS plausibility check for splits that post-date the last filing.
6. **Fiscal-period identity (3a)** — the deepest fix; consider keying period identity on
   `period_end` rather than SEC's `fy`/`fp`.

Re-run the parity report after these and re-classify the remaining 13 before making a
cutover decision. The current report should not be approved as-is: its coverage gaps are
artifacts of the above, not real absences in the source data.

---

# Fixes applied (items 1–3)

| # | change | file | effective |
|---|---|---|---|
| 1 | amendment resolution is now **per field** — newest accession that actually reports a fact wins; only rows sharing the newest `period_end` are merged, so a mislabelled filing is never blended in | `fundamentals_derivation.py` | **read time — immediately** |
| 2 | appended `RevenueFromContractWithCustomerIncludingAssessedTax` and `RevenuesNetOfInterestExpense` to the revenue concept list | `sec_facts_parser.py` | parse time — **needs reparse** |
| 3 | YTD span tolerance 20 → 25 days, covering 4-4-5 retail calendars | `sec_facts_parser.py` | parse time — **needs reparse** |

Fix 2 is deliberately **additive**: the new tags go at the end of the priority list, so
every issuer that already resolved keeps the same concept and only issuers that resolved
to nothing gain a value. A regression test pins that ordering.

Tests: 7 added across `test_sec_facts_parser.py` and `test_fundamentals_derivation.py`.
The 5 behaviour-changing ones were confirmed to fail against the pre-fix code; the other 2
are invariance guards that pass both ways. Full unit suite: 795 passed.

## Validation against live SEC data

Re-ran the parser + derivation on live companyfacts. Every targeted name recovers, and
the recovered values independently agree with the legacy provider:

| name | cause | revenue growth before → after | legacy | TTM EPS after |
|---|---|---|---|---|
| COST | 4-4-5 Q3 | null → **9.2311** | 9.23 | 19.88 |
| PEP | 4-4-5 Q3 | null → **5.6197** | 5.62 | 7.63 |
| KHC | concept (Including) | null → **−1.7457** | −1.75 | −4.85 |
| DVN | partial 10-K/A | null → **0.0956** | −1.51 | 3.59 |
| ARE | concept (Including) | null → **−5.3462** | −9.53 | −6.27 |
| JPM | concept (bank) | null → **3.3388** | 108.98 | 20.89 |
| GS | concept (bank) | null → **11.1974** | 6.67 | 54.75 |
| WFC | concept (bank) | null → **4.1847** | 72.75 | 6.47 |

COST/PEP/KHC matching legacy to two decimals is strong evidence the parse is now correct.
The banks are the opposite case and worth noting for the cutover argument: legacy's JPM
109% and WFC 73% "revenue growth" are not plausible for a bank, while the SEC-derived
3.3% and 4.2% are — here the candidate is **better** than what it would replace. DVN and
ARE still differ from legacy; DVN is the `Revenues` vs ASC-606 ambiguity noted in §2 and
is the one open definition question.

Regression check on names that were already correct — IRM, KLAC, BKNG — reproduces their
previous values exactly (IRM 15.6375, KLAC 13.3895, BKNG 14.9506; TTM EPS unchanged).
Nothing that worked before moved.

### Concept consistency across the bank chains (checked, clean)

Because `Revenues` still outranks `RevenuesNetOfInterestExpense`, a filer could resolve the
FY row to one concept and its quarters to the other — which would make
`Q4 = YTD(FY) − YTD(Q3)` a subtraction across two definitions, and poison every TTM window
containing it. Checked all 15 recovered banks (`scratchpad/concept_check.py`):

- **14 resolve a single concept across the whole chain** (GS, WFC, MS, TFC, MTB, FITB, RF,
  SYF, BNY, BX, BLK, SPGI, ACGL, CBOE).
- **JPM is mixed but benign**: its FY2025 row tags both, at an *identical* 182,447M, so Q4
  subtracts like for like. No filer showed the two tags disagreeing where both appear.

So the "candidate beats legacy for banks" claim above is safe as stated. **Residual risk:**
a future filer whose two tags differ would fail silently. Cheapest hardening is to treat
the two as one logical revenue concept rather than separate priority entries; the detector
script above turns this into a one-command check.

## Operational note — the parser fixes need a deliberate reparse

`sec_fundamentals_importer.promote()` treats snapshots as **immutable per accession**: a
re-run skips any accession already stored and records a `snapshot_discrepancy` SystemEvent
instead. So fixes 2 and 3 change nothing for rows already in the database — recovering
COST/PEP/JPM/etc. requires deleting the affected snapshot rows and re-importing, or adding
an explicit reparse path. Usefully, the discrepancy warning names exactly which stored
accessions now reconstruct differently, so a dry run over existing data will enumerate the
blast radius before anything is rewritten.

---

# Second pass — all 25 lost names now classified

Re-ran `diag.py --history` over every previously unclassified name, with fixes 1–3 in
place. (One name, DPZ, had been dropped from the unclassified list when this document was
rewritten; it is included here.)

## 11 of 25 recover

COST, PEP, KHC, DVN, ARE, **AZO, MOS, SJM, SWKS, HAL, DPZ** — and again the recovered
revenue growth matches the legacy provider to two decimals on every one:

| name | candidate | legacy | | name | candidate | legacy |
|---|---|---|---|---|---|---|
| AZO | 5.7405 | 5.74 | | SWKS | 2.3303 | 2.33 |
| MOS | 12.3388 | 12.34 | | HAL | −1.7201 | −1.72 |
| SJM | 3.7222 | 3.72 | | DPZ | 5.1573 | 5.16 |

Precisely: all 11 clear the ≥2-metric floor and regain a fundamental score. P/E returns for
AZO, MOS, SWKS, DPZ, COST, PEP and DVN. ARE, KHC and SJM have genuinely negative TTM EPS,
so their P/E stays null correctly. **HAL's TTM EPS is still null and the cause is not yet
established** — it scores on revenue growth + surprise. Loose end.

## 14 remain, in four causes

| cause | names | count |
|---|---|---|
| **fiscal-year label collisions (§3a)** | CRM, FRT, STX, BXP, CRWD, MTD, NTAP, WDAY, PPL | **9** |
| **EPS concept gap (new — §5 below)** | FCX, REG | 2 |
| CIK identity (§4) | XOM | 1 |
| new registrant — expected, not a bug | PSKY, Q | 2 |

The label bug is now the dominant cause by a wide margin, and it is more varied than first
described — it is not only colliding `fiscal_year` values:

- **BXP** — a *10-Q* for period end 2026-03-31 is labelled `2026 **FY**`. The **fiscal
  period** is wrong, not just the year, so `_select_ytd` then measures the 90-day fact
  against the 365-day FY expectation and rejects it too.
- **NTAP, WDAY, MTD, CRWD** — two different period-ends colliding on one key (the pattern
  first seen on CRM/FRT).
- **PPL** — the worst observed: **four** rows keyed `2022 Q3`, with period ends 2022-09-30,
  2023-03-31, 2023-06-30 and 2023-09-30.

## 5. New cause — EPS concept coverage

`_EPS_CONCEPTS = ["EarningsPerShareDiluted"]` is the only tag read. Confirmed by listing
every `USD/shares` duration concept in the relevant filings:

- **REG** tags only `IncomeLossFromContinuingOperationsPerDilutedShare`, on every filing —
  EPS is null everywhere, so no TTM EPS and no P/E, ever.
- **FCX** is the nastier shape: its **10-Qs** tag `EarningsPerShareDiluted`, but its
  **10-K** tags only `IncomeLossFromContinuingOperationsPerDilutedShare`. The FY row loses
  EPS, so `Q4 = YTD(FY) − YTD(Q3)` is undefined and TTM dies — an issuer that switches
  concept *by form type* looks like partial data rather than a mapping gap.

**Fix:** append `IncomeLossFromContinuingOperationsPerDilutedShare` to `_EPS_CONCEPTS`.
Same additive shape as the revenue fix; recovers REG outright and FCX's FY row.

**Related decision, not a fix:** PPL's 2026 Q1 tags *no diluted variant at all* — only
`EarningsPerShareBasic` and `IncomeLossFromContinuingOperationsPerBasicShare`. Adding the
diluted continuing-ops tag does not help it. Falling back to basic EPS is a definition
change (basic ≠ diluted) and should be an explicit call, not a silent one.

---

# Third pass — fixes #2 and #3 applied

| # | change | file | effective |
|---|---|---|---|
| 2a | `_guard_split_sensitive_metrics()` now returns whether the *latest* period is split-suspect, and `derive()` nulls `ttm_diluted_eps` (setting `ttm_diluted_eps_caveat`) when it is | `fundamentals_derivation.py` | read time — immediately |
| 3 | appended `IncomeLossFromContinuingOperationsPerDilutedShare` to `_EPS_CONCEPTS` | `sec_facts_parser.py` | parse time — needs reparse |

5 tests added; the 3 behaviour-changing ones confirmed to fail against pre-fix code, 2 are
invariance guards. Full unit suite: **800 passed**.

## Validated on live data

| name | before | after | |
|---|---|---|---|
| FCX | TTM EPS null | **1.89** | recovered |
| REG | TTM EPS null | **2.92** | recovered |
| BKNG | TTM EPS 156.86 → P/E **1.10** | **null** + caveat | false perfect score removed |
| COF | TTM EPS 3.92 → P/E 51.01 | **null** + caveat | see side effect below |
| KLAC | TTM EPS 35.31 → P/E **6.19** | unchanged | **still wrong — 2b not fixed** |
| IRM, COST | — | unchanged | no regression |

FCX and REG regain a fundamental score (EPS + surprise clears the ≥2 floor). Their
**revenue growth is still null** — both are also blocked by the label bug (REG has a
mislabelled duplicate `2024 Q2`; FCX is missing its 2024 FY row entirely).

## Threshold decision — RESOLVED: keep 25%

Measured against the database (`scratchpad/share_change_check.sql`): **15 of 467 comparable
issuers (3.2%)** trip the ≥25% guard on their latest period.

| band | names | cause |
|---|---|---|
| ≥200% | BKNG 23.8×, ORLY 14.5×, NFLX 9.8×, NOW 5.0×, TPL 3.0× | forward splits |
| 50–142% | CHTR (query artifact), **AMCR −68% (1-for-5 reverse split)**, WAT, COF | split + stock-funded M&A |
| 25–47% | OMC, BG, HBAN, FITB, COHR, RKLB | stock-funded M&A, ordinary dilution |

**Keep the threshold at 25%**, for three reasons — the first of which is empirical and came
out of checking AMCR:

1. **A real split trips at only 68%.** AMCR's 1-for-5 reverse consolidation
   (2,308,359,941 → 462,045,690 shares, ratio 4.996, between the Nov 2025 and Feb 2026
   10-Qs) shows up as −68%. Raising the bar to 100% to spare the M&A cases would have let a
   genuine split straight through. Split magnitude and M&A magnitude overlap in practice,
   not just in theory.
2. **The cost is milder than first described.** Losing P/E leaves revenue growth + earnings
   surprise = 2 metrics, which still clears the ≥2 floor. Affected issuers keep a
   fundamental score; they lose one of three inputs.
3. **The severities are asymmetric.** A missed split yields a P/E off by 10–25×, clamping to
   a *perfect 100* sub-score. Over-nulling yields a missing input the scorer already handles
   by renormalising.

Honest caveat: the guard is blunt — it detects that a share base moved, not how much damage
resulted. AMCR's pre-fix P/E was 28.61 against legacy's 29.47, i.e. only ~10-15% off, because
most of its YTD figures had already been restated on the post-split basis. So the guard
sometimes removes a roughly-usable number. That is the accepted price of a rule that cannot
measure the split factor.

Two data notes from the same check:

- **CHTR is a query artifact, not a guard trip.** The SQL picks the newest period *with* a
  share count, while `derive()` picks the newest period and then reads shares off it. CHTR's
  recent snapshots have a null `shares_outstanding`, so the query fell back to the 2016 Time
  Warner merger. In the real path its change is None and the guard never fires — so the true
  count is ~14. But it also means **CHTR has no recent share count, which breaks its market
  cap in the API** — a separate small bug.
- **AMCR was suspected of being a `shares_outstanding` parsing bug and is not.** It is a real
  corporate action, correctly detected. `abs()` in the guard already handles reverse splits.

## Side effect — COF

The guard fires on *any* ≥25% YoY share-count move, not only splits. COF's 383M → 639M jump
is the Discover acquisition, so it now nulls too and **loses the P/E of 51.01** that this
document previously called "arithmetically correct on a GAAP TTM basis".

I think nulling is right: TTM EPS sums four quarters whose per-share figures use different
weighted-average denominators, and across a 67% share change that sum is not a meaningful
per-share number regardless of whether the cause was a split or an acquisition. It follows
the formula without being a valid result.

But the cost is real and worth stating plainly: **any issuer doing a large stock-funded
acquisition loses its P/E for four quarters.** That frequency has not been measured — it
needs a count of `|share_count_change_yoy| ≥ 25%` across the universe, which needs the
database. If it turns out to be common, the alternative is a higher or split-shaped
threshold, at the cost of letting more BKNG-class errors through.

## 2b is genuinely unfixed

KLAC's split post-dates its most recent 10-Q, so no snapshot carries any share-count
evidence and no guard built on share counts can fire. Its P/E is still 6.19 — the true P/E
divided by the split factor. I did not ship a heuristic for this: the obvious one, flagging
implausibly low P/Es, would misfire on genuinely cheap names — CHTR (3.42) and CMCSA (4.30)
sit below KLAC's corrupted 6.19 in this very report. Detecting it needs an actual
corporate-actions source, or a price-vs-share-count reconciliation against an external
market-cap reference.

---

# Fourth pass — the reparse path

Snapshots are immutable per accession, so the parser fixes never reached stored rows.
`promote()` skipped them and logged a discrepancy. Reparse is the deliberate exception:
immutability protects *SEC's* record, but the stored row is **our reconstruction** — after a
parser fix, keeping it is preserving a stale cache, not preserving history.

| change | file |
|---|---|
| `run_import(..., force=True)` bypasses the unchanged-revision no-op. The revision tracks the *source*; a fix on our side leaves it unchanged, so the gate would skip the run | `data_import.py` |
| `SecFundamentalsImporter(reparse=True)` — forces full-history staging, and `promote()` rewrites the accessions whose reconstruction changed, stamping `import_run_id` | `sec_fundamentals_importer.py` |
| `scripts/reparse_fundamentals.py` — **dry run by default**, `--apply` to write | new |

Unchanged rows are never touched; only accessions appearing in `staged.discrepancies` are
rewritten. The update writes the full `_SNAPSHOT_COLS` set via the same `_row_values()` the
insert uses, so a rewritten row can never be half old-parse and half new-parse. `created_at`
keeps its original value. Nothing is wired into the scheduler.

## A real bug the tests caught: false-positive discrepancies

`test_reparse_leaves_unchanged_rows_untouched` failed on first run — reparsing *identical*
data reported a change. Cause: `accepted_at` is written tz-aware UTC but
`DateTime(timezone=True)` only preserves tzinfo on Postgres; SQLite returns it naive, so
`_diff_fields` compared representations and saw a difference.

Left alone this would have made the dry-run report claim **every row needs rewriting** —
exactly the misleading signal that makes a blast-radius report worthless. `_diff_fields` now
compares datetime *instants* via `_same_value()`. This also fixes a latent false positive in
the pre-existing `snapshot_discrepancy` warning, which shares the same code path.

## Verification

4 reparse tests added, driven through the real import framework with the fake SEC client.
The key one seeds the database through the **pre-fix parser** (monkeypatching
`_YTD_TOLERANCE_DAYS` back to 20 so a 4-4-5 Q3 is rejected and stored as null), then reparses
with the fixed parser and asserts the row is rewritten in place with new provenance — the
production scenario end to end. Also covered: unchanged rows keep their original
`import_run_id`; `reparse=False` still reports and refuses to mutate; `force` bypasses the
no-op. Full suite: **804 passed**.

Not verifiable here: this reads and writes production Postgres, which is unreachable from
this machine, so the SQLite harness is the limit of what could be self-tested. The UPDATE is
plain SQLAlchemy Core with no dialect-specific constructs.

## Running it

```
python scripts/reparse_fundamentals.py            # dry run, writes nothing
python scripts/reparse_fundamentals.py --apply    # rewrite changed rows
```

Two cautions for whoever runs it:

- **Read the dry run for *kinds* of change, not just the count.** The tolerance 20→25 change
  newly accepts facts for arbitrary filers, not only the names investigated here. Sample
  changed rows for issuers that were never on the list and confirm they are recovered nulls
  and corrected values — not something unexpected.
- **It refetches Company Facts for every tracked issuer** under the SEC throttle, because the
  facts a fixed parser now accepts were never stored. Expect a long run; the dry run pays
  that cost too, so budget for two passes.

Scope: this rewrites `fundamental_snapshots` only. Those rows currently feed the fundamentals
API/UI and the parity report — scoring still reads the legacy `fundamental_data` table, and
nothing in the backtest path touches `FundamentalSnapshot`. So a reparse **cannot** move
composite scores or backtests until the A5 cutover happens. The plan's "changed history
changes backtests" caution applies to workstream B's OHLCV rewrites, not to this.

---

# Fifth pass — period identity

The parser's own stated rule was *"period identity comes from `end == reportDate`, never
`fy/fp`"* — but `_fiscal_context()` derived the stored `fiscal_year`/`fiscal_period` by
majority-voting exactly those fy/fp fields. The labelling contradicted the module's own
principle, and SEC's labels are unreliable enough to break the quarter chain.

`_period_identity()` now derives both from `period_end` against the issuer's
`submissions.fiscalYearEnd`: **the form decides FY vs quarter** (a 10-Q can no longer be
labelled FY), and **distance to the fiscal-year end decides which quarter**. The MMDD is
threaded through `parse_snapshots(..., fiscal_year_end=...)`; without it the old fy/fp path
is used unchanged, so nothing regresses for issuers lacking a calendar.

**Rejected approach:** classifying the period by fact spans. Every 10-Q carries both a YTD
*and* a discrete fact ending at reportDate, so "best span match" reads COST's Q2 (167d) as a
Q1; and taking the *longest* span mislabelled IRM's Q3 2020 10-Q as FY because that filing
carries a 12-month fact. The prototype caught this as a regression on a working name before
any code was written. Distance-to-year-end needs no facts at all and is unambiguous — the
quarter bands sit 91 days apart, so ±35 absorbs even a 4-4-5 filer's 16-week Q4.

**Labels no longer match issuer naming in one case, deliberately.** A filer whose year ends
in early January (DPZ, `fiscalYearEnd` 0102) shifts by one. That is harmless: `fiscal_year`
and `fiscal_period` appear nowhere in the API schemas or routers — they are internal keys the
derivation uses for ordering, YTD differencing and YoY pairing, and the API surfaces
`period_end`. The requirement is uniqueness, monotonicity and YoY alignment, not nomenclature.
DPZ's derived values are byte-identical before and after the shift, which is the proof.

## Prototype evidence (before implementing)

Collisions = two period ends on one key, one silently discarded. Inversions = a period
sorting before one that precedes it.

| | CRM | FRT | STX | BXP | PPL | MTD | NTAP | WDAY | CRWD | COST | PEP | IRM | DPZ | AMCR | AAPL |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| collisions before | 1 | 1 | 0 | 0 | 4 | 5 | 2 | 3 | 2 | 0 | 0 | 0 | 1 | 0 | 0 |
| collisions after | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| inversions before | 1 | 1 | 1 | 0 | 2 | 10 | 2 | 3 | 5 | 1 | 0 | 0 | 2 | 0 | 0 |
| inversions after | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

## Validated on live data

8 of the 9 recover fully, every one matching the legacy provider to two decimals:

| name | TTM EPS | revenue growth | legacy |
|---|---|---|---|
| CRM | 8.63 | 10.9818 | 10.98 |
| FRT | 5.77 | 7.4263 | 7.43 |
| STX | 10.54 | 28.9227 | 28.92 |
| BXP | 1.99 | 1.6227 | 1.62 |
| MTD | 42.57 | 6.7785 | 6.78 |
| NTAP | 6.35 | 5.3713 | 5.37 |
| WDAY | 3.21 | 13.3165 | 13.32 |
| CRWD | −0.10 | 23.1667 | 23.17 |

**PPL is partial**: revenue growth recovers (8.3353) but TTM EPS is still null — its 2026 Q1
tags no diluted EPS variant at all, which is the open basic-vs-diluted decision, not this bug.
Note legacy claims −58.81% revenue growth for a utility; 8.34% is far more plausible.

**Two bonus recoveries**: FCX and REG had recovered EPS in the fourth pass but their revenue
growth was still blocked by label collisions. REG now reads 7.7569 against legacy's 7.76.
FCX reads 5.4378 against legacy's −24.23 — a genuine disagreement, likely the same
`Revenues` vs ASC-606 ambiguity flagged for DVN in §2, and worth resolving with that decision.

**Regression check — all byte-identical:** IRM 0.92/15.637543, COST 19.88/9.231107,
PEP 7.63/5.619741, DPZ 17.64/5.157289, AMCR null/64.834349, JPM 20.89/3.338823,
DVN 3.59/0.095648, AZO 145.39/5.740494. Nothing that worked before moved.

7 tests added at `_period_identity` covering each production shape (10-Q-labelled-FY,
December collision, January and mid-year ends, 4-4-5 quarters, the January-crossing shift,
and the no-calendar fallback). Full suite: **811 passed**.

## Reparse note

This changes `fiscal_year`/`fiscal_period` for a large share of rows — every non-December
filer, not only the broken ones. The dry-run count will be **much** larger than for the
earlier fixes, and that is expected. Read it by field: `fiscal_year`/`fiscal_period` churn is
the intended relabelling; changes to *value* columns are the recoveries.

## Where the 25 stand now

22 of 25 have a fundamental score again. Remaining: **XOM** (CIK identity, still unfixed) and
**PSKY / Q**, which are new registrants without enough filing history — correct behaviour,
not a bug.

---

# Sixth pass — CIK identity, and a much larger finding about share counts

## XOM: pinned, plus the validation that should have caught it

`company_tickers.json` maps XOM to CIK 2115436 "ExxonMobil Holdings Corp", which has **zero
XBRL filings**, while every 10-K/10-Q — including one filed 2026-05-04 — is still under CIK
34088. Which registrant is the real filer is a judgement about a corporate event, so it is
**pinned explicitly** rather than guessed:

- `sec_universe.cik_overrides()` reads a `{symbol: cik}` JSON map from
  `SystemSetting['sec_cik_overrides']` and applies it ahead of `company_tickers.json`.
  A malformed setting is logged and ignored, never fatal.
- **To fix XOM, set:** `sec_cik_overrides = {"XOM": 34088}`.

The more valuable half is that nothing noticed. A tracked issuer resolving to a registrant
with no XBRL filings can never produce a snapshot, and is restaged on *every* run forever.
The importer now records those in `staged.no_xbrl_filings`, reports them in the validation
summary (`no_xbrl_filings_count`), and raises a `no_xbrl_filings` SystemEvent naming the CIKs
and pointing at the override setting. It warns rather than fails — one misresolved ticker
must not block the whole import.

3 tests added. Full suite: **814 passed**.

## CHTR was not a bug, and the real problem is much bigger

I previously called this "a separate small bug". Both halves were wrong.

CHTR's `dei:EntityCommonStockSharesOutstanding` facts stop at **2016-06-30** — exactly when
the Time Warner Cable / Bright House deal closed and Charter became a multi-class issuer.
Since then the cover page reports the count **per share class**, which is dimensional, and
companyfacts is non-dimensional — so the facts are simply not in the API. Its recent filings
tag no consolidated common-share concept at all, only preferred and treasury.

This is not specific to CHTR. Of 12 issuers checked, **7 have no share count at all**:

| issuer | latest `shares_outstanding` | dei fact history |
|---|---|---|
| META | null (4/4 recent) | **never tagged** (n=0) |
| CMCSA | null (4/4 recent) | stops 2009-12-31 |
| BRK-B | null (4/4 recent) | stops 2011-04-29 |
| CHTR | null (4/4 recent) | stops 2016-06-30 |
| FOXA, NWSA, LEN | null (4/4 recent) | — |
| GOOGL / GOOG | 12,230,000,000 | works via the `us-gaap` fallback |

So **market cap is silently unavailable for a meaningful slice of the large-cap universe**,
and it is a source limitation rather than a parser defect: the two obvious workarounds are
both already-rejected design decisions — class sums are impossible (the per-class facts are
not in companyfacts at all), and the weighted-average diluted count is explicitly excluded
because market cap needs a point-in-time value.

**No code change made.** Substituting weighted-average diluted shares would silently
overturn a deliberate design decision and produce a subtly wrong market cap for exactly the
biggest, most-watched names. That is a call to make explicitly, so it is listed as a decision
below rather than quietly implemented.

---

# Seventh pass — multi-class share counts (decision taken: weighted-average fallback)

## Why this fallback, and why not the alternatives

Two candidates existed. The one **not** taken: derive the count as
`net_income ÷ diluted_eps` from columns already stored — no migration at all, and measured
accurate (GOOGL +0.48%, MRNA −0.45%, AAPL +0.19%, MSFT +0.18%). Rejected because it depends
on the derived quarter chain — the very thing these fixes have been repairing, and FOXA
already fails it — and because the two-class EPS method makes `net_income` differ from the
EPS numerator for exactly the multi-class issuers this targets.

Taken instead: store the **reported** `WeightedAverageNumberOfDilutedSharesOutstanding`.
It is the number the filer computed, needs no chain, and covers one issuer more.

| control | point-in-time | wavg diluted (latest qtr) | ratio |
|---|---|---|---|
| GOOGL | 12,230,000,000 | 12,309,000,000 | 0.9936 |
| MRNA | 396,786,259 | 395,000,000 | 1.0045 |
| AAPL | 14,687,356,000 | 14,725,873,000 | 0.9974 |
| MSFT | 7,428,434,704 | 7,445,000,000 | 0.9978 |

## Shape of the change

- **Migration 027** adds `fundamental_snapshots.weighted_avg_diluted_shares`. A separate
  column, never backfilled into `shares_outstanding`, so the point-in-time column keeps its
  strict meaning and the fallback stays a read-time decision.
- **Parser** stores the **shortest**-span fact ending at `period_end` (the most recent
  quarter's average, closest to the current count) — deliberately not the YTD one, since an
  average is not cumulative and the YTD convention does not apply.
- **Derivation** falls back only when the cover-page count is absent, and sets
  `shares_outstanding_estimated`.
- **API** exposes `shares_estimated`, so `market_cap_est` and `fcf_yield` are never presented
  as exact when they rest on a period average.

## Validated on live data

| issuer | shares_outstanding | estimated |
|---|---|---|
| GOOGL, AAPL, MSFT, MRNA | unchanged point-in-time values | **False** |
| META | 2,564,000,000 | True |
| CMCSA | 3,570,000,000 | True |
| CHTR | 126,849,271 | True |
| FOXA | 432,000,000 | True |
| NWSA | 555,700,000 | True |
| LEN | 240,776,000 | True |
| **BRK-B** | **still null** | False |

6 of 7 recovered, no regression on the controls. **BRK-B remains unavailable** and honestly
so: Berkshire reports per *equivalent Class A share*, dimensionally, so it has no consolidated
weighted-average fact either. Nothing in companyfacts can give it a share count.

Known caveat, accepted: for issuers using the two-class method the count is the EPS
denominator. For CHTR that is Class A only — which is also the basis on which Charter's equity
market cap is normally quoted, so it is the right number for this purpose, but it is not
"all shares of all classes".

3 tests added. Full suite: **817 passed**. Alembic single head at 027.

**Needs the reparse to land:** existing rows have `weighted_avg_diluted_shares = NULL` until
`scripts/reparse_fundamentals.py --apply` runs, so market cap stays missing for these issuers
until then.

---

# Eighth pass — revenue basis (decision: keep ASC-606, no change)

The two concepts measure different things: `RevenueFromContractWithCustomerExcludingAssessedTax`
is customer-contract revenue (an E&P's oil/gas/NGL sales), while `Revenues` is the total
income-statement line, which for commodity producers folds in mark-to-market derivative
gains/losses. That is why DVN's ASC-606 figure is *larger*: 4,508M of sales minus ~701M of
hedging losses gives the 3,807M `Revenues` line.

Measured across 21 issuers (deliberately energy-weighted, where the gap concentrates):

- Both tags present and differing >1%: **5 of 21** — DVN +18.4%, COP −14.3%, OXY +6.5%,
  FCX −2.8%, PPL +1.6%. Everyone else tags one, or they are identical (COST +0.0%).
- Concept choice **flips within an issuer's chain: 0 of 21**. Whichever tag wins, the series
  is internally consistent, so YoY never compares two definitions.

**Decision: keep ASC-606 first, change nothing.** Derivative gains/losses are mean-reverting
and sign-flipping; folding them into "revenue growth" turns the sub-score into a partial
hedging-P&L read for exactly the affected names. The consistency argument for switching is
empirically absent (zero flips), and changing would churn every dual-tagging issuer's stored
value — widening the reparse diff — to make ~5 names noisier.

**Correction to the fourth/fifth-pass note:** FCX's disagreement with legacy (+5.44% vs
−24.23%) is **not** this ambiguity. Its two tags differ by only 2.8%, and FCX's own revenue
rose 22,703M → 25,186M YoY, so −24% is not credible — legacy is simply wrong there, and this
decision does not touch it. So the basis choice moves only DVN, COP, OXY.

The mirror hazard — an issuer where ASC-606 is only a *fragment* of revenue (a bank's fee
income) — was checked (all 15 recovered banks resolve total revenue, not a fragment). A
fragment-detection warning was prototyped and then **removed**: with no UI surface it would
only have lived in the run summary, and the case it guards against is not currently present.
Documented and closed rather than shipped as dead plumbing. If a fragment case ever appears,
it shows up as an implausibly low revenue in the next parity report.

---

# Ninth pass — basic-EPS fallback (PPL) and HAL resolved

## PPL: basic-EPS fallback (decision taken)

PPL's 2026 Q1 tags no diluted EPS variant at all, only basic — a single-filing omission
(its other quarters tag diluted), but that one missing period broke the quarter chain and
nulled TTM. `EarningsPerShareBasic` / `IncomeLossFromContinuingOperationsPerBasicShare` are
now appended to `_EPS_CONCEPTS`, last, so they only fire when no diluted variant exists.

Evidence (19-name scan): a basic fallback helps exactly **1 name (PPL)**. Basic-vs-diluted is
~0.5–1.2% for most, +1.2% for PPL. The one name where it genuinely diverges (TSLA +13.3%)
already tags diluted, so it never reaches the fallback. Basic is always ≥ diluted, so the
result slightly overstates EPS / understates P/E — accepted, since it fires only on an
otherwise-null period.

Validated: PPL TTM EPS null → **1.63** (≈$36 / 1.63 = 22.1 vs legacy P/E 22.43). AAPL, MSFT,
DUK, HAL unchanged — diluted still wins wherever present. 2 tests added. Full suite: **819
passed**.

## HAL: resolved, and it was never our bug

HAL's TTM EPS is now **1.81** (≈$33 / 1.81 = 18.2 vs legacy P/E 18.01) — the period-identity
and EPS-concept work already fixed it. The "unexplained null" is closed.

Its 2024 EPS values are garbage (680000, 1480000, …) because **Halliburton's own 2024 XBRL
tags `EarningsPerShareDiluted = 680000` in unit USD/shares** — a filer scale error in the
source, faithfully stored. It only poisons TTM windows that include 2024, which the current
point-in-time report does not use, so no code change: clamping EPS to "plausible" values would
risk masking real ones. Documented as a known source-data quirk.

This does surface a latent robustness point (not acted on): a single fat-fingered per-share
value poisons any TTM window it lands in. It is invisible in the current report and out of
scope here, but worth a note if historical TTM series are ever surfaced.

## All 25 lost names accounted for

| status | names |
|---|---|
| **recovered** (22) | ARE, AZO, BXP, COST, CRM, CRWD, DPZ, DVN, FCX, FRT, HAL, KHC, MOS, MTD, NTAP, PEP, PPL, REG, SJM, STX, SWKS, WDAY |
| **XOM** | fixed by the `sec_cik_overrides` pin (needs the setting applied) |
| **PSKY, Q** | new registrants without enough filing history — correct behaviour, not a bug |

## Still outstanding

Revised after the second pass, in the order I would take them:

Everything actionable without a live database is now done. What remains is one hard
data limitation and two operational steps that only run against production.

1. ~~**§3a fiscal-period identity**~~ — **done**, fifth pass.
2. ~~**§1 split contamination, part (a)**~~ — **done**, third pass.
3. ~~**§5 EPS concept gap**~~ — **done**, third pass.
4. ~~**§4 XOM CIK remap** + zero-filings validation~~ — **done**, sixth pass.
5. ~~**Reparse path**~~ — **done**, fourth pass.
6. ~~**Multi-class share counts**~~ — **done**, seventh pass (weighted-average fallback).
7. ~~**DVN/FCX revenue basis**~~ — **decided**, eighth pass (keep ASC-606, no change).
8. ~~**PPL basic-EPS fallback**~~ — **done**, ninth pass.
9. ~~**HAL null TTM EPS**~~ — **resolved**, ninth pass (already fixed; 2024 is a filer error).

## Genuinely unfixable from this data

- **§1 part (b)** — a split post-dating the last filing (KLAC). No snapshot carries
  share-count evidence, so no guard built on share counts can fire. Needs a corporate-actions
  source or an external market-cap reconciliation.
- **BRK-B market cap** — Berkshire reports per equivalent Class A share, dimensionally, so it
  has neither a cover-page count nor a weighted-average one. Nothing in companyfacts can give
  it a share count.

## Operational steps (production only — cannot run from here)

- Apply the setting `sec_cik_overrides = {"XOM": 34088}`.
- Run `scripts/reparse_fundamentals.py` — dry run first, then `--apply`. This is what lands
  every parser-side fix (revenue/EPS concepts, Q3 span, period identity, weighted-average
  shares via migration 027) onto existing rows. Until it runs, those fixes are inert in prod.

## Standing decision, revisit only if it bites

- **COF-class share-change threshold** — kept at 25%. Revisit only if the 3.2% universe
  hit-rate proves painful.

## Known source-data quirk, not acted on

- A single fat-fingered per-share value in a filer's XBRL (HAL 2024) poisons any TTM window
  it lands in. Invisible in the current point-in-time report; relevant only if historical TTM
  series are ever surfaced.

PSKY and Q need nothing — they are new registrants without enough filing history, which is
correct behaviour.