dennisthiessenandClaude Opus 5
c15b51439e
snapshot_discrepancy named the accessions but not the columns, so it could not distinguish "our numbers moved" from "the same filing is attributed twice". The fields were already computed for validation_json and simply dropped from the message; they are now in it. A difference in cik ALONE is no longer reported as a reconstruction change at all. Every fact matched, so two tracked CIKs are claiming one filing and the fix is the universe, not the parser: it raises accession_cik_collision naming both CIKs and sec_cik_overrides. It also never self-heals — the losing CIK stores no row, so _ciks_with_snapshots never sees it and it is full-history backfilled and re-reported every run until its ticker is re-pointed or retired. Observed 2026-08-19 for EQR: after Equity Residential renamed to Vivmark Residential (VMRK, CIK 906107), SEC's own company_tickers.json left the old symbol on ERP Operating LP (CIK 931182), the non-traded co-registrant of their combined 10-Qs. Both were tracked, both reconstructed the same two filings. The reparse path now excludes cik-only differences from its rewrite set: rewriting one would re-stamp the filing onto the co-registrant, taking it from the issuer that actually filed it, which no parser fix asks for. No stored value was wrong in that incident — reports/ carries the full reproduction for this and for the companyfacts staleness behind the gate fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Lo99z3jWqu9X9ueBU3Z3D
151 lines
9.2 KiB
Markdown
151 lines
9.2 KiB
Markdown
# SEC fundamentals alerts, 2026-08-21
|
||
|
||
Two `sec_facts` warnings, investigated against live SEC data. Both originate in SEC's
|
||
own published data — a stale per-company Company-Facts file (1) and a stale
|
||
ticker→CIK mapping (2) — and neither is a parser defect: no stored fundamental value
|
||
is wrong. Every SEC-side probe below reproduces offline from public endpoints; the
|
||
four database facts used are quoted where they appear.
|
||
|
||
## 1. `filing_gap_aged` — 43 gaps, all `not_in_companyfacts`
|
||
|
||
**Root cause: SEC's per-company Company-Facts files are stale for these issuers,
|
||
while the same filings are present in SEC's own `frames` aggregation.**
|
||
|
||
All ten named filings are real 10-Qs filed 2026-07-28/29, present in the issuer's
|
||
`submissions` with `isXBRL=1`, with complete R-files and XBRL in the EDGAR archive
|
||
— and absent from `companyfacts/CIK*.json`:
|
||
|
||
| CIK | issuer | accession | filed | in `companyfacts` | newest fact in file |
|
||
|---|---|---|---|---|---|
|
||
| 0000001800 | Abbott | 0001628280-26-050134 | 2026-07-28 | no | 2026-04-29 |
|
||
| 0000021344 | Coca-Cola | 0001628280-26-050503 | 2026-07-29 | no | 2026-04-30 |
|
||
| 0000024741 | Corning | 0000024741-26-000255 | 2026-07-29 | no | 2026-05-01 |
|
||
| 0000029989 | Omnicom | 0000029989-26-000019 | 2026-07-29 | no | 2026-04-29 |
|
||
| 0000037996 | Ford | 0000037996-26-000156 | 2026-07-29 | no | 2026-04-30 |
|
||
| 0000040533 | General Dynamics | 0000040533-26-000032 | 2026-07-29 | no | 2026-07-01 |
|
||
| 0000048898 | Hubbell | 0001628280-26-050405 | 2026-07-29 | no | 2026-06-04 |
|
||
| 0000049071 | Humana | 0000049071-26-000050 | 2026-07-29 | no | 2026-04-29 |
|
||
| 0000049196 | Huntington Bancshares | 0000049196-26-000066 | 2026-07-28 | no | 2026-04-30 |
|
||
| 0000062996 | Masco | 0000062996-26-000027 | 2026-07-29 | no | 2026-04-22 |
|
||
|
||
Ruled out, with evidence:
|
||
|
||
- **Not a global SEC outage.** Company Facts is current for other issuers filing the
|
||
same days — MSFT `0001193125-26-323660` @2026-07-29, AAPL @2026-07-31, P&G
|
||
@2026-08-04, Chevron @2026-08-06, JPMorgan @2026-08-20.
|
||
- **Not a CDN/cache artifact.** A cache-busted request with `Cache-Control: no-cache`
|
||
returns the identical stale 3.39 MB payload; the response carries no cache headers.
|
||
- **Not our filter.** The scan covers every taxonomy/concept/unit in the payload.
|
||
- **Not a metadata discriminator.** Gap and non-gap filings are identical on
|
||
`isXBRL`, `isInlineXBRL`, `reportDate`, `primaryDocDescription`.
|
||
- **SEC does have the facts.** `frames/us-gaap/Assets/USD/CY2026Q2I.json` lists
|
||
Abbott at exactly the missing accession `0001628280-26-050134`, and Coca-Cola and
|
||
Ford at theirs. The per-company endpoints are the degraded ones:
|
||
`companyconcept/CIK0000001800/us-gaap/Assets.json` returns `"units":{"USD":{}}`.
|
||
|
||
**Consequence, and why the gate changed.** Retrying `companyfacts` cannot recover
|
||
these — Abbott's file has been stale since April. And because `active_gaps`
|
||
supersedes a gap only on a *successfully ingested later* filing, a stale file also
|
||
swallows Q3: the pause was open-ended, not seasonal, on 43 large caps.
|
||
|
||
**Fix** (`app/services/fundamentals_quality_service.py`): once `filing_gap_aged` has
|
||
escalated a gap (`escalated_at`), it stops pausing setups **if** the issuer's own
|
||
newest stored 10-K/10-Q is under `GAP_GATE_RECENT_FILING_DAYS` (180) old. Pause hands
|
||
off to the alert; an issuer with nothing that recent stays paused. `active_gaps` is
|
||
deliberately untouched, so `_retry_backlog` keeps retrying and a recovered filing
|
||
still resolves normally. The bound is applied to the queue path *and* the
|
||
`validation_json` summary path, which mirrors the same filings — bounding only one
|
||
leaves the behaviour unchanged in production.
|
||
|
||
**This is a bounded reprieve, not a removal — know the two ways it ends.** Abbott's
|
||
newest ingested filing is `0001628280-26-028357`, filed 2026-04-29, so its recency
|
||
window closes around **2026-10-26**; most of the 43 sit on late-April filings and
|
||
turn back to paused within days of each other. That crossing is **silent**: the
|
||
importer escalates only gaps with `escalated_at IS NULL`, so `filing_gap_aged` does
|
||
not re-fire for a gap it has already reported. Separately, a Q3 10-Q that also fails
|
||
to ingest creates a *new* un-escalated gap on the same CIK, which re-pauses it at
|
||
once (that one does raise its own `filing_gap_aged` 14 days later). Whether the
|
||
silent re-block deserves a re-escalation signal is an open call, deliberately not
|
||
made here — "one actionable escalation rather than a daily warning" is the existing
|
||
design intent.
|
||
|
||
**Not done, with reasons.** A `frames`-backed recovery source was considered and
|
||
rejected: frames are calendar-aligned with a tolerance (off-fiscal filers drop out)
|
||
and carry one fact per issuer per period, so amendment/restatement semantics differ
|
||
from Company Facts — lossy as a snapshot source, not merely expensive. Parsing the
|
||
filing's own inline-XBRL instance is the authoritative alternative but is a new
|
||
subsystem (contexts, dimensions, unit refs) duplicating the parser's fact model.
|
||
|
||
## 2. `snapshot_discrepancy` — 0000906107-15-000012 / -000016
|
||
|
||
**Root cause: two tracked tickers claim the same filing, because SEC's
|
||
`company_tickers.json` still points the old symbol at a non-traded co-registrant.
|
||
No stored value is wrong and no reparse is warranted.**
|
||
|
||
CIK 0000906107 is **Vivmark Residential** (VMRK, formerly Equity Residential). Both
|
||
alerted accessions are **combined EQR + ERP Operating LP 10-Qs** — one accession, two
|
||
registrants (0000906107 and 0000931182) — the pattern behind the existing
|
||
co-registrant recovery path.
|
||
|
||
The stored rows are **byte-identical** to what the current parser reconstructs from
|
||
EQR's own Company Facts — every column, verified: `cik` (`0000906107`), `form`,
|
||
`filed_date`, `accepted_at`, both period dates, `fiscal_year`/`fiscal_period`,
|
||
`revenue`, `net_income`, `operating_income`, `diluted_eps`, `cfo`, the two nulls,
|
||
`cash_and_st_investments`, `total_debt` (340,900,000 / null),
|
||
`shares_outstanding`, `shares_outstanding_date`, `weighted_avg_diluted_shares`. Both
|
||
carry `import_run_id = 6`, and CIK 0000906107 holds all 69 of its filings across runs
|
||
6–30, so the issuer's own history is complete.
|
||
|
||
Run 63 (2026-08-19) recorded
|
||
`fields: ["cik"]` for both accessions, and the universe explains it:
|
||
|
||
```
|
||
tickers: VMRK -> 0000906107 (Vivmark Residential, ex-Equity Residential)
|
||
EQR -> 0000931182 (ERP Operating Ltd Partnership)
|
||
```
|
||
|
||
SEC's own `company_tickers.json` carries `{"cik_str": 931182, "ticker": "EQR",
|
||
"title": "ERP OPERATING LTD PARTNERSHIP"}` — after the rename, the old symbol stayed
|
||
attached to the **non-traded operating partnership**, the co-registrant on those
|
||
combined 10-Qs. `resolve_ciks` reads `active_only` tickers and follows SEC, so
|
||
0000931182 is tracked. Its Company Facts holds 7 accessions, exactly 2 of them
|
||
EQR-prefixed, so its backfill reconstructs exactly those two rows, stamps them
|
||
`cik=0000931182`, and collides with the rows already stored under 0000906107 —
|
||
identical in every fact, differing only in attribution.
|
||
|
||
It cannot self-heal. The collision loser never stores a row (the insert is skipped as
|
||
immutable), so `_ciks_with_snapshots` never sees 0000931182, and it is full-history
|
||
backfilled — refetching every submissions shard and its companyfacts — **on every
|
||
run**, re-raising the warning each time. `fundamental_snapshots` for 0000906107 holds
|
||
all 69 filings across runs 6–30, so the issuer's own history is complete and correct.
|
||
|
||
### Fixes
|
||
|
||
**Code** (`sec_fundamentals_importer.py`): a `cik`-only difference is no longer
|
||
reported as a reconstruction discrepancy. It raises `accession_cik_collision`, naming
|
||
both CIKs and pointing at `sec_cik_overrides`, because the fix is the universe, not
|
||
the parser. The reparse path also excludes these from its rewrite set — rewriting a
|
||
cik-only difference would re-stamp the filing onto the co-registrant and take it from
|
||
the issuer that filed it. (A reparse run while both CIKs are tracked fails validation
|
||
on `duplicate accession in staged snapshots` instead, which is a safe stop.)
|
||
|
||
**Data — needs an operator, and the alert repeats daily until then.** `EQR` is a stale
|
||
symbol: the security now trades as `VMRK`, which is already tracked at the correct
|
||
CIK. Retiring the `EQR` ticker ends the loop. A `sec_cik_overrides` pin of
|
||
`EQR -> 906107` would silence the collision but leave two tickers on one security,
|
||
double-counting the issuer in scans — retirement is the right action.
|
||
|
||
**Not fixed, deliberately:** the permanent-backfill loop itself. A tracked CIK whose
|
||
only parseable filings belong to another CIK is re-backfilled every run; ending that
|
||
in code means teaching `_ciks_with_snapshots` about foreign-owned accessions, which is
|
||
more state for a condition that is now loudly and specifically reported.
|
||
|
||
### Separate observation: `total_debt` on this issuer looks wrong
|
||
|
||
Independent of the alert, and unchanged by any fix here: the parser reconstructs
|
||
`total_debt = 340,900,000` for EQR's 2015 Q1 and `null` for Q2, while the REIT carried
|
||
roughly $10bn of debt. `_compose_debt` returns the short-term component alone when
|
||
every `_LONG_TERM_DEBT_AGG` concept **and** the `LongTermDebtNoncurrent`/`Current`
|
||
pair miss — which is what happened here, and Q2 matched neither. Worth checking
|
||
against a current REIT filer before trusting `total_debt` for that sector.
|