Files
signal-platform/reports/sec-companyfacts-stale-20260821-findings.md
dennisthiessenandClaude Opus 5 8453b87290
Deploy / lint (push) Successful in 10s
Deploy / test (push) Successful in 1m20s
Deploy / deploy (push) Successful in 37s
fix(sec): compose total debt across the styles filers actually tag
total_debt read LongTermDebt, else LongTermDebtNoncurrent/Current, plus one of
ShortTermBorrowings/CommercialPaper. That misses two whole tagging styles, and
it feeds net_debt -> net_debt_to_ebitda -> the categorical leverage read, so the
misses were not absences but confident wrong answers: Coca-Cola scored on 0.25bn
of commercial paper against ~39bn of debt, Verizon on 21.78bn of current
maturities against ~165bn, AT&T and Exxon produced no value at all against 134bn
and 33bn tagged. Measured over 19 large caps and 14 REITs, 11 were wrong or
absent and the rest are unchanged.

Each concept's span is now respected. LongTermDebt already includes current
maturities (Apple tags all three: 71.34 + 11.01 = 82.30), so only true
short-term borrowing is added. LongTermDebtAndCapitalLeaseObligations — what KO,
HD, T, XOM and CVX tag, and nothing read before — is noncurrent and takes a
current complement, and DebtCurrent *is* that whole complement rather than an
addition to it.

The REIT branch needed disambiguating: NotesPayable is not the same line across
issuers. MAA tags NotesPayable 5.66bn = UnsecuredDebt 5.30bn + SecuredDebt
0.36bn exactly, so there it is the total and adding the secured side
double-counts; EQR tags it alongside a larger SecuredDebt, where it is only the
unsecured component. UnsecuredDebt's presence separates them.

A component alone is no longer reported as a total. Chevron tags full debt only
in its 10-K, so its 10-Q carried 0.40bn of short-term borrowing; Boston
Properties tags SecuredDebt 4.28bn against ~15bn real. net_debt needs both sides
and yields nothing when either is missing, so None costs a leverage read where
the fragment produced a confidently wrong one.

Snapshots are immutable, so this corrects new filings only; stored history needs
scripts/reparse_fundamentals.py, which cannot complete until the EQR/931182
collision is retired.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Lo99z3jWqu9X9ueBU3Z3D
2026-08-21 17:44:18 +02:00

243 lines
14 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SEC fundamentals alerts, 2026-08-21
Two `sec_facts` warnings, investigated against live SEC data. Both originate in SEC's
own published data — a stale per-company Company-Facts file (1) and a stale
ticker→CIK mapping (2) — and neither is a parser defect: no stored fundamental value
is wrong. Every SEC-side probe below reproduces offline from public endpoints; the
four database facts used are quoted where they appear.
## 1. `filing_gap_aged` — 43 gaps, all `not_in_companyfacts`
**Root cause: SEC's per-company Company-Facts files are stale for these issuers,
while the same filings are present in SEC's own `frames` aggregation.**
All ten named filings are real 10-Qs filed 2026-07-28/29, present in the issuer's
`submissions` with `isXBRL=1`, with complete R-files and XBRL in the EDGAR archive
— and absent from `companyfacts/CIK*.json`:
| CIK | issuer | accession | filed | in `companyfacts` | newest fact in file |
|---|---|---|---|---|---|
| 0000001800 | Abbott | 0001628280-26-050134 | 2026-07-28 | no | 2026-04-29 |
| 0000021344 | Coca-Cola | 0001628280-26-050503 | 2026-07-29 | no | 2026-04-30 |
| 0000024741 | Corning | 0000024741-26-000255 | 2026-07-29 | no | 2026-05-01 |
| 0000029989 | Omnicom | 0000029989-26-000019 | 2026-07-29 | no | 2026-04-29 |
| 0000037996 | Ford | 0000037996-26-000156 | 2026-07-29 | no | 2026-04-30 |
| 0000040533 | General Dynamics | 0000040533-26-000032 | 2026-07-29 | no | 2026-07-01 |
| 0000048898 | Hubbell | 0001628280-26-050405 | 2026-07-29 | no | 2026-06-04 |
| 0000049071 | Humana | 0000049071-26-000050 | 2026-07-29 | no | 2026-04-29 |
| 0000049196 | Huntington Bancshares | 0000049196-26-000066 | 2026-07-28 | no | 2026-04-30 |
| 0000062996 | Masco | 0000062996-26-000027 | 2026-07-29 | no | 2026-04-22 |
Ruled out, with evidence:
- **Not a global SEC outage.** Company Facts is current for other issuers filing the
same days — MSFT `0001193125-26-323660` @2026-07-29, AAPL @2026-07-31, P&G
@2026-08-04, Chevron @2026-08-06, JPMorgan @2026-08-20.
- **Not a CDN/cache artifact.** A cache-busted request with `Cache-Control: no-cache`
returns the identical stale 3.39 MB payload; the response carries no cache headers.
- **Not our filter.** The scan covers every taxonomy/concept/unit in the payload.
- **Not a metadata discriminator.** Gap and non-gap filings are identical on
`isXBRL`, `isInlineXBRL`, `reportDate`, `primaryDocDescription`.
- **SEC does have the facts.** `frames/us-gaap/Assets/USD/CY2026Q2I.json` lists
Abbott at exactly the missing accession `0001628280-26-050134`, and Coca-Cola and
Ford at theirs. The per-company endpoints are the degraded ones:
`companyconcept/CIK0000001800/us-gaap/Assets.json` returns `"units":{"USD":{}}`.
**Consequence, and why the gate changed.** Retrying `companyfacts` cannot recover
these — Abbott's file has been stale since April. And because `active_gaps`
supersedes a gap only on a *successfully ingested later* filing, a stale file also
swallows Q3: the pause was open-ended, not seasonal, on 43 large caps.
**Fix** (`app/services/fundamentals_quality_service.py`): once `filing_gap_aged` has
escalated a gap (`escalated_at`), it stops pausing setups **if** the issuer's own
newest stored 10-K/10-Q is under `GAP_GATE_RECENT_FILING_DAYS` (180) old. Pause hands
off to the alert; an issuer with nothing that recent stays paused. `active_gaps` is
deliberately untouched, so `_retry_backlog` keeps retrying and a recovered filing
still resolves normally. The bound is applied to the queue path *and* the
`validation_json` summary path, which mirrors the same filings — bounding only one
leaves the behaviour unchanged in production.
**This is a bounded reprieve, not a removal — know the two ways it ends.** Abbott's
newest ingested filing is `0001628280-26-028357`, filed 2026-04-29, so its recency
window closes around **2026-10-26**; most of the 43 sit on late-April filings and
turn back to paused within days of each other. That crossing is **silent**: the
importer escalates only gaps with `escalated_at IS NULL`, so `filing_gap_aged` does
not re-fire for a gap it has already reported. Separately, a Q3 10-Q that also fails
to ingest creates a *new* un-escalated gap on the same CIK, which re-pauses it at
once (that one does raise its own `filing_gap_aged` 14 days later). Whether the
silent re-block deserves a re-escalation signal is an open call, deliberately not
made here — "one actionable escalation rather than a daily warning" is the existing
design intent.
**Not done, with reasons.** A `frames`-backed recovery source was considered and
rejected: frames are calendar-aligned with a tolerance (off-fiscal filers drop out)
and carry one fact per issuer per period, so amendment/restatement semantics differ
from Company Facts — lossy as a snapshot source, not merely expensive. Parsing the
filing's own inline-XBRL instance is the authoritative alternative but is a new
subsystem (contexts, dimensions, unit refs) duplicating the parser's fact model.
## 2. `snapshot_discrepancy` — 0000906107-15-000012 / -000016
**Root cause: two tracked tickers claim the same filing, because SEC's
`company_tickers.json` still points the old symbol at a non-traded co-registrant.
No stored value is wrong and no reparse is warranted.**
CIK 0000906107 is **Vivmark Residential** (VMRK, formerly Equity Residential). Both
alerted accessions are **combined EQR + ERP Operating LP 10-Qs** — one accession, two
registrants (0000906107 and 0000931182) — the pattern behind the existing
co-registrant recovery path.
The stored rows are **byte-identical** to what the current parser reconstructs from
EQR's own Company Facts — every column, verified: `cik` (`0000906107`), `form`,
`filed_date`, `accepted_at`, both period dates, `fiscal_year`/`fiscal_period`,
`revenue`, `net_income`, `operating_income`, `diluted_eps`, `cfo`, the two nulls,
`cash_and_st_investments`, `total_debt` (340,900,000 / null),
`shares_outstanding`, `shares_outstanding_date`, `weighted_avg_diluted_shares`. Both
carry `import_run_id = 6`, and CIK 0000906107 holds all 69 of its filings across runs
630, so the issuer's own history is complete.
Run 63 (2026-08-19) recorded
`fields: ["cik"]` for both accessions, and the universe explains it:
```
tickers: VMRK -> 0000906107 (Vivmark Residential, ex-Equity Residential)
EQR -> 0000931182 (ERP Operating Ltd Partnership)
```
SEC's own `company_tickers.json` carries `{"cik_str": 931182, "ticker": "EQR",
"title": "ERP OPERATING LTD PARTNERSHIP"}` — after the rename, the old symbol stayed
attached to the **non-traded operating partnership**, the co-registrant on those
combined 10-Qs. `resolve_ciks` reads `active_only` tickers and follows SEC, so
0000931182 is tracked. Its Company Facts holds 7 accessions, exactly 2 of them
EQR-prefixed, so its backfill reconstructs exactly those two rows, stamps them
`cik=0000931182`, and collides with the rows already stored under 0000906107 —
identical in every fact, differing only in attribution.
It cannot self-heal. The collision loser never stores a row (the insert is skipped as
immutable), so `_ciks_with_snapshots` never sees 0000931182, and it is full-history
backfilled — refetching every submissions shard and its companyfacts — **on every
run**, re-raising the warning each time. `fundamental_snapshots` for 0000906107 holds
all 69 filings across runs 630, so the issuer's own history is complete and correct.
### Fixes
**Code** (`sec_fundamentals_importer.py`): a `cik`-only difference is no longer
reported as a reconstruction discrepancy. It raises `accession_cik_collision`, naming
both CIKs and pointing at `sec_cik_overrides`, because the fix is the universe, not
the parser. The reparse path also excludes these from its rewrite set — rewriting a
cik-only difference would re-stamp the filing onto the co-registrant and take it from
the issuer that filed it. (A reparse run while both CIKs are tracked fails validation
on `duplicate accession in staged snapshots` instead, which is a safe stop.)
**Data — needs an operator, and the alert repeats daily until then.** `EQR` is a stale
symbol: the security now trades as `VMRK`, which is already tracked at the correct
CIK. Retiring the `EQR` ticker ends the loop. A `sec_cik_overrides` pin of
`EQR -> 906107` would silence the collision but leave two tickers on one security,
double-counting the issuer in scans — retirement is the right action.
**Not fixed, deliberately:** the permanent-backfill loop itself. A tracked CIK whose
only parseable filings belong to another CIK is re-backfilled every run; ending that
in code means teaching `_ciks_with_snapshots` about foreign-owned accessions, which is
more state for a condition that is now loudly and specifically reported.
### Separate observation: `total_debt` on this issuer looks wrong
Independent of the alert, and unchanged by any fix here: the parser reconstructs
`total_debt = 340,900,000` for EQR's 2015 Q1 and `null` for Q2, while the REIT carried
roughly $10bn of debt. `_compose_debt` returns the short-term component alone when
every `_LONG_TERM_DEBT_AGG` concept **and** the `LongTermDebtNoncurrent`/`Current`
pair miss — which is what happened here, and Q2 matched neither. Worth checking
against a current REIT filer before trusting `total_debt` for that sector.
---
## 3. Follow-ups from the two alerts above
### 3a. The reprieve in (1) ended silently — now it doesn't
The hand-off in section 1 is a **bounded** reprieve. It ends two ways, and neither
said anything: the issuer's stored filings age past `GAP_GATE_RECENT_FILING_DAYS`
(for the 43, their last good filings are late April, so ~2026-10-26), or a newer
filing gap arrives and the all-escalated condition fails. `filing_gap_aged` cannot
report either, because it only escalates gaps whose `escalated_at` is NULL and so
never fires twice for the same gap.
`sec_filing_gaps.exempted_at` (migration `034`) makes the transition observable: set
quietly while the issuer is exempt, cleared when the exemption lapses, and the clear
is what raises `filing_gap_repaused`. Once per lapse, re-arming if the issuer's data
recovers and ages out again. A gap that was never exempt has no transition and stays
silent — it is simply still paused, which `filing_gap_aged` already said.
The exemption rule itself is not duplicated: `fundamentals_quality_service.gap_exempt_ciks`
is now public and the importer alerts on membership changes in exactly the set the
gate reads.
### 3b. `total_debt` was materially wrong for a third of large caps
The EQR observation in section 2 was not a REIT edge case. Measured over 19 large
caps, the old composition — `LongTermDebt`, else `LongTermDebtNoncurrent`/`Current`,
plus one of `ShortTermBorrowings`/`CommercialPaper` — missed two whole tagging styles:
| issuer | before | after | what was missed |
|---|---:|---:|---|
| T | None | 143.95b | `LongTermDebtAndCapitalLeaseObligations` |
| XOM | None | 47.66b | same |
| VZ | 21.78b | 165.23b | same (read only the current maturities) |
| KO | 0.25b | 39.31b | same (read only commercial paper) |
| HD | 3.50b | 48.33b | same |
| O | 1.40b | 26.53b | REIT parts (`NotesPayable` + `SecuredDebt`) |
| VMRK | 1.50b | 9.09b | same |
| CVX | 0.40b | **None** | partial suppressed — see below |
| PFE | 63.10b | 63.19b | `DebtCurrent` is the completer current side |
| 10 others | — | unchanged | already composed correctly |
`total_debt` feeds `net_debt``net_debt_to_ebitda` → the peer percentile and the
categorical leverage read, so Coca-Cola at 0.25bn of debt was not a missing value —
it was a confident *"conservative leverage"* on an issuer carrying ~39bn.
The composition now spans four mutually exclusive styles, with each concept's span
respected: `LongTermDebt` already includes current maturities (Apple tags all three
and 71.34 + 11.01 = 82.30 confirms it), `LongTermDebtAndCapitalLeaseObligations` is
noncurrent and needs a current complement, and `DebtCurrent` *is* that whole
complement rather than an addition to it.
**A short-term component alone is no longer reported as a total.** Chevron tags full
debt only in its 10-K, so its 10-Q carries 0.40bn of short-term borrowing and nothing
else. `_net_debt` needs both sides and yields nothing when either is missing, so None
costs a leverage read where the partial value produced a confidently wrong one.
The REIT branch needed disambiguating, because `NotesPayable` does not mean the same
thing across issuers (measured over 14 REITs): MAA tags `NotesPayable` 5.66bn =
`UnsecuredDebt` 5.30bn + `SecuredDebt` 0.36bn **exactly**, so there it is the total and
adding the secured side double-counts — while EQR tags it alongside a *larger*
`SecuredDebt` (5.38bn vs 6.38bn in 2013), where it is only the unsecured component.
`UnsecuredDebt`'s presence separates the two: where tagged it is the unambiguous
unsecured side and `NotesPayable` is ignored; where absent, `NotesPayable` is that
side. Both sides are required, which is also what stops the branch inventing a total
from a fragment.
| REIT | before | after | |
|---|---:|---:|---|
| MAA | None | 5.66b | matches its own `NotesPayable` total exactly |
| KIM | None | 8.74b | |
| O / VMRK | 1.40b / 1.50b | 26.53b / 9.09b | |
| BXP | 0.75b | **None** | tagged only `SecuredDebt` + paper against ~15bn real debt |
| VTR | 0.27b | **None** | same shape |
| 8 others | — | unchanged | already composed correctly |
Known limit: where EQR tags both the parts and the aggregate, the parts sum 2.612.2%
*below* it, so this branch approximates. It is last in line — any issuer tagging an
aggregate never reaches it — and the alternative there is no value at all.
### Sequencing the history fix
Snapshots are immutable, so **3b corrects new filings only**; every stored quarter
keeps its old `total_debt`. `scripts/reparse_fundamentals.py` exists for exactly this
("after a parser fix, keeping the stored row is preserving a stale cache").
**Retire the `EQR` ticker before reparsing.** A reparse backfills every tracked CIK,
so while both 0000906107 and 0000931182 are tracked, both stage the same two 2015
accessions and the run fails validation on `duplicate accession in staged snapshots`.
That is a safe stop — nothing is written — but the reparse will not complete until the
collision is gone.