Recover SEC facts misfiled under a co-registrant CIK
Deploy / lint (push) Successful in 10s
Deploy / test (push) Successful in 1m47s
Deploy / deploy (push) Successful in 39s

The fundamentals import had been failing for three days on two tracked
filings the index listed but Company Facts appeared not to have. They were
not lagging: SEC filed the XBRL of NEE's and DOW's 2026-07-24 combined
parent/subsidiary 10-Qs under the co-registrant's CIK (Florida Power &
Light, Dow Chemical), so the ticker-carrying filer's own facts file never
receives that accession. This does not self-correct - an NEE filing
misattributed the same way in 2014 is still misfiled.

Because source_max_date advances only on a promoted run, the failure was
self-perpetuating: every later run re-walked the same index day and re-hit
the same two filings.

- Recover from the co-registrant file. The daily index lists every
  co-registrant of an accession, which is the only pointer to where the
  facts actually landed. Rows are re-stamped to the real filer, since
  parse_snapshots stamps the CIK of the payload it read.
- Guard the recovery with a share-count continuity check against the
  issuer's own history, so a subsidiary's standalone facts can never be
  stored as the parent's. No history, no recovery.
- Bound the blocking: a filing still unresolvable after
  MISSING_XBRL_RETRY_DAYS promotes with a named unresolved_filing warning
  instead of wedging every later import.
- Name the offending filings in the alert and record them in
  validation_json, separating not_in_companyfacts from not_in_submissions.
  The gate previously reported a count and discarded the accessions.

Verified against live SEC data: both filings recover with the correct CIK
and the guard rejects a mismatched reference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-27 11:52:59 +02:00
co-authored by Claude Opus 5
parent d2a27d4a78
commit a7aefa6fe7
2 changed files with 436 additions and 24 deletions
+287 -22
View File
@@ -18,6 +18,23 @@ Guardrails (design + reviews):
a tracked XBRL index accession missing from Company Facts fails the run (the two
are separate SEC products that can lag) so we retry rather than record a
null/partial snapshot. Non-XBRL amendments are skipped with a recorded reason.
A failure here blocks every later run (``source_max_date`` only advances on a
promoted run), so it names the offending filings in the alert and separates the
causes — ``not_in_companyfacts`` (facts lag) vs ``not_in_submissions`` (the
index row is absent from the issuer's own filing list, which no retry fixes).
- **Co-registrant recovery**, because "missing from Company Facts" is often not
missing at all: SEC files some combined parent/subsidiary filings' XBRL under
the co-registrant's CIK, so the ticker-carrying parent's own file never gets
that accession. The daily index lists every co-registrant of an accession, so
the facts are found there and re-stamped to the real filer — guarded by a
share-count continuity check so a subsidiary's standalone numbers can never be
stored as the parent's. Confirmed 2026-07-27 (NEE via FPL, DOW via Dow Chemical)
and it is not transient: an NEE filing misattributed in 2014 is still misfiled.
- **Bounded blocking.** Anything still unresolvable after ``MISSING_XBRL_RETRY_DAYS``
stops failing the run and is promoted around, with a named ``unresolved_filing``
warning. One filing SEC misfiled must not wedge every later import; the index
only moves forward, so a tolerated accession returns only via a reparse, and
only once SEC has re-filed it under the filer's own CIK.
- ``promote`` inserts snapshots ``ON CONFLICT (accession) DO NOTHING`` (immutable),
reports differing existing accessions, and applies ticker updates in the same
transaction.
@@ -32,8 +49,8 @@ Guardrails (design + reviews):
from __future__ import annotations
import logging
from collections import defaultdict
from dataclasses import dataclass, field
from collections import Counter, defaultdict
from dataclasses import dataclass, field, replace
from datetime import date, datetime, timedelta, timezone
from typing import Any, Callable
@@ -57,6 +74,16 @@ _XBRL_FORMS = {"10-K", "10-Q", "10-K/A", "10-Q/A"}
# On the one-time backfill, require this fraction of tracked issuers to yield at
# least one snapshot (guards a broken fetch/parse from promoting a hollow table).
MIN_BACKFILL_COVERAGE = 0.5
# How long an index accession may stay unresolvable before the run stops failing
# on it. Genuine index↔facts lag clears within a day (a weekend stretches it to
# three); past that it is misfiled, not late, and blocking forever costs more
# than the missing filing does — see the unresolved-filing guardrail below.
MISSING_XBRL_RETRY_DAYS = 3
# Share-count band a co-registrant-recovered row must land in, relative to the
# issuer's own last snapshot. Wide enough for buybacks/issuance, nowhere near
# wide enough to let a subsidiary shell's token float through (see _shares_continuous).
RECOVERY_SHARES_MIN = 0.5
RECOVERY_SHARES_MAX = 2.0
_SNAPSHOT_COLS = (
"cik", "accession", "form", "filed_date", "accepted_at", "period_start",
@@ -78,7 +105,11 @@ class StagedFundamentals:
skipped_filings: list[dict[str, str]] = field(default_factory=list)
field_issues: list[dict[str, str]] = field(default_factory=list)
skipped_non_xbrl: list[dict[str, str]] = field(default_factory=list)
missing_xbrl: list[dict[str, str]] = field(default_factory=list)
# Index rows we could not resolve to Company Facts, with a per-row reason
# (not_in_companyfacts | not_in_submissions | ...) — see _missing().
missing_xbrl: list[dict[str, Any]] = field(default_factory=list)
# Accessions parsed out of a co-registrant's Company Facts file.
recovered: list[dict[str, Any]] = field(default_factory=list)
invalid_payloads: list[dict[str, str]] = field(default_factory=list)
existing_accessions: set[str] = field(default_factory=set)
# Tracked issuers whose registrant has NO XBRL 10-K/10-Q at all: they can
@@ -117,6 +148,9 @@ class SecFundamentalsImporter:
# cached by detect_revision, consumed by stage:
self._resolved: ResolvedUniverse | None = None
self._index_rows: list[dict[str, Any]] = []
# accession -> the OTHER CIKs the daily index lists it under (co-registrants
# of a combined filing). Only populated for accessions a tracked issuer filed.
self._coregistrants: dict[str, list[int]] = {}
self._latest_index_date: date | None = None
self._backfill = False
@@ -132,6 +166,7 @@ class SecFundamentalsImporter:
# Reparse needs every accession restaged, not just those filed since
# the last run — the facts a fixed parser now accepts were never
# stored, so a reparse cannot be served from the database.
self._coregistrants = {}
if last_processed is None or self.reparse:
self._backfill = True
self._index_rows = []
@@ -151,10 +186,12 @@ class SecFundamentalsImporter:
staged = StagedFundamentals(resolved=resolved, backfill=self._backfill)
cik_to_tids = resolved.cik_to_ticker_ids
filed_by_cik: dict[int, list[str]] = defaultdict(list)
# Whole index rows (not bare accessions): form + index date are what make
# an unresolvable filing diagnosable without re-walking the index by hand.
filed_by_cik: dict[int, list[dict[str, Any]]] = defaultdict(list)
for r in self._index_rows:
if r["cik"] in cik_to_tids:
filed_by_cik[r["cik"]].append(r["accession"])
filed_by_cik[r["cik"]].append(r)
existing = await self._ciks_with_snapshots(db, set(cik_to_tids))
if self._backfill:
@@ -165,10 +202,15 @@ class SecFundamentalsImporter:
backfill_ciks = {c for c in cik_to_tids if c not in existing}
incremental_ciks = set(filed_by_cik) - backfill_ciks
# Continuity reference for co-registrant recovery, read once up front.
last_shares = await self._last_shares_outstanding(db, set(cik_to_tids))
async with self._client_factory() as client:
for cik in sorted(backfill_ciks | incremental_ciks):
is_backfill = cik in backfill_ciks
await self._stage_issuer(client, cik, is_backfill, filed_by_cik, staged)
await self._stage_issuer(
client, cik, is_backfill, filed_by_cik, staged, last_shares
)
# Read-only discrepancy detection: an accession we reconstructed that is
# already stored, differing in ANY source field (immutable → report in
@@ -185,7 +227,9 @@ class SecFundamentalsImporter:
staged.discrepancies.append({"accession": row.accession, "fields": fields})
return staged
async def _stage_issuer(self, client, cik, is_backfill, filed_by_cik, staged) -> None:
async def _stage_issuer(
self, client, cik, is_backfill, filed_by_cik, staged, last_shares
) -> None:
cf = await client.companyfacts(cik)
bad = _companyfacts_structure_error(cf)
if bad is not None:
@@ -201,31 +245,61 @@ class SecFundamentalsImporter:
{"cik": cik10(cik), "name": sub.get("name"), "tickers": sub.get("tickers")}
)
fiscal_year_end = sub.get("fiscal_year_end")
recovered_rows: list[SnapshotRow] = []
if is_backfill:
accns = set(xbrl_meta)
else:
present = parser.companyfacts_accessions(cf)
accns = set()
for accn in filed_by_cik.get(cik, []):
for index_row in filed_by_cik.get(cik, []):
accn = index_row["accession"]
if accn in nonxbrl:
staged.skipped_non_xbrl.append({"cik": cik10(cik), "accession": accn})
elif accn in xbrl_meta and accn in present:
elif accn not in xbrl_meta:
# The daily index lists it but the issuer's own filing list does
# not (submissions lagging the index, no usable period metadata,
# or a co-registrant filing). NOT a Company-Facts lag — separate
# cause, separate fix, so it gets its own reason.
staged.missing_xbrl.append(
_missing(cik, index_row, "not_in_submissions", self.today)
)
elif accn in present:
accns.add(accn)
else:
# XBRL (or unknown) filing not yet in Company Facts → the products
# have lagged; fail+retry rather than record nothing for it.
staged.missing_xbrl.append({"cik": cik10(cik), "accession": accn})
# Filed, XBRL, but absent from this issuer's Company Facts. Try
# the co-registrant file before treating it as missing data.
row, source_cik = await self._recover_from_coregistrant(
client, cik, accn, xbrl_meta, fiscal_year_end,
last_shares.get(cik10(cik)), staged,
)
if row is not None:
recovered_rows.append(row)
staged.recovered.append({
"cik": cik10(cik),
"accession": accn,
"source_cik": source_cik,
"form": index_row.get("form"),
})
else:
staged.missing_xbrl.append(_missing(
cik, index_row,
# Found, but it did not look like this issuer's own
# numbers — say so; it is not the same as absent.
"coregistrant_facts_rejected" if source_cik
else "not_in_companyfacts",
self.today,
))
# fiscalYearEnd (MMDD) is what lets the parser derive period identity from
# reportDate instead of SEC's unreliable fy/fp fields.
result = parser.parse_snapshots(
cf, xbrl_meta, accns, fiscal_year_end=sub.get("fiscal_year_end")
)
result = parser.parse_snapshots(cf, xbrl_meta, accns, fiscal_year_end=fiscal_year_end)
staged.rows.extend(result.rows)
staged.rows.extend(recovered_rows)
staged.skipped_filings.extend(result.skipped_filings)
staged.field_issues.extend(result.field_issues)
staged.issuers_fetched += 1
if result.rows:
if result.rows or recovered_rows:
staged.issuers_with_rows += 1
# SIC proposal for this issuer's tickers (read-only; applied in promote).
@@ -234,14 +308,65 @@ class SecFundamentalsImporter:
for tid in staged.resolved.cik_to_ticker_ids.get(cik, []):
staged.sic_updates.append((tid, sic, desc))
async def _recover_from_coregistrant(
self, client, cik: int, accn: str, xbrl_meta, fiscal_year_end, reference, staged,
) -> tuple[SnapshotRow | None, str | None]:
"""Look for ``accn``'s facts in a co-registrant's Company Facts file.
SEC sometimes files a combined parent/subsidiary filing's XBRL under the
co-registrant's CIK rather than the filer's — the ticker-carrying parent's
own file simply never gets that accession. Verified 2026-07-27 for NEE
(facts under Florida Power & Light) and DOW (under Dow Chemical); an NEE
filing misattributed the same way in **2014** is still misattributed, so
this does not self-correct and no amount of retrying recovers it.
Returns ``(row, source_cik)`` on success, ``(None, source_cik)`` when the
facts were found but rejected by the continuity guard, ``(None, None)``
when no co-registrant has them.
Incremental path only: the co-registrant map comes from the daily index,
which a backfill/reparse does not walk. A reparse therefore recovers a
filing only once SEC re-files it under the filer's own CIK.
"""
for co in self._coregistrants.get(accn, []):
try:
cf_co = await client.companyfacts(co)
except SecError:
continue # a co-registrant shell often has no facts file at all
if _companyfacts_structure_error(cf_co) is not None:
continue
result = parser.parse_snapshots(
cf_co, xbrl_meta, {accn}, fiscal_year_end=fiscal_year_end
)
if not result.rows:
continue
row = result.rows[0]
if not _shares_continuous(row.shares_outstanding, reference):
return None, cik10(co)
# A recovered row is the one most worth flagging, so its parser caveats
# travel with it rather than being dropped on the way out.
staged.field_issues.extend(result.field_issues)
# parse_snapshots stamps the CIK of the payload it read — re-stamp to
# the issuer that actually filed, or the row lands under the shell.
return replace(row, cik=cik10(cik)), cik10(co)
return None, None
async def validate(self, db, staged: StagedFundamentals) -> ValidationResult:
messages: list[str] = []
# Consistency gate — before any write.
if staged.missing_xbrl:
# Consistency gate — before any write. Only filings still inside the retry
# window block: a failure here stops every later run too (source_max_date
# advances on promotion alone), so blocking forever on a filing SEC has
# misfiled would cost far more than the one filing it withholds. Older
# ones are carried by promote() as a warning instead. The message names
# the filings: "which ones" has to be in the alert itself, not merely
# reconstructible by re-walking the index.
blocking = _within_retry_window(staged.missing_xbrl)
if blocking:
messages.append(
f"{len(staged.missing_xbrl)} tracked XBRL filing(s) not yet in "
"Company Facts (index/facts lag) — retry"
f"{len(blocking)} tracked XBRL filing(s) unresolved within the "
f"{MISSING_XBRL_RETRY_DAYS}-day retry window "
f"({_reason_counts(blocking)}) — retry: {_missing_detail(blocking)}"
)
# Malformed companyfacts payloads must fail, not degrade to skipped rows.
if staged.invalid_payloads:
@@ -272,7 +397,11 @@ class SecFundamentalsImporter:
"skipped_non_xbrl": len(staged.skipped_non_xbrl),
"no_xbrl_filings": staged.no_xbrl_filings[:50],
"no_xbrl_filings_count": len(staged.no_xbrl_filings),
"missing_xbrl": len(staged.missing_xbrl),
"missing_xbrl": staged.missing_xbrl[:50],
"missing_xbrl_count": len(staged.missing_xbrl),
"missing_xbrl_blocking": len(blocking),
"recovered_from_coregistrant": staged.recovered[:50],
"recovered_count": len(staged.recovered),
"invalid_payloads": staged.invalid_payloads,
"cik_updates": len(staged.resolved.cik_updates),
# differing existing accessions (immutable — kept, reported here)
@@ -332,6 +461,48 @@ class SecFundamentalsImporter:
created_at=_now(),
))
# Recovered rows are real data from an unexpected place — record where they
# came from, so a wrong recovery is auditable rather than invisible.
if staged.recovered:
named = ", ".join(
f"{r['accession']} <- CIK {r['source_cik']}" for r in staged.recovered[:10]
)
db.add(SystemEvent(
severity="warning",
source="sec_facts",
code="coregistrant_recovery",
message=(
f"{len(staged.recovered)} filing(s) were absent from the filer's "
f"own Company Facts and were parsed from a co-registrant's file "
f"instead (share count checked against the issuer's history): {named}"
)[:4000],
dedup_key=f"sec_facts:coregistrant_recovery:{run_id}",
created_at=_now(),
))
# Filings past the retry window: promoted WITHOUT them so one misfiled
# filing cannot wedge every later import. This is the deliberate trade —
# loud and named, because the index only moves forward and nothing will
# revisit them on its own.
tolerated = _past_retry_window(staged.missing_xbrl)
if tolerated:
db.add(SystemEvent(
severity="warning",
source="sec_facts",
code="unresolved_filing",
message=(
f"{len(tolerated)} tracked filing(s) still unresolvable after "
f"{MISSING_XBRL_RETRY_DAYS} days; promoting without them rather "
f"than blocking every later import. They are NOT retried. "
f"scripts/reparse_fundamentals.py recovers them ONLY once SEC "
f"re-files under the filer's own CIK — it walks no index, so it "
f"cannot reach facts still sitting under a co-registrant: "
f"{_missing_detail(tolerated)}"
)[:4000],
dedup_key=f"sec_facts:unresolved_filing:{run_id}",
created_at=_now(),
))
# A tracked issuer whose registrant has no XBRL filings can never produce a
# snapshot, and it is restaged on every run forever. That is a resolution
# problem, not missing data, and it is silent without this.
@@ -390,8 +561,24 @@ class SecFundamentalsImporter:
rows: list[dict[str, Any]] = []
day = last_processed + timedelta(days=1)
while day <= latest:
# Group the whole day first: a combined filing is listed once per
# co-registrant CIK, and those sibling CIKs are the only pointer to
# where SEC may have put the XBRL (see _recover_from_coregistrant).
by_accession: dict[str, list[dict[str, Any]]] = defaultdict(list)
for r in await client.daily_index(day):
if r["form"] in _XBRL_FORMS and r["cik"] in tracked:
if r["form"] in _XBRL_FORMS:
by_accession[r["accession"]].append(r)
for accession, group in by_accession.items():
filers = {r["cik"] for r in group}
tracked_filers = filers & tracked
if not tracked_filers:
continue
siblings = sorted(filers - tracked_filers)
if siblings:
self._coregistrants[accession] = siblings
for r in group:
if r["cik"] in tracked_filers:
r["index_date"] = day # not hashed (revision uses cik/accession)
rows.append(r)
day += timedelta(days=1)
return rows
@@ -409,6 +596,30 @@ class SecFundamentalsImporter:
).scalars().all()
return {int(c) for c in found}
async def _last_shares_outstanding(self, db, ciks: set[int]) -> dict[str, float]:
"""Latest known shares outstanding per tracked issuer — the continuity
reference co-registrant recovery is checked against."""
if not ciks:
return {}
rows = (
await db.execute(
select(FundamentalSnapshot.cik, FundamentalSnapshot.shares_outstanding)
.where(
FundamentalSnapshot.cik.in_([cik10(c) for c in ciks]),
FundamentalSnapshot.shares_outstanding.is_not(None),
)
# Last write per cik wins, so the sort must be total: an amendment
# and its original share a period_end, and an undefined tie there
# would make recovery non-deterministic across runs and dialects.
.order_by(
FundamentalSnapshot.period_end,
FundamentalSnapshot.filed_date,
FundamentalSnapshot.accession,
)
)
).all()
return {cik: float(shares) for cik, shares in rows}
async def _existing_by_accession(self, db, accessions: list[str]) -> dict[str, FundamentalSnapshot]:
if not accessions:
return {}
@@ -435,6 +646,60 @@ def _companyfacts_structure_error(cf: Any) -> str | None:
return None
def _missing(cik: int, row: dict[str, Any], reason: str, today: date) -> dict[str, Any]:
"""One unresolvable index row, carrying everything needed to look the filing
up by hand (EDGAR accession + the index date it was seen on) and to decide
whether it is still young enough to be worth blocking on."""
index_date = row.get("index_date")
return {
"cik": cik10(cik),
"accession": row["accession"],
"form": row.get("form"),
"index_date": index_date,
# No index date (older cached rows) => age 0 => blocks, the safe default.
"age_days": (today - index_date).days if isinstance(index_date, date) else 0,
"reason": reason,
}
def _within_retry_window(missing: list[dict[str, Any]]) -> list[dict[str, Any]]:
return [m for m in missing if m.get("age_days", 0) <= MISSING_XBRL_RETRY_DAYS]
def _past_retry_window(missing: list[dict[str, Any]]) -> list[dict[str, Any]]:
return [m for m in missing if m.get("age_days", 0) > MISSING_XBRL_RETRY_DAYS]
def _shares_continuous(shares: float | None, reference: float | None) -> bool:
"""Does a co-registrant-recovered share count look like this issuer's own?
The failure worth preventing is storing a subsidiary's standalone facts as the
parent's. A co-registrant shell holds a token float — Florida Power & Light
against NextEra's 2.09bn shares — so any sane band separates them while still
tolerating buybacks and issuance. With no history to compare against (a newly
tracked issuer) or no share count at all, recovery is refused, not guessed.
"""
if not shares or not reference:
return False
return RECOVERY_SHARES_MIN <= shares / reference <= RECOVERY_SHARES_MAX
def _reason_counts(missing: list[dict[str, Any]]) -> str:
counts = Counter(m["reason"] for m in missing)
return ", ".join(f"{reason}={n}" for reason, n in sorted(counts.items()))
def _missing_detail(missing: list[dict[str, Any]], limit: int = 10) -> str:
detail = ", ".join(
f"{m['cik']}/{m['accession']} {m.get('form') or '?'} "
f"[{m.get('index_date') or '?'}] {m['reason']}"
for m in missing[:limit]
)
if len(missing) > limit:
detail += f", +{len(missing) - limit} more"
return detail
def _filing_meta(sub: dict[str, Any]) -> tuple[dict[str, FilingMeta], set[str]]:
"""(xbrl_meta, nonxbrl_accessions) from a submissions payload. xbrl_meta only
includes 10-K/10-Q(/A) filings that are XBRL and have full period metadata."""
+148 -1
View File
@@ -3,6 +3,7 @@ import framework with a fake SEC client (no network)."""
from __future__ import annotations
import json
import os
import tempfile
from datetime import date, datetime, timezone
@@ -14,6 +15,7 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_asyn
from app.database import Base
import app.models # noqa: F401
from app.models.fundamental_snapshot import FundamentalSnapshot
from app.models.system_event import SystemEvent
from app.models.ticker import Ticker
from app.services.data_import import STATUS_FAILED, STATUS_PROMOTED, run_import
from app.services.sec_fundamentals_importer import SecFundamentalsImporter
@@ -212,10 +214,155 @@ async def test_consistency_gate_fails_when_facts_lag_index(engine):
run = await run_import(_importer(incr, today=date(2026, 5, 3)), engine=engine)
assert run.status == STATUS_FAILED
assert "Company Facts" in (run.error_details or "")
# The gate blocks every later run until it clears, so the alert itself has to
# name the filing and say why it could not be resolved.
details = run.error_details or ""
assert "GHOST" in details and "not_in_companyfacts" in details
assert "2026-05-01" in details # index date the filing was seen on
summary = json.loads(run.validation_json or "{}")
assert summary["missing_xbrl_count"] == 1
assert summary["missing_xbrl"][0]["accession"] == "GHOST"
assert summary["missing_xbrl"][0]["form"] == "10-Q"
assert await _count(factory, FundamentalSnapshot) == 2 # nothing new written
async def test_gate_separates_missing_submissions_from_missing_facts(engine):
"""An index row the issuer's own filing list does not carry is a different
failure from a Company-Facts lag, and must not be reported as one."""
factory = _factory(engine)
await _seed(factory, ["AAPL"])
backfill = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])},
submissions={320193: _submissions(SUB_FILINGS)},
latest_index=date(2026, 1, 31),
)
await run_import(_importer(backfill), engine=engine)
incr = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])},
submissions={320193: _submissions(SUB_FILINGS)}, # submissions never lists ORPHAN
latest_index=date(2026, 5, 2),
daily={date(2026, 5, 1): [{"form": "10-Q", "cik": 320193, "accession": "ORPHAN"}]},
)
run = await run_import(_importer(incr, today=date(2026, 5, 3)), engine=engine)
assert run.status == STATUS_FAILED
assert "ORPHAN" in (run.error_details or "")
assert "not_in_submissions" in (run.error_details or "")
summary = json.loads(run.validation_json or "{}")
assert summary["missing_xbrl"][0]["reason"] == "not_in_submissions"
def _coregistrant_client(share_fact):
"""Incremental client where Q2A's facts landed in co-registrant 99999's file
instead of the filer's own — the NEE-via-FPL / DOW-via-Dow-Chemical shape."""
cf_q2 = _rev("2025-09-28", "2026-03-28", 254940, 2026, "Q2", "Q2A")
return FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={
320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1]), # filer's own: no Q2A
99999: _companyfacts([cf_q2], [share_fact], cik=99999),
},
submissions={320193: _submissions(SUB_FILINGS + [
_filing("Q2A", "10-Q", "2026-03-28", "2026-05-01", "2026-05-01T10:01:00.000Z")])},
latest_index=date(2026, 5, 2),
daily={date(2026, 5, 1): [
# one combined filing, listed by the index under both co-registrants
{"form": "10-Q", "cik": 320193, "accession": "Q2A"},
{"form": "10-Q", "cik": 99999, "accession": "Q2A"},
]},
)
async def test_recovers_facts_misfiled_under_coregistrant(engine):
factory = _factory(engine)
await _seed(factory, ["AAPL"])
backfill = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])},
submissions={320193: _submissions(SUB_FILINGS)},
latest_index=date(2026, 1, 31),
)
await run_import(_importer(backfill), engine=engine)
# 14687 shares is continuous with the issuer's own history (14681 last quarter).
incr = _coregistrant_client(_shares("2026-04-17", 14687, "Q2A", 2026, "Q2"))
run = await run_import(_importer(incr, today=date(2026, 5, 3)), engine=engine)
assert run.status == STATUS_PROMOTED
summary = json.loads(run.validation_json or "{}")
assert summary["recovered_count"] == 1
assert summary["recovered_from_coregistrant"][0]["source_cik"] == "0000099999"
assert summary["missing_xbrl_count"] == 0
async with factory() as s:
q2 = (await s.execute(
select(FundamentalSnapshot).where(FundamentalSnapshot.accession == "Q2A")
)).scalar_one()
codes = (await s.execute(select(SystemEvent.code))).scalars().all()
# Stamped to the issuer that filed, NOT the co-registrant whose file it came from.
assert q2.cik == "0000320193"
assert q2.revenue == 254940 and q2.shares_outstanding == 14687
assert "coregistrant_recovery" in codes
async def test_coregistrant_recovery_rejects_discontinuous_share_count(engine):
"""A co-registrant shell's standalone facts must never be stored as the
parent's — a token float is the signature and it has to be refused."""
factory = _factory(engine)
await _seed(factory, ["AAPL"])
backfill = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])},
submissions={320193: _submissions(SUB_FILINGS)},
latest_index=date(2026, 1, 31),
)
await run_import(_importer(backfill), engine=engine)
incr = _coregistrant_client(_shares("2026-04-17", 100, "Q2A", 2026, "Q2"))
run = await run_import(_importer(incr, today=date(2026, 5, 3)), engine=engine)
assert run.status == STATUS_FAILED
assert "coregistrant_facts_rejected" in (run.error_details or "")
assert await _count(factory, FundamentalSnapshot) == 2 # nothing recovered
async def test_unresolved_filing_stops_blocking_after_retry_window(engine):
"""A filing SEC has misfiled must not wedge every later import forever."""
factory = _factory(engine)
await _seed(factory, ["AAPL"])
backfill = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])},
submissions={320193: _submissions(SUB_FILINGS)},
latest_index=date(2026, 1, 31),
)
await run_import(_importer(backfill), engine=engine)
incr = FakeSecClient(
tickers={"AAPL": 320193},
companyfacts={320193: _companyfacts([CF_K, CF_Q1], [SH_K, SH_Q1])}, # no GHOST
submissions={320193: _submissions(SUB_FILINGS + [
_filing("GHOST", "10-Q", "2026-03-28", "2026-05-01", "2026-05-01T10:01:00.000Z")])},
latest_index=date(2026, 5, 2),
daily={date(2026, 5, 1): [{"form": "10-Q", "cik": 320193, "accession": "GHOST"}]},
)
# 9 days after the index date — well past the retry window.
run = await run_import(_importer(incr, today=date(2026, 5, 10)), engine=engine)
assert run.status == STATUS_PROMOTED # promoted around it, not blocked by it
assert run.source_max_date == date(2026, 5, 2) # and the index advances
summary = json.loads(run.validation_json or "{}")
assert summary["missing_xbrl_count"] == 1 and summary["missing_xbrl_blocking"] == 0
async with factory() as s:
events = (await s.execute(select(SystemEvent))).scalars().all()
unresolved = [e for e in events if e.code == "unresolved_filing"]
assert len(unresolved) == 1 and "GHOST" in unresolved[0].message
async def test_non_xbrl_amendment_skipped_not_failed(engine):
factory = _factory(engine)
await _seed(factory, ["AAPL"])