feat(sec): reparse path, CIK overrides, resolution validation

Operational plumbing to land the parser fixes and to make silent resolution
failures visible.

- Reparse: SecFundamentalsImporter(reparse=True) restages every accession with
  the current parser and rewrites the ones that now reconstruct differently,
  writing the full column set so a row is never half old-parse. Snapshots stay
  immutable with respect to SEC; the stored row is our reconstruction, and after
  a parser fix keeping it is a stale cache, not history. run_import(force=True)
  bypasses the unchanged-revision no-op, since the staleness is on our side, not
  the source's. Exposed as scripts/reparse_fundamentals.py, dry-run by default.
- CIK overrides: sec_universe reads a {symbol: cik} pin from
  SystemSetting['sec_cik_overrides'], applied ahead of company_tickers.json, for
  when SEC maps a ticker to a successor shell with no filings (XOM -> a zero-
  filing "ExxonMobil Holdings Corp" while every 10-K/Q is under CIK 34088).
- Resolution validation: a tracked issuer resolving to a registrant with no XBRL
  filings now records no_xbrl_filings and raises a warning naming the CIKs and
  the override setting, instead of silently yielding nothing on every run.
- _diff_fields compares datetime instants, not representations: accepted_at
  round-trips naive from SQLite but tz-aware from Postgres, which otherwise made
  a reparse of identical data report every row as changed (and false-positived
  the pre-existing discrepancy warning).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 10:24:05 +02:00
co-authored by Claude Opus 4.8
parent 921f3d06fb
commit e54f03cba6
5 changed files with 449 additions and 12 deletions
+7 -1
View File
@@ -145,11 +145,17 @@ async def run_import(
importer: SourceImporter,
*,
engine: AsyncEngine | None = None,
force: bool = False,
) -> DataImportRun | None:
"""Run one import for ``importer``.
Returns the recorded ``DataImportRun`` (promoted / no_op / failed), or None
when the per-source advisory lock is already held (another run is active).
``force`` runs even when the revision is unchanged. The revision tracks the
*source*, so a re-import driven by a change on our side — a parser fix that
makes stored rows stale — is a no_op under the normal gate. Manually invoked
only; scheduled jobs must leave it False so an unchanged source stays a no_op.
"""
engine = engine or app_engine
source = importer.source
@@ -189,7 +195,7 @@ async def run_import(
revision = await importer.detect_revision(session)
run.revision = revision
last_rev = await _last_promoted_revision(session, source)
if revision is not None and revision == last_rev:
if not force and revision is not None and revision == last_rev:
run.status = STATUS_NO_OP
run.completed_at = _now()
await session.commit()