4 Commits

Include renames

Author	SHA1	Message	Date
dennisthiessen	00c28ccb76	fix(sec): remove unused sqlalchemy import	2026-07-23 13:32:39 +02:00
dennisthiessenandClaude Opus 4.8	96d3b40560	fix(sec): A3 sign-off hardening — validate per-concept units structure ... - Extend the companyfacts structural check to reject a concept with a missing/non-dict `units` mapping (not just the top-level `facts`), so a partially-malformed payload fails promotion instead of silently dropping that concept's facts. New fixture proves it fails. - Strengthen the newly-added-issuer test: keep latest_index equal to the prior run so ONLY the universe fingerprint changes the revision — proving the fingerprint alone prevents a new ticker from being starved/no_op'd. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-22 20:04:11 +02:00
dennisthiessenandClaude Opus 4.8	8dcdcac2a6	fix(sec): A3 slice-2b review — no index cap, full discrepancy + malformed gate ... 1. Removed the 45-day index-walk cap: it discarded the older part of a long outage while still advancing source_max_date, permanently losing filings. The walk now covers every unprocessed date (a large gap is one-time cost). 2. Discrepancy detection meets the immutability contract: it compares ALL source snapshot fields (not five), read-only during stage/validate, reports the differing accessions + fields in validation_json, and promote emits a warning system event (in-transaction) — never mutating the stored row. 3. Malformed companyfacts (missing facts/units structure) are recorded separately and FAIL validation, instead of silently degrading to skipped rows that the 50% backfill coverage floor could still pass. Also corrected the stale "sum share classes" / DEI-only wording in the snapshot model docstring and the A3 design doc to describe the us-gaap fallback. Tests: +4 regressions (>45-day gap loses nothing, newly-added issuer backfills without filing, malformed payload fails, shares discrepancy detected + evented). 23 passed, 1 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-22 19:55:58 +02:00
dennisthiessenandClaude Opus 4.8	f7ce85a33e	feat(sec): A3 slice 2b — SEC fundamentals importer (shadow ingestion) ... SecFundamentalsImporter (SourceImporter, source=sec_facts): populates immutable fundamental_snapshots from Company Facts and back-fills tickers.cik/sic, driven by the EDGAR daily index. Shadow only. Guardrails per review: - detect_revision caches the resolved universe + exact tracked index rows and composes the revision from them; stage consumes those same cached inputs (no index/universe refetch) so promoted data matches the computed revision. - Resolution is read-only in stage (proposals only); ticker writes happen in promote via apply_ticker_updates. - validate runs the index<->Company-Facts consistency gate before any write: a tracked XBRL index accession missing from Company Facts fails the run (they lag independently) so we retry, not record null. Non-XBRL amendments are skipped with a recorded reason. Backfill has a coverage floor. - promote inserts ON CONFLICT (accession) DO NOTHING (immutable), reports differing existing accessions without mutating, and applies ticker updates in the same transaction. - Full-history backfill on first run / for newly-added issuers (include_history); incremental fetch only for issuers that filed. Parser: split parse result into skipped_filings vs field_issues (coverage must not count field warnings); header notes the us-gaap shares fallback; added companyfacts_accessions() for the gate. Verified live end-to-end (AAPL + GOOGL backfill): 112 snapshots, cik/sic set, GOOGL shares via us-gaap fallback, AAPL via dei. Tests: 6 importer (backfill, incremental, consistency-gate fail, non-XBRL skip, read-only-on-failure, conflict-discrepancy) + parser ParseResult updates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-22 19:39:46 +02:00