feat(sec): A3 slice 1 — SEC client + CIK/SIC resolution + composite revision

First A3 implementation checkpoint (design: docs/dolt-sec-a3-design.md).

- sec_client.py: async SEC EDGAR client honoring fair-access — identifying
  User-Agent (config), request spacing < 10 req/s, exponential backoff on 429,
  and 403 -> SecForbiddenError (alert and stop, never retry-loop). Fetchers:
  company_tickers (normalised, multi-class share CIK), submissions (merges the
  paginated filings.files shards so full history is visible), companyfacts,
  daily_index (fixed-width form.idx parse), latest_index_date.
- sec_universe.py: resolve_ciks (tickers.cik backfill), refresh_sic
  (sic/sic_description), and the composite-revision pieces — universe_fingerprint
  (a new ticker changes the revision, so it's never no_op'd/starved),
  index_content_hash, compose_revision.
- config + .env.example: SEC_USER_AGENT (must be a real contact email) + spacing
  / retries / timeout.

Verified live against real SEC: AAPL->320193, GOOG==GOOGL, BRK-B resolved;
submissions shard-merge proven (131 filings back to 1993); daily index parsed.
Tests: 11 (mocked-transport parsing + 403/429 handling + resolution/fingerprint).
Full suite 713 passed. Next slice: companyfacts -> snapshot parser + importer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 14:15:56 +02:00
co-authored by Claude Opus 4.8
parent b1397fa82e
commit cc67aebe61
6 changed files with 657 additions and 0 deletions
+8
View File
@@ -43,6 +43,14 @@ DOLT_MIN_FREE_DISK_GB=5.0
# import connection + advisory lock.
DOLT_COMMAND_TIMEOUT_SECONDS=600.0
# SEC EDGAR (fundamentals, workstream A). SEC fair-access REQUIRES an identifying
# User-Agent with a REAL contact email — set it, or requests get 403'd. Stay well
# under 10 req/s (spacing below).
SEC_USER_AGENT=signal-platform/1.0 (contact: you@example.com)
SEC_REQUEST_SPACING_SECONDS=0.2
SEC_MAX_RETRIES=4
SEC_REQUEST_TIMEOUT_SECONDS=30.0
# Regime Monitor — FRED (VIX + HY credit spreads). Free key: https://fred.stlouisfed.org/docs/api/api_key.html
# Optional: without it the volatility (V1) and credit (C1) pillars show as n/a.
FRED_API_KEY=