diff --git a/docs/dolt-integration-plan.md b/docs/dolt-integration-plan.md index fa22ac5..80cea90 100644 --- a/docs/dolt-integration-plan.md +++ b/docs/dolt-integration-plan.md @@ -426,8 +426,9 @@ workstream B — Alpaca remains the price source throughout. API values across the tracked universe, report per-field deltas and resulting fundamental-score/ranking changes, require explicit approval. Definition changes (e.g. TTM vs provider convention) called out, not averaged away. - The read-only report job and Admin summary/download are implemented; production - observation and explicit cutover approval remain pending. + **Status 2026-07-24: the gate has been exercised and the evidence supports + approval** — see the handoff section below. What remains of A5 is the + activation itself: implementing step (c) and flipping it on. - A6. Remove FMP/Finnhub/Alpha Vantage; keep monitoring + manual fallback. **Workstream B (independent, start when wanted):** @@ -479,6 +480,47 @@ workstream B — Alpaca remains the price source throughout. - Score-parity diff reviewed and approved before `fundamental_data` cutover. - Scheduled imports never block the API event loop. +## Handoff — remaining work after the A5 parity investigation (2026-07-24) + +The 2026-07-23 parity report surfaced coverage gaps and wrong values; a nine-pass +investigation traced every one to parser/identity bugs (not source data), fixed them, +and reparsed production twice. Full evidence trail: +`reports/fundamentals-parity-20260723-findings.md` (root causes, decisions, validation) +plus the before/after reports (`fundamentals-parity-20260723T…` / `…20260724T….json`). +Post-fix: candidate scores 504 of 511 vs legacy's 507 (gap = PSKY/Q new registrants + +FITB, all explained); revenue-growth agreement 0.0038 median abs delta where both exist. +Dennis reviewed the evidence 2026-07-24 and directed proceeding to cutover. + +**Task 1 — A5 activation (implement step (c) above, ~line 207).** The post-activation +local refresh of `fundamental_data` does not exist yet. Per the spec: `pe_ratio` and +`market_cap` from newest valid snapshots × latest PostgreSQL close, `revenue_growth` +from snapshots, `earnings_surprise`/`next_earnings_date` from `earnings_events`; mark +affected cached fundamental scores stale; must run identically when SEC is unreachable. +Implementation notes from the parity work: consume `fundamentals_derivation.derive()` +outputs, NOT raw snapshot fields — that path carries the split guard (`ttm_diluted_eps` +nulls when contaminated, with `ttm_diluted_eps_caveat`) and the multi-class share +fallback (`shares_outstanding` + `shares_outstanding_estimated`). Activation should be +an explicit switch (SystemSetting, like `sec_cik_overrides`), default off. + +**Task 2 — A6 decommissioning.** After a short observation window: remove +FMP/Finnhub/Alpha Vantage providers, config and env keys; keep monitoring + manual +fallback. Gated by the acceptance criteria above — especially forward-calendar +timeliness from `dolt_earnings` (its `source_max_date` ran ~5 weeks ahead as of +2026-07-23, which passes). + +**Known caveats to carry (documented in the findings report, not bugs to fix):** +- KLAC-class post-filing splits: P/E wrong until the next 10-Q; undetectable from + snapshots. Workstream B's `corporate_actions` table is the natural future fix. +- BRK-B: no share count exists anywhere in companyfacts → no market cap, correctly. +- FITB: unscored (split guard + no taggable revenue) — the one name that lost its + score relative to legacy; composite renormalises. +- Share-change guard at 25% nulls P/E for stock-funded M&A too (COF, WAT…); + revisit only if the ~3% universe hit-rate proves painful. +- `sec_cik_overrides` SystemSetting pins XOM → 34088 (applied in prod); the + `no_xbrl_filings` SystemEvent says when a new pin is needed. +- After any future parser change, stored rows need `scripts/reparse_fundamentals.py` + (dry-run default; `--apply` rewrites) — snapshots are otherwise immutable. + ## Deferred (explicitly, until a concrete need appears) - Workstream B itself is deferred relative to A and blocks nothing in A.