Three follow-ups from review of the A6 commits.
The cache-summary re-finalize raised a second durable system event on a failed
run: _runtime_finish emits for `error`/`rate_limited`, and the dedup key
includes the message, so "SEC unavailable" and "SEC unavailable · cache 511 · 2
score inputs changed" landed as two unacknowledged Admin events. Adds
`emit_event` so a re-finalize that only rewords an outcome stays silent, with a
regression test asserting exactly one event.
The rollback section still claimed disabling the SEC job freezes the cache — the
opposite of what the same page says two lines earlier, and of what the code now
does. Rewritten: there is no Admin cache-off switch, restoring `fundamental_data`
alone is temporary because the next run rebuilds it from the same snapshots and
code, and a real freeze means stopping the service.
Remaining "shadow" wording: the two import jobs have never been shadow since
activation, so `_run_shadow_import` -> `_run_source_import`, its section heading,
the deployment doc's job label, and the plan doc's "production switch remains"
handoff paragraph are all brought up to date.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The A5 cutover has been on and observed in production, so SEC Company Facts +
DoltHub earnings are already the live source for `fundamental_data`. This
removes everything the legacy path still occupied.
Gone: the three providers and their config/env keys; the weekly
`fundamental_collector` job; the cutover toggle (SEC + Dolt is now the
unconditional path, so `off` can no longer silently freeze scoring inputs); the
A5 parity report, whose deltas became structurally zero once the candidate
builder started writing the table it compared against; and the FMP tier of
universe bootstrap.
Two behavioral notes:
- Disabling **SEC Fundamentals Import** now stops the SEC network fetch only.
The local cache refresh moved outside the job-enable check, because candidates
also derive from daily closes and earnings events — freezing those on an
ingestion pause would stale scoring with no fallback left to recover from.
- `/ingestion/fetch?sources=fundamentals` still accepts the key and reports
`skipped`; there is no per-ticker fetch any more.
Migration 029 does not blanket-delete the leftover settings rows. Migrations run
before the service restart, and pre-A6 code reads an absent `job_*_enabled` row
as *enabled* — so the two behavior-bearing keys become tombstones pinned to safe
values (hidden in Admin) and only the inert three are deleted. Removing the
provider keys from the production `.env` is the matching rollout step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parity gate has been exercised: nine-pass investigation, fixes, two prod
reparses, and an after-report at 504/511 candidate coverage with agreement
unchanged. What remains of workstream A is the activation itself (step c, the
post-approval fundamental_data refresh — not yet implemented) and A6
decommissioning, both now specified in a handoff section with the caveats the
next implementer must carry (KLAC splits, BRK-B, FITB, the 25% guard, CIK
overrides, reparse-after-parser-changes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. net_debt requires BOTH cash and total_debt; a missing side is null, not
treated as zero (which would be a partial, misleading value).
2. net_debt_to_ebitda is null when TTM EBITDA <= 0 — a negative denominator
would otherwise rank a distressed issuer as favorably low-leverage.
3. The quarter tape is the CONSECUTIVE run ending at the latest period (stops at
a gap), so trend text never compares non-adjacent quarters as if consecutive.
4. YoY growth is null when the prior-year TTM is <= 0 (e.g. loss->profit), which
is not a meaningful percentage.
Also corrected the plan's net-debt formula to total debt − (cash + ST) matching
the positive-means-net-debt implementation. +4 tests. 10 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1. Multi-class shares: prefer the single dei:EntityCommonStockSharesOutstanding
cover-page fact; else fall back to us-gaap:CommonStockSharesOutstanding at
period end (Alphabet has no dei fact). Never sum class facts (companyfacts is
non-dimensional) and never use weighted-average/diluted; conflicting values ->
null, counted as an "ambiguous shares outstanding" note in validation. Plan's
"sum class-specific" wording corrected. Verified live: Alphabet shares now
populate (12.1B), Apple still uses its dei cover date.
2. Fiscal context is the majority (fy, fp) among facts ending at reportDate, with
ties rejected — no longer the arbitrary first fact.
3. Hardening: catalog selectors require taxonomy == "us-gaap"; indexing drops
malformed facts (missing accession/end, non-finite value) so a custom concept
or bad date can't be selected.
Tests: +8 (dei precedence, us-gaap fallback, conflict->null, no weighted-average,
tie-context skip, foreign-taxonomy/malformed ignored, ambiguous-shares note).
14 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
post-no-preference/earnings approved for private/internal ingestion under
CC BY-SA 4.0. Conditions the A2 importer must honor: preserve upstream license /
attribution / transformation notes; no public API, bulk export, or
redistribution; re-review before any public or commercial access. The stocks
repo (workstream B) is not covered and will be reviewed separately if B begins.
A0 rollout item marked done (dolt binary pin + DOLT_DATA_DIR still pending at
deploy time).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review items 3-4 on the plan doc:
- Replace remaining "diluted shares" (the share *count*) with point-in-time
shares_outstanding (dei:EntityCommonStockSharesOutstanding) across schema,
metrics catalog and market-cap note. "diluted EPS" is left as-is (correctly a
duration fact). Adds the multi-class rule: derive the issuer-wide count from
the consolidated cover-page figure OR by summing class-specific facts (GOOG +
GOOGL) — never both, to avoid double counting.
- The framework stages into a representation *outside the live tables* (in-memory
for workstream A; a file/table handle is fine if B needs it), not physical
"staging tables" — wording now matches the implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the Dolt bulk-data integration hand-off plan (authored in prior
session) and applies clarifications found in a pre-handoff review:
- step (c): fields refresh from three distinct sources, not "snapshots +
close" — earnings_surprise/next_earnings_date come from earnings_events,
not SEC facts. pe_ratio = close / TTM diluted EPS and market_cap =
issuer-wide diluted shares × close stated as separate formulas.
- fundamental_snapshots made implementable: add period_start, fiscal_year,
fiscal_period; duration facts store the filing's cumulative YTD/FY values,
balance-sheet facts store period-end values. Discrete quarters, Q4, TTM
and YoY are derived at read time (correct for non-calendar fiscal years;
amendments never freeze a stale derived quarter).
- flag Q4 derivation / fiscal-period alignment as the primary A3 risk.
- anchor "tracked universe" to ticker_universe_service + per-run CIK
resolution.
All code anchors in the doc verified accurate against the current tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dennisthiessenandClaude Opus 5