fix(sec): keep the market-cap fallback through an amendment merge
Found in review. _merge_amendments rebuilds a period from _MERGED_FIELDS + _CARRIED_FIELDS alone, so a column in neither list is absent from the merged row, not just stale — and callers read it with getattr(..., None), which silently yields None. weighted_avg_diluted_shares was never added when the market-cap fallback landed (_SNAPSHOT_COLS in the importer was updated, its counterpart in the derivation was not). The failure needed both of this branch's fixes at once: a multi-class issuer with a partial amendment on its latest period (META with a Part-III-only 10-K/A) would silently lose market cap and FCF yield again. Adds the field, a regression test for that case, and a guard test asserting the merge/carry lists cover every SnapshotRow field, so the next column added fails loudly rather than losing data quietly. Confirmed the guard catches the original bug. Also from review: - Expose pe_caveat in the valuation payload, so a P/E suppressed by split contamination says why instead of looking like missing data (the caveat was set but never read). - no_xbrl_filings now names both causes; the old text advised pinning a CIK override, which is wrong for a genuine new registrant that simply has not filed yet and clears itself. - Document that fiscalYearEnd is the issuer's current calendar, so a fiscal- year-end change degrades old periods (fallback/newest-wins), not current ones. - Parser-level tests for _select_weighted_avg_shares (shortest-span-wins and concept priority), which only had derivation-level coverage. 823 unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -162,6 +162,10 @@ def _build_valuation(derived, subject_price, peer_derived, peer_price_by_cik, tw
|
||||
"shares_estimated": bool(
|
||||
market_cap is not None and derived.shares_outstanding_estimated
|
||||
),
|
||||
# A null P/E is ambiguous: no earnings data, or earnings we deliberately
|
||||
# suppressed. Only the latter carries a caveat, so a split-contaminated
|
||||
# TTM says why instead of looking like missing data.
|
||||
"pe_caveat": derived.ttm_diluted_eps_caveat if pe is None else None,
|
||||
"pe_industry": pe_industry,
|
||||
"fcf_yield_industry": fcf_yield_industry,
|
||||
"price_date": _iso(price_date),
|
||||
|
||||
@@ -46,7 +46,7 @@ _FLOW_FIELDS = (
|
||||
_MERGED_FIELDS = (
|
||||
*_FLOW_FIELDS,
|
||||
"cash_and_st_investments", "total_debt", "shares_outstanding",
|
||||
"shares_outstanding_date",
|
||||
"shares_outstanding_date", "weighted_avg_diluted_shares",
|
||||
# period_start is set alongside revenue by the parser, so it follows the same
|
||||
# fallback: a bare amendment reports neither and must not blank it.
|
||||
"period_start",
|
||||
|
||||
@@ -310,6 +310,12 @@ def _period_identity(
|
||||
ends in early January (DPZ) shifts by one — they need to be unique, monotonic
|
||||
and YoY-aligned, which is all the derivation asks of them. Nothing outside the
|
||||
derivation reads these columns.
|
||||
|
||||
Known limitation: ``fiscalYearEnd`` is the issuer's *current* calendar, so a
|
||||
company that has changed its fiscal year end gets its historical periods
|
||||
measured against the new one. The quarter tolerance shunts most of those to
|
||||
the fy/fp fallback, and a same-key collision resolves newest-wins, so the
|
||||
failure mode is a degraded old year rather than a scrambled current one.
|
||||
"""
|
||||
fy = _fiscal_year_of(meta.report_date, fiscal_year_end)
|
||||
if fy is None:
|
||||
|
||||
@@ -345,8 +345,10 @@ class SecFundamentalsImporter:
|
||||
code="no_xbrl_filings",
|
||||
message=(
|
||||
f"{len(staged.no_xbrl_filings)} tracked issuer(s) resolved to a "
|
||||
f"registrant with no XBRL 10-K/10-Q — pin the right CIK via the "
|
||||
f"'{sec_universe.CIK_OVERRIDES_KEY}' setting: {named}"
|
||||
f"registrant with no XBRL 10-K/10-Q. Either a successor shell "
|
||||
f"(pin the real filer via the '{sec_universe.CIK_OVERRIDES_KEY}' "
|
||||
f"setting) or a new registrant that has not filed its first "
|
||||
f"10-K/10-Q yet, which needs nothing and clears itself: {named}"
|
||||
)[:4000],
|
||||
dedup_key=f"sec_facts:no_xbrl_filings:{run_id}",
|
||||
created_at=_now(),
|
||||
|
||||
Reference in New Issue
Block a user