fix(sec): alert when a filing gap's reprieve lapses instead of re-pausing quietly
An escalated gap stops pausing setups while the issuer's own fundamentals are still recent. That reprieve ends on its own — the stored filings age past GAP_GATE_RECENT_FILING_DAYS, or a newer gap arrives and the all-escalated condition fails — and nothing reported either, because filing_gap_aged only escalates gaps whose escalated_at is NULL and so never fires twice for the same gap. For the 43 issuers behind the previous commit that lands around 2026-10-26, when their late-April filings age out together. sec_filing_gaps.exempted_at (migration 034) makes the transition observable: stamped quietly while the issuer is exempt, cleared when the exemption lapses, and the clear is what raises filing_gap_repaused — once per lapse, re-arming if the issuer's data recovers and ages out again. A gap that was never exempt has no transition and stays silent; it is simply still paused, which filing_gap_aged already said. gap_exempt_ciks is public so the importer alerts on membership changes in exactly the set the gate reads, rather than restating the rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Lo99z3jWqu9X9ueBU3Z3D
This commit is contained in:
@@ -30,3 +30,8 @@ class SecFilingGap(Base):
|
||||
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
last_attempted_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
escalated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
# Set while this gap's issuer is exempt from the setup pause (escalated, and
|
||||
# its own fundamentals still recent — see fundamentals_quality_service).
|
||||
# Cleared when the exemption lapses, which is the moment the pause silently
|
||||
# comes back and the only moment worth alerting on.
|
||||
exempted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
@@ -70,13 +70,16 @@ async def active_gaps(
|
||||
return list((await db.execute(stmt)).scalars().all())
|
||||
|
||||
|
||||
async def _gap_exempt_ciks(
|
||||
async def gap_exempt_ciks(
|
||||
db: AsyncSession, gaps: list[SecFilingGap]
|
||||
) -> set[str]:
|
||||
"""CIKs whose gaps have stopped pausing setups (see GAP_GATE_RECENT_FILING_DAYS).
|
||||
|
||||
Every one of a CIK's active gaps must be escalated: one fresh gap alongside an
|
||||
old one still means a filing we might yet ingest, which is worth pausing for.
|
||||
|
||||
Public because the importer alerts on this exact transition (a CIK dropping
|
||||
out of this set is a pause coming back on) and the rule must not exist twice.
|
||||
"""
|
||||
by_cik: dict[str, list[SecFilingGap]] = defaultdict(list)
|
||||
for gap in gaps:
|
||||
@@ -135,7 +138,7 @@ async def blocked_reasons_by_cik(
|
||||
gaps = await active_gaps(db, ciks)
|
||||
# Escalated gaps on issuers that still have recent fundamentals no longer
|
||||
# pause setups, on either path below — the summary mirrors the same filings.
|
||||
exempt = await _gap_exempt_ciks(db, gaps)
|
||||
exempt = await gap_exempt_ciks(db, gaps)
|
||||
reasons = {
|
||||
gap.cik: "sec_filing_gap" for gap in gaps if gap.cik not in exempt
|
||||
}
|
||||
|
||||
@@ -776,6 +776,60 @@ class SecFundamentalsImporter:
|
||||
.values(escalated_at=now)
|
||||
)
|
||||
|
||||
# The escalation above fires once per gap, so nothing would report the
|
||||
# *end* of the reprieve it grants. An escalated gap stops pausing setups
|
||||
# while the issuer's own fundamentals are still recent, and that lapses
|
||||
# on its own — the stored filings age past the window, or a newer gap
|
||||
# appears — putting the pause back on with no alert anywhere. Track the
|
||||
# exemption as state and alert on the transition, once per lapse.
|
||||
current_gaps = await fundamentals_quality_service.active_gaps(db)
|
||||
escalated_gaps = [g for g in current_gaps if g.escalated_at is not None]
|
||||
if escalated_gaps:
|
||||
exempt_ciks = await fundamentals_quality_service.gap_exempt_ciks(
|
||||
db, escalated_gaps
|
||||
)
|
||||
newly_exempt = [
|
||||
g for g in escalated_gaps
|
||||
if g.cik in exempt_ciks and g.exempted_at is None
|
||||
]
|
||||
lapsed = [
|
||||
g for g in escalated_gaps
|
||||
if g.cik not in exempt_ciks and g.exempted_at is not None
|
||||
]
|
||||
if newly_exempt:
|
||||
# Silent on purpose: filing_gap_aged already announced this gap,
|
||||
# and setups resuming is the behaviour that alert describes.
|
||||
await db.execute(
|
||||
update(SecFilingGap)
|
||||
.where(SecFilingGap.id.in_([g.id for g in newly_exempt]))
|
||||
.values(exempted_at=now)
|
||||
)
|
||||
if lapsed:
|
||||
named = ", ".join(
|
||||
f"{gap.cik}/{gap.accession}" for gap in lapsed[:10]
|
||||
)
|
||||
db.add(SystemEvent(
|
||||
severity="warning",
|
||||
source="sec_facts",
|
||||
code="filing_gap_repaused",
|
||||
message=(
|
||||
f"{len(lapsed)} SEC filing gap(s) pause setups again: the "
|
||||
"issuer's own fundamentals have aged out of the "
|
||||
f"{fundamentals_quality_service.GAP_GATE_RECENT_FILING_DAYS}"
|
||||
"-day window, or a newer gap arrived, so there is nothing "
|
||||
f"recent left to score on: {named}"
|
||||
)[:4000],
|
||||
dedup_key=f"sec_facts:filing_gap_repaused:{run_id}",
|
||||
created_at=now,
|
||||
))
|
||||
# Cleared, not stamped: the issuer can recover and age out again,
|
||||
# and each lapse is worth its own alert.
|
||||
await db.execute(
|
||||
update(SecFilingGap)
|
||||
.where(SecFilingGap.id.in_([g.id for g in lapsed]))
|
||||
.values(exempted_at=None)
|
||||
)
|
||||
|
||||
# Recovered rows are real data from an unexpected place — record where they
|
||||
# came from, so a wrong recovery is auditable rather than invisible.
|
||||
if staged.recovered:
|
||||
|
||||
Reference in New Issue
Block a user