feat(tickers): record delisting instead of deleting the symbol
Retiring a symbol meant delete_ticker or bootstrap_universe(prune_missing), both of which cascade through OHLCV, setups and scores. That destroys exactly the history four research documents already apologise for: today's tracked universe projected backward is survivorship-biased, and hard-deleting every delisted name is what causes it. Keeping the rows preserves the option to fix that — it does not fix it, which needs the replay to model a delisting as an exit event. tickers gains delisted_on / delisted_reason (migration 032). NULL means actively traded. The filter is opt-in via ticker_service.active_only rather than folded into a shared getter: the registry and admin views deliberately keep delisted rows so the delisting is visible, and a silent default would undo that. Applied to the live path only — scanner, momentum ranking, scoring, breadth, fundamentals candidates, SEC universe, earnings import, ingestion loops. run_backtest keeps them on purpose. Detection runs off OHLCV staleness, not off the SEC fundamentals import: that importer stalls for days on unrelated Company-Facts gaps and would take detection down with it. On a stale symbol the scheduler asks SEC for a Form 25/25-NSE/15 and retires it only on a hit, so a halt or a rename (SATS->ECHO) keeps the existing warning. The probe waits 3 stale days so a market-data outage cannot turn into one SEC request per symbol per run. Safe to automate because it is reversible: clear_delisted un-retires a false positive, where a delete had already taken the history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -45,6 +45,13 @@ _CA_VERIFY: str | bool = _CA if _CA and Path(_CA).exists() else True
|
||||
|
||||
_FORMS_10 = frozenset({"10-K", "10-Q", "10-K/A", "10-Q/A"})
|
||||
|
||||
# Exchange delisting (25 / 25-NSE) and registration termination (15 family).
|
||||
# Their presence is SEC confirming a security stopped trading — see
|
||||
# ``SecClient.delisting_filing``.
|
||||
_DELISTING_FORMS = frozenset({
|
||||
"25", "25-NSE", "15-12B", "15-12G", "15F-12B", "15F-12G",
|
||||
})
|
||||
|
||||
|
||||
class SecError(ProviderError):
|
||||
"""SEC request failed (403, exhausted 429/5xx, timeout, transport, parse)."""
|
||||
@@ -253,6 +260,34 @@ class SecClient:
|
||||
"filings": filings,
|
||||
}
|
||||
|
||||
async def delisting_filing(self, cik: int | str) -> dict[str, Any] | None:
|
||||
"""Newest exchange-delisting / deregistration filing, or ``None``.
|
||||
|
||||
Form 25/25-NSE strikes a security from listing; Form 15 terminates the
|
||||
registration. Either is SEC confirming the security stopped trading —
|
||||
which is what separates a real delisting from a multi-day halt or a
|
||||
ticker rename, neither of which files one. That precision is the reason
|
||||
this can mark a symbol automatically.
|
||||
|
||||
Reads ``filings.recent`` directly: ``submissions()`` keeps only the
|
||||
10-K/10-Q family, so these forms never survive its parser.
|
||||
"""
|
||||
base = await self.get_json(f"{_DATA}/submissions/CIK{cik10(cik)}.json")
|
||||
arrays = (base.get("filings") or {}).get("recent") or {}
|
||||
forms = arrays.get("form") or []
|
||||
dates = arrays.get("filingDate") or []
|
||||
best: dict[str, Any] | None = None
|
||||
for i, form in enumerate(forms):
|
||||
if form not in _DELISTING_FORMS or i >= len(dates) or not dates[i]:
|
||||
continue
|
||||
try:
|
||||
filed = date.fromisoformat(dates[i])
|
||||
except ValueError:
|
||||
continue
|
||||
if best is None or filed > best["filing_date"]:
|
||||
best = {"form": form, "filing_date": filed}
|
||||
return best
|
||||
|
||||
async def companyfacts(self, cik: int | str) -> dict[str, Any]:
|
||||
"""Raw companyfacts JSON ({cik, entityName, facts})."""
|
||||
return await self.get_json(f"{_DATA}/api/xbrl/companyfacts/CIK{cik10(cik)}.json")
|
||||
|
||||
Reference in New Issue
Block a user