Detection could retire an actively traded symbol — silently, since it then
vanishes from every signal. Three causes:
- Form 25 is filed per security class. An issuer removing its notes, preferred
or warrants files one while the common keeps trading. The filing's own
descriptionClassSecurity distinguishes them, so the primary document is now
fetched and read; anything not recognisably common equity is rejected, as is
anything unreadable (pre-2009 filings have no primary_doc.xml). Fail closed.
- Form 15 ends a reporting obligation and is no evidence trading stopped. The
whole family is dropped.
- A historical filing for a long-gone class could retire a symbol whose bars ran
years later, stamping the old date. Filings before the last bar (less a 30-day
lead for the exchange) are now ignored.
Rule 12d2-2 makes removal effective ten days after filing, so delisted_on is the
effective date rather than the filing date.
bootstrap_universe(prune_missing=True) still ran a cascading delete over
delisted rows, undoing the retention this branch exists for; it now skips them
and reports kept_delisted so the count is explicable.
clear_delisted had no route, which made "safe to automate because it is
reversible" false — reversal needed SQL. POST/DELETE /tickers/{symbol}/delisting
now mark and un-mark, giving an operator a non-destructive alternative to the
cascading DELETE that was the only option.
Shared-CIK siblings (GOOG/GOOGL) stay safe by construction: the probe is
per-symbol and gated on that symbol's own staleness, so a class that still
trades is never probed.
Not addressed: pruning a symbol merely dropped from the index still destroys its
history — the same survivorship problem in a different costume, needing a
tracked/membership state separate from delisting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mark_delisted returned early on any already-delisted row, so the sequence an
operator actually hits — mark EA by hand today, Form 25-NSE surfaces three days
later dated 2026-08-04 — left the estimated date and "manual" reason in place
permanently. Form 25 carries the real effective date, so it now replaces an
operator's estimate; a confirmed row is never downgraded or re-probed.
Also cover _get_ohlcv_priority_tickers, the one place active_only wraps a
compound select rather than a bare one — the unit suite reached none of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Retiring a symbol meant delete_ticker or bootstrap_universe(prune_missing),
both of which cascade through OHLCV, setups and scores. That destroys exactly
the history four research documents already apologise for: today's tracked
universe projected backward is survivorship-biased, and hard-deleting every
delisted name is what causes it. Keeping the rows preserves the option to fix
that — it does not fix it, which needs the replay to model a delisting as an
exit event.
tickers gains delisted_on / delisted_reason (migration 032). NULL means
actively traded.
The filter is opt-in via ticker_service.active_only rather than folded into a
shared getter: the registry and admin views deliberately keep delisted rows so
the delisting is visible, and a silent default would undo that. Applied to the
live path only — scanner, momentum ranking, scoring, breadth, fundamentals
candidates, SEC universe, earnings import, ingestion loops. run_backtest keeps
them on purpose.
Detection runs off OHLCV staleness, not off the SEC fundamentals import: that
importer stalls for days on unrelated Company-Facts gaps and would take
detection down with it. On a stale symbol the scheduler asks SEC for a Form
25/25-NSE/15 and retires it only on a hit, so a halt or a rename (SATS->ECHO)
keeps the existing warning. The probe waits 3 stale days so a market-data
outage cannot turn into one SEC request per symbol per run.
Safe to automate because it is reversible: clear_delisted un-retires a false
positive, where a delete had already taken the history.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dennisthiessenandClaude Opus 5