dennisthiessenandClaude Opus 5
6501b7e9a0
Retiring a symbol meant delete_ticker or bootstrap_universe(prune_missing), both of which cascade through OHLCV, setups and scores. That destroys exactly the history four research documents already apologise for: today's tracked universe projected backward is survivorship-biased, and hard-deleting every delisted name is what causes it. Keeping the rows preserves the option to fix that — it does not fix it, which needs the replay to model a delisting as an exit event. tickers gains delisted_on / delisted_reason (migration 032). NULL means actively traded. The filter is opt-in via ticker_service.active_only rather than folded into a shared getter: the registry and admin views deliberately keep delisted rows so the delisting is visible, and a silent default would undo that. Applied to the live path only — scanner, momentum ranking, scoring, breadth, fundamentals candidates, SEC universe, earnings import, ingestion loops. run_backtest keeps them on purpose. Detection runs off OHLCV staleness, not off the SEC fundamentals import: that importer stalls for days on unrelated Company-Facts gaps and would take detection down with it. On a stale symbol the scheduler asks SEC for a Form 25/25-NSE/15 and retires it only on a hit, so a halt or a rename (SATS->ECHO) keeps the existing warning. The probe waits 3 stale days so a market-data outage cannot turn into one SEC request per symbol per run. Safe to automate because it is reversible: clear_delisted un-retires a false positive, where a delete had already taken the history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
50 lines
2.0 KiB
Python
50 lines
2.0 KiB
Python
"""Record delisting on tickers instead of deleting them
|
|
|
|
Revision ID: 032
|
|
Revises: 031
|
|
Create Date: 2026-08-11 00:00:00.000000
|
|
|
|
Until now the only way to retire a symbol was ``delete_ticker`` (or
|
|
``bootstrap_universe(prune_missing=True)``), both of which cascade through
|
|
OHLCV, setups and scores. That destroys exactly the history four research
|
|
documents already apologise for: today's tracked universe projected backward
|
|
is survivorship-biased, and hard-deleting every delisted name is what causes
|
|
it. Keeping the rows preserves the option to fix that later — it does not fix
|
|
it by itself, which needs the replay to model a delisting as an exit event.
|
|
|
|
``delisted_on`` is the effective date (from SEC Form 25/25-NSE/15 where we can
|
|
confirm it, else the day it was marked); ``delisted_reason`` is a short code
|
|
for how we learned. NULL in both means actively traded — the live signal path
|
|
filters on that, while list and admin views keep showing the row so the
|
|
delisting is visible rather than silently absent.
|
|
|
|
Nullable and reversible by design: clearing ``delisted_on`` un-retires a
|
|
symbol, which is what makes automatic marking safe where a delete would not be.
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
from alembic import op
|
|
import sqlalchemy as sa
|
|
|
|
|
|
revision: str = "032"
|
|
down_revision: Union[str, None] = "031"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column("tickers", sa.Column("delisted_on", sa.Date(), nullable=True))
|
|
op.add_column(
|
|
"tickers", sa.Column("delisted_reason", sa.String(length=32), nullable=True)
|
|
)
|
|
# The live path filters "actively traded" on every universe scan; the index
|
|
# keeps that predicate cheap as delisted rows accumulate.
|
|
op.create_index("ix_tickers_delisted_on", "tickers", ["delisted_on"])
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_index("ix_tickers_delisted_on", table_name="tickers")
|
|
op.drop_column("tickers", "delisted_reason")
|
|
op.drop_column("tickers", "delisted_on")
|