dennisthiessenandClaude Opus 5
ad3e47dc50
bootstrap_universe(prune_missing=True) ran DELETE FROM tickers, cascading OHLCV, setups, scores and S/R levels away for any symbol that had merely dropped out of the index. Migration 032's own comment predicted this. The survivorship audit measured what it costs: 2.09 Sharpe against 1.57 on universe composition alone. Migration 035 adds untracked_on, the sibling of delisted_on — delisted means the security stopped trading, untracked means a prune took it out of the scanner. Nothing is deleted either way. Migration 036 separates three more things that were still conflated: universe_memberships what the index actually contained, when tickers.untracked_on excluded from scanning by a prune tickers.scan_disabled_on the operator's own "stop scanning this" tickers.tracking_source who owns the row: a bootstrap, or an operator The operator flag is its own column because with one column serving both writers, a prune -> re-entry cycle would clear an explicit "off" — re-entry cannot tell who set it. Re-entry clears untracked_on and never touches scan_disabled_on. untracked_on is NOT current index membership. The scheduled sync runs prune_missing=False, so a symbol can have left the index and still be scanned; the two legitimately differ and nothing asserts they agree. Tracking ownership is sticky towards manual: prune touches index-owned rows only, and a bootstrap never converts manual to index — a hand-added symbol that joins the index would otherwise lose its protection the moment it left again, which is exactly when it is needed. An explicit add of an index-owned symbol claims it as manual without touching either off-switch. Membership is observed, never asserted. first_seen_on is the first live sighting; absent_since is the first observation that found the symbol GONE, so the interval is [first_seen_on, absent_since) — named for what it stores rather than "last_seen_on", which would have claimed the opposite. Intervals are append-only, so a symbol that leaves and returns has two and a point-in-time query for the gap correctly answers "not a member". members_on() implements the boundary rule once. A date before the first live sync returns empty: that means "we were not looking", not "the index was empty". A failed fetch drives nothing. The cache and seed fallbacks mean the live fetch failed, and the seed lists are ~44 symbols against a ~500-symbol universe — a regression test caught a seed fallback untracking 2 of 3 tickers. Recording, pruning and clearing prune flags all require a live source; only adding still runs, so a cold start can still bootstrap from seeds. Migration decisions: existing rows become manual (fail-safe — a wrong guess costs an un-pruned symbol, not one swept out of the scanner), and existing untracked_on values are copied into scan_disabled_on, since the original writer is ambiguous and re-entry clears the prune column. Backtest eligibility is unchanged. _load_backtest_symbols deliberately does not use active_only, so today's scanner state cannot erase yesterday's candidates, and untracking a symbol still cannot silence a coverage gap in the report. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HAH7vVtn7qbcn1EVheecHQ
52 lines
1.7 KiB
Python
52 lines
1.7 KiB
Python
from app.models.ticker import Ticker
|
|
from app.models.ohlcv import OHLCVRecord
|
|
from app.models.user import User
|
|
from app.models.sentiment import SentimentScore
|
|
from app.models.fundamental import FundamentalData
|
|
from app.models.fundamental_snapshot import FundamentalSnapshot
|
|
from app.models.earnings_event import EarningsEvent
|
|
from app.models.data_import_run import DataImportRun
|
|
from app.models.score import DimensionScore, CompositeScore
|
|
from app.models.sr_level import SRLevel
|
|
from app.models.trade_setup import TradeSetup
|
|
from app.models.watchlist import WatchlistEntry
|
|
from app.models.settings import SystemSetting, IngestionProgress
|
|
from app.models.alert import AlertLog
|
|
from app.models.paper_trade import PaperTrade
|
|
from app.models.regime_snapshot import RegimeSnapshot
|
|
from app.models.regime_fundamental_observation import RegimeFundamentalObservation
|
|
from app.models.benchmark_price import BenchmarkPrice
|
|
from app.models.signal_context_snapshot import SignalContextSnapshot
|
|
from app.models.system_event import SystemEvent
|
|
from app.models.universe_membership import UniverseMembership
|
|
from app.models.sec_filing_gap import SecFilingGap
|
|
from app.models.job_run_state import JobRunState
|
|
|
|
__all__ = [
|
|
"Ticker",
|
|
"OHLCVRecord",
|
|
"User",
|
|
"SentimentScore",
|
|
"FundamentalData",
|
|
"FundamentalSnapshot",
|
|
"EarningsEvent",
|
|
"DataImportRun",
|
|
"DimensionScore",
|
|
"CompositeScore",
|
|
"SRLevel",
|
|
"TradeSetup",
|
|
"WatchlistEntry",
|
|
"SystemSetting",
|
|
"IngestionProgress",
|
|
"AlertLog",
|
|
"PaperTrade",
|
|
"RegimeSnapshot",
|
|
"RegimeFundamentalObservation",
|
|
"BenchmarkPrice",
|
|
"SignalContextSnapshot",
|
|
"SystemEvent",
|
|
"SecFilingGap",
|
|
"JobRunState",
|
|
"UniverseMembership",
|
|
]
|