feat(universe): separate scanner membership from retained history

bootstrap_universe(prune_missing=True) ran DELETE FROM tickers, cascading OHLCV,
setups, scores and S/R levels away for any symbol that had merely dropped out of
the index. Migration 032's own comment predicted this. The survivorship audit
measured what it costs: 2.09 Sharpe against 1.57 on universe composition alone.

Migration 035 adds untracked_on, the sibling of delisted_on — delisted means the
security stopped trading, untracked means a prune took it out of the scanner.
Nothing is deleted either way.

Migration 036 separates three more things that were still conflated:

  universe_memberships       what the index actually contained, when
  tickers.untracked_on       excluded from scanning by a prune
  tickers.scan_disabled_on   the operator's own "stop scanning this"
  tickers.tracking_source    who owns the row: a bootstrap, or an operator

The operator flag is its own column because with one column serving both
writers, a prune -> re-entry cycle would clear an explicit "off" — re-entry
cannot tell who set it. Re-entry clears untracked_on and never touches
scan_disabled_on.

untracked_on is NOT current index membership. The scheduled sync runs
prune_missing=False, so a symbol can have left the index and still be scanned;
the two legitimately differ and nothing asserts they agree.

Tracking ownership is sticky towards manual: prune touches index-owned rows
only, and a bootstrap never converts manual to index — a hand-added symbol that
joins the index would otherwise lose its protection the moment it left again,
which is exactly when it is needed. An explicit add of an index-owned symbol
claims it as manual without touching either off-switch.

Membership is observed, never asserted. first_seen_on is the first live
sighting; absent_since is the first observation that found the symbol GONE, so
the interval is [first_seen_on, absent_since) — named for what it stores rather
than "last_seen_on", which would have claimed the opposite. Intervals are
append-only, so a symbol that leaves and returns has two and a point-in-time
query for the gap correctly answers "not a member". members_on() implements the
boundary rule once. A date before the first live sync returns empty: that means
"we were not looking", not "the index was empty".

A failed fetch drives nothing. The cache and seed fallbacks mean the live fetch
failed, and the seed lists are ~44 symbols against a ~500-symbol universe — a
regression test caught a seed fallback untracking 2 of 3 tickers. Recording,
pruning and clearing prune flags all require a live source; only adding still
runs, so a cold start can still bootstrap from seeds.

Migration decisions: existing rows become manual (fail-safe — a wrong guess
costs an un-pruned symbol, not one swept out of the scanner), and existing
untracked_on values are copied into scan_disabled_on, since the original writer
is ambiguous and re-entry clears the prune column.

Backtest eligibility is unchanged. _load_backtest_symbols deliberately does not
use active_only, so today's scanner state cannot erase yesterday's candidates,
and untracking a symbol still cannot silence a coverage gap in the report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAH7vVtn7qbcn1EVheecHQ

app/routers/tickers.py

@@ -6,7 +6,12 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.dependencies import get_db, require_access
 from app.models.user import User
 from app.schemas.common import APIEnvelope
-from app.schemas.ticker import TickerCreate, TickerDelistingUpdate, TickerResponse
+from app.schemas.ticker import (
+    TickerCreate,
+    TickerDelistingUpdate,
+    TickerResponse,
+    TickerTrackingUpdate,
+)
 from app.services import ticker_service
 
 router = APIRouter(tags=["tickers"])
@@ -18,7 +23,23 @@ async def create_ticker(
     _user: User = Depends(require_access),
     db: AsyncSession = Depends(get_db),
 ):
-    """Add a new ticker to the registry."""
+    """Add a ticker to the registry, or claim an existing index-owned one.
+
+    Responses:
+
+    - **New symbol** — created as manually tracked and returned.
+    - **Existing, index-owned** — 200 with the existing row, now marked manually
+      tracked. Not a 409: the operator is saying "keep this stock", and refusing
+      while leaving it prunable would defeat the protection they just asked for.
+      Ownership only ever moves towards manual, so a bootstrap cannot take it
+      back when the symbol next joins the index.
+    - **Existing, already manual** — 409 Duplicate. Nothing left to change.
+
+    Claiming never clears ``scan_disabled_on`` or ``untracked_on``: wanting a
+    stock kept is not the same as asking for it to be scanned, and silently
+    re-enabling something switched off by hand would undo a decision nobody
+    revisited. ``POST /tickers/{symbol}/tracking`` is the explicit control.
+    """
     ticker = await ticker_service.add_ticker(db, body.symbol)
     return APIEnvelope(
         status="success",
@@ -70,6 +91,24 @@ async def mark_ticker_delisted(
     return APIEnvelope(status="success", data={"changed": changed})
 
 
+@router.post("/tickers/{symbol}/tracking", response_model=APIEnvelope)
+async def set_ticker_tracking(
+    symbol: str,
+    body: TickerTrackingUpdate,
+    _user: User = Depends(require_access),
+    db: AsyncSession = Depends(get_db),
+):
+    """Include or exclude one symbol from scanning, keeping its history.
+
+    Untracking is not delisting — the security still trades, we just stop
+    scanning it. The case this exists for: a manually added symbol no provider
+    can serve, which would otherwise sit in the universe forever producing no
+    bars and flagging every backtest as incomplete.
+    """
+    changed = await ticker_service.set_tracked(db, symbol, tracked=body.tracked)
+    return APIEnvelope(status="success", data={"changed": changed})
+
+
 @router.delete("/tickers/{symbol}/delisting", response_model=APIEnvelope)
 async def clear_ticker_delisting(
     symbol: str,