fix: match shadow book to its pipeline's scan by run id, not timestamp

A manually triggered rr_scanner and the scheduled near-close pipeline are
separate APScheduler jobs; max_instances=1 serialises a job only against
itself, so they can overlap. A manual scan starting just before the
pipeline can finish just after it began and overwrite the scan markers.
Its completion timestamp is then later than the pipeline start, so the
previous 'completed >= pipeline_start' check accepted its batch as though
it were the pipeline's own -- exactly when the pipeline's scan may have
failed.

Replace the timestamp comparison with an exact run-id match. A new
pipeline_run module holds a per-task run-id contextvar (separate module so
the scanner and scheduler import it without a cycle). _run_pipeline binds a
fresh id per invocation; scan_all_tickers stamps that id -- or a fresh one
when run standalone -- into the scan markers, written with started/completed
in a single commit. The shadow step requires the stored run id to equal its
pipeline's id exactly, so a concurrent manual scan (its own id) or a failed
pipeline scan (a prior run's id) can never be mistaken for it. Direct Admin
triggers have no pipeline context and keep the freshness fallback.

Known residual: the id match governs whether shadow proceeds; setup
selection remains detected_at >= scan start, so a fully per-run setup
isolation would need a run_id column on trade_setups (not required here).

Tests cover the reported race (manual scan finishing last is refused), a
failed pipeline scan, the id-match accept path, and contextvar propagation
and non-leakage across tasks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-21 10:47:32 +02:00
co-authored by Claude Fable 5
parent 807cc4bdfa
commit 05ba138d35
6 changed files with 220 additions and 71 deletions
+24 -24
View File
@@ -173,24 +173,22 @@ async def _last_scan_start(
db: AsyncSession,
*,
now: datetime,
require_scan_after: datetime | None = None,
expected_run_id: str | None = None,
) -> datetime | None:
"""Start of the last successful scan, if it is the one we may act on.
"""Start of the scan we may act on, or None if there is none.
Returns None — "no scan to act on" — unless the scanner's COMPLETED marker
passes the applicable check:
* ``expected_run_id`` set (pipeline step): the stored run id must match it
exactly. This is the airtight guarantee — a scan that was disabled or
failed in *this* pipeline never stamped this id, and a concurrent manual
scan (a separate APScheduler job, not serialised against the pipeline)
stamps its own id even when it finishes last, so neither can be mistaken
for the pipeline's own scan. Timestamp order alone cannot tell them apart.
* ``expected_run_id`` None (direct Admin trigger): fall back to the freshness
window. There is no pipeline scan to bind to, so acting on a recent scan is
the operator's explicit choice.
* ``require_scan_after`` set (pipeline step): the scan must have completed
at/after the pipeline began. This is the airtight guarantee — a scan that
was disabled or failed in *this* pipeline leaves the marker at a previous
run, and a fresh but earlier manual scan completed before the pipeline
started, so neither can stand in for the pipeline's own scan.
* ``require_scan_after`` None (direct Admin trigger): fall back to the
freshness window. There is no pipeline scan to bind to, so acting on a
recent scan is the operator's explicit choice.
Either way the returned value is the scan's start, the lower bound for the
setups belonging to that run.
The three markers are written in one commit, so the returned STARTED belongs
to the same run as the matched RUN_ID and correctly bounds its setups.
"""
from app.services import rr_scanner_service as rr
@@ -198,10 +196,11 @@ async def _last_scan_start(
completed = _parse_dt(
await settings_store.get_value(db, rr.KEY_LAST_SCAN_COMPLETED)
)
run_id = await settings_store.get_value(db, rr.KEY_LAST_SCAN_RUN_ID)
if started is None or completed is None:
return None
if require_scan_after is not None:
if completed < require_scan_after:
if expected_run_id is not None:
if not run_id or run_id != expected_run_id:
return None
elif now - completed > MAX_SCAN_AGE:
return None
@@ -222,7 +221,7 @@ async def _todays_qualified_setups(
config: dict,
*,
now: datetime,
require_scan_after: datetime | None = None,
expected_run_id: str | None = None,
) -> list[TradeSetup]:
"""Long-only qualified setups from the scan that just ran, best rank first.
@@ -241,7 +240,7 @@ async def _todays_qualified_setups(
4. Qualify, then rank by ``strategy_rank`` (unranked sort last).
"""
run_start = await _last_scan_start(
db, now=now, require_scan_after=require_scan_after
db, now=now, expected_run_id=expected_run_id
)
if run_start is None:
return []
@@ -275,7 +274,7 @@ async def open_shadow_positions(
*,
activation_config: dict,
opened_at: datetime | None = None,
require_scan_after: datetime | None = None,
expected_run_id: str | None = None,
) -> dict:
"""Fill free capacity with the top-ranked qualified setups.
@@ -283,9 +282,10 @@ async def open_shadow_positions(
skip anything already held or locked out by post-stop gate-reset, and stop
at capacity. Returns a summary for the job log.
``require_scan_after`` binds this run to a scan that completed at/after that
instant (the pipeline's start), so a stale or manual scan cannot substitute
for a scan that failed in this pipeline pass. See ``_last_scan_start``.
``expected_run_id`` binds this run to the scan that stamped that exact id
(the pipeline's own scan), so a scan that failed in this pipeline — or a
concurrent manual scan that finished last — cannot substitute for it. See
``_last_scan_start``.
"""
summary = {
"opened": 0,
@@ -312,7 +312,7 @@ async def open_shadow_positions(
timestamp = opened_at or datetime.now(timezone.utc)
candidates = await _todays_qualified_setups(
db, activation_config, now=timestamp, require_scan_after=require_scan_after
db, activation_config, now=timestamp, expected_run_id=expected_run_id
)
for setup in candidates:
if free_slots <= 0: