CI failed on test_run_backtest_rolls_back_a_failed_ticker_fetch: it asserted
"degraded" and got "failed". The code was right and the test was wrong.
run_backtest calls refresh_benchmark_prices unless it is in offline snapshot
mode, so whether SPY closes exist depends on network access and credentials.
That decides the status, because a missing benchmark means residual 12-1
momentum silently ran as raw 12-1 — a run that changed its own signal
definition, which the ledger reports as failed. A dev box reaches Alpaca and
sees "degraded"; CI does not and correctly sees "failed".
Any test asserting on status now pins the benchmark instead of inheriting
whatever the machine can reach. The every-ticker-fails test is pinned too, so
its "failed" verdict is attributable to the fetch failures rather than passing
for the wrong reason on a benchmark-less box, and the healthy-completeness test
can now assert "ok" outright.
Reproduced with BACKTEST_SNAPSHOT_OFFLINE=1, which skips the refresh and leaves
the benchmark empty exactly as CI has it: the old test fails there, the new one
passes. Full suite verified green both with and without a reachable benchmark.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HAH7vVtn7qbcn1EVheecHQ
dennisthiessenandClaude Opus 5
25e821435d