parallelize the backtest across worker processes (true multi-core)
Deploy / lint (push) Successful in 6s
Deploy / test (push) Successful in 38s
Deploy / deploy (push) Successful in 25s

The replay was CPU-bound and single-core: the earlier asyncio.to_thread offload
kept the API responsive but, because of the GIL, ran on one core. Per-ticker
replay is independent, so fan it out across worker processes (which sidestep the
GIL) for real multi-core speedup.

- New `settings.backtest_workers` (default 4), capped to cpu_count-1 so a core
  stays free for the web server.
- Uses a `forkserver` context (workers forked from a clean single-threaded
  server — avoids the fork-with-threads deadlock); falls back to `fork`. On
  spawn-only platforms (Windows) and for 1-ticker runs it uses the thread path,
  so dev/tests are unaffected.
- Worker takes primitive column arrays (cheap to pickle), rebuilds bars, and
  returns (candidates, plain-dict signal series) — both picklable across the
  process boundary. Bars are still fetched in the event loop (ORM-safe).
- Pool creation is guarded: if the pool can't start, the job falls back to the
  sequential thread path instead of failing.

334 backend tests pass (parallel path is POSIX/server-only, so it's covered by
construction + the picklability/worker-count tests; the thread fallback is
exercised by the run_backtest smoke test).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-23 23:20:20 +02:00
parent e71c07e554
commit 7060b9a019
3 changed files with 171 additions and 35 deletions
+6
View File
@@ -73,6 +73,12 @@ class Settings(BaseSettings):
# backtests become computable. ~252 trading days/year.
ohlcv_history_days: int = 1825
# Backtest parallelism: replay tickers across this many worker processes on
# POSIX (forkserver), capped to cpu_count-1 so a core stays free for the web
# server. 1 disables it (sequential). No effect on Windows / spawn-only
# platforms — those fall back to a single worker thread.
backtest_workers: int = 4
# Database Pool
db_pool_size: int = 5
db_pool_timeout: int = 30