fix(backtest): rebuild the recommendation on read, not only on run
Deploy / lint (push) Successful in 8s
Deploy / test (push) Successful in 1m13s
Deploy / deploy (push) Successful in 36s

Every fix so far only applied to reports generated after deploy. The cached
report is served verbatim, so it keeps the recommendation the OLD build stored —
quoting the legacy policy book, naming a rejected exit as "recommended", and
carrying no basis_lookback, which let the lookback selector default to 3y and
put 3-year tiles beside an all-history recommendation with no divergence notice.
Exactly the contradiction the last three commits set out to remove, silently
present on the first page load after deploy and until the next scheduled run
overwrote it.

The recommendation is a pure function of the numbers already in the report — its
own note says it is derived from them on every run — so it is now re-derived on
read. A corrected recommendation appears immediately instead of after the next
backtest. On failure it is dropped rather than falling back to the stored one,
which is the stale derivation this replaces.

The test drives the real shape: an old-build report with a legacy recommendation
written straight to the settings row, read back through get_backtest_report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-12 08:33:32 +02:00
co-authored by Claude Opus 5
parent 21a5fc8a52
commit 044a3447f6
2 changed files with 85 additions and 2 deletions
+56
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import json
import math
from datetime import date, timedelta
from types import SimpleNamespace
@@ -1862,3 +1863,58 @@ def test_build_recommendation_states_no_baseline_without_a_production_row():
assert "benchmark" not in topics
assert "exit" not in topics
async def test_cached_report_recommendation_is_rebuilt_on_read(session):
"""A report cached by an older build carries that build's recommendation.
Served verbatim, the page would show the legacy wording and no
basis_lookback — which let the lookback selector default elsewhere, putting
3y tiles beside an all-history recommendation with no warning. This is the
shape of the report sitting in production right now.
"""
from app.services.admin_service import update_setting
stale = {
"generated_at": "2026-08-12T05:00:00+00:00",
"tickers": 512, "candidates": 100, "qualified": 10,
"params": {"horizon_days": 30},
"overall_qualified": {"net_avg_r": 0.13, "net_avg_r_ex_top5": 0.20},
"portfolio_sim": {"policies": [
{"policy": "hold", "cagr_pct": 31.9, "total_return_pct": 175.0,
"spy_return_pct": 101.9, "max_drawdown_pct": 23.7},
]},
"portfolio_monitor": {
"production_strategy": "prod",
"runs": [{
"strategy": "prod", "lookback": "all", "lookback_label": "All history",
"cagr_pct": 40.0, "sharpe": 1.72, "max_drawdown_pct": 17.7,
"total_return_pct": 297.8, "spy_return_pct": 101.9,
}],
},
# What the old build stored: sourced from the policy book, and naming an
# exit the production book replaced.
"recommendation": {
"headline": "Trade the qualified list long-only; hold 30 trading days.",
"items": [
{"topic": "benchmark", "text": "Book vs SPY: beats buy-and-hold by "
"+73.1 points (+175.0% vs +101.9%)."},
{"topic": "robustness", "text": "Robustness: expectancy survives removing "
"the top 5% of winners (+0.20R net/trade "
"under the recommended 30d hold)."},
],
"note": "stale",
},
}
await update_setting(session, bt.KEY_REPORT, json.dumps(stale))
report = await bt.get_backtest_report(session)
assert report is not None
rec = report["recommendation"]
# Rebuilt: the basis is published, so the page cannot default elsewhere.
assert rec["basis_lookback"] == "all"
texts = " | ".join(i["text"] for i in rec["items"])
# ...and it quotes the production book, not the policy sim it used to.
assert "+297.8%" in texts and "175.0" not in texts
assert "recommended 30d hold" not in texts
assert "Production baseline" in (rec["headline"] or "")