Compare commits

...
5 Commits
Author SHA1 Message Date
dennisthiessenandClaude Opus 5 044a3447f6 fix(backtest): rebuild the recommendation on read, not only on run
Deploy / lint (push) Successful in 8s
Deploy / test (push) Successful in 1m13s
Deploy / deploy (push) Successful in 36s
Every fix so far only applied to reports generated after deploy. The cached
report is served verbatim, so it keeps the recommendation the OLD build stored —
quoting the legacy policy book, naming a rejected exit as "recommended", and
carrying no basis_lookback, which let the lookback selector default to 3y and
put 3-year tiles beside an all-history recommendation with no divergence notice.
Exactly the contradiction the last three commits set out to remove, silently
present on the first page load after deploy and until the next scheduled run
overwrote it.

The recommendation is a pure function of the numbers already in the report — its
own note says it is derived from them on every run — so it is now re-derived on
read. A corrected recommendation appears immediately instead of after the next
backtest. On failure it is dropped rather than falling back to the stored one,
which is the stale derivation this replaces.

The test drives the real shape: an old-build report with a legacy recommendation
written straight to the settings row, read back through get_backtest_report.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 08:33:32 +02:00
dennisthiessenandClaude Opus 5 21a5fc8a52 fix(backtest): flag a lookback the recommendation was not computed on
Selecting a different window or a comparison strategy silently made the tiles
stop matching the recommendation below, which is baked into the report and
cannot follow a dropdown. On load they now agree by construction; moving off
that basis says so.

Also: an absent production row produced no headline and no benchmark, but any
passing gate finding still rendered a green "no warnings" chip — a success badge
for missing data, directly beside "this report predates the portfolio monitor".
Missing baseline now reads "baseline unavailable".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 08:03:12 +02:00
dennisthiessenandClaude Opus 5 11dffcd695 fix(backtest): one window, and stop calling a rejected exit "recommended"
Two ways the recommendation still disagreed with the page it sits on.

It preferred the "all" monitor row while the UI defaulted its selector to "3y",
so a default page load showed one set of returns in the tiles and a different
set in the recommendation. The row it used is now published as basis_lookback
and the page defaults to it, so the two cannot open on different windows. The
test fixture gains a second monitor row with different numbers — with only an
"all" row present, a lookback mix-up could not fail.

Robustness picked its basis between "the recommended Nd hold" and "the S/R
target exit", naming an exit the production book replaced as recommended. There
is no ATR-trail ex-top-5% figure in the report, so it now always reports the
gate-level grading and says that is what it is, rather than dressing a legacy
number as a verdict on the production book. time_exit_sweep is no longer read
here at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 08:03:12 +02:00
dennisthiessenandClaude Opus 5 3a2d548610 feat(backtest): make the tiles answer "is that good?", and stop the layout jumping
Five UI problems, all reported from using the page.

Expanding "How this is measured" shoved every control down, because the
disclosure and the run controls shared one flex row. They no longer do: run
status and the controls that start a new run sit together on one line, and the
explainer is below them where growing it moves nothing.

A long strategy name wrapped the dropdown trigger onto three lines and dragged
the row out of alignment. The trigger now truncates with the full text on hover
— a wrapping dropdown is broken anywhere, so the fix is in the primitive — and
the twelve-character "Production: " prefix is a bullet.

"Sortino 2.72" answered nothing. Each risk-adjusted metric now carries a meter:
a track showing where the value sits, ticks at the band edges, and the band word.
Colour never travels alone. Bands are deliberately stricter than textbook ranges
because this universe is today's survivors replayed backward, which flatters
every ratio — that caveat is stated next to them rather than left implied.

The two tile rows were different sizes, which read as inconsistent rather than
as hierarchy. Every tile is the same size now and grouping carries the ranking:
top row is raw outcome and takes no meters, second row is risk-adjusted ratios
and all take meters. Sharpe moved down to join them — it is one of those ratios,
and leaving it above made it the only metered tile in a row of bare ones.

The recommendation led with a long bold sentence that describes the
configuration, not a verdict, while the actual findings were small grey text.
Findings now come first, each split into label and detail on the colon the
backend strings already carry, and the configuration is a footer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 23:25:11 +02:00
dennisthiessenandClaude Opus 5 28b3273150 fix(backtest): quote one book, not two
The recommendation's "Book vs SPY" line read from portfolio_sim — the hold/target
policy book — while the tiles directly above read from portfolio_monitor, the
production ATR-trail book. Same SPY figure, different portfolio return, on one
screen. It now reads the same production row the tiles do.

Also removed, for the same reason: the "legacy exit diagnostic" comparing hold
against the S/R target. Both are exits the production book replaced, so a
recommendation between them could not lead to an action. And the fallback
headline, which advised the fixed-hold exit whenever a report had no production
row — a report that cannot describe the production baseline now states none.

portfolio_sim stays in the report payload: scripts/run_backtest_snapshot.py and
reports/compare_reports.py read it, and it is no longer surfaced in the UI. The
test fixture now carries a production monitor whose numbers differ from its
policy sim, so re-sourcing that line from the old place fails rather than passes
unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 23:24:58 +02:00
9 changed files with 503 additions and 159 deletions
+60 -63
View File
@@ -3944,40 +3944,11 @@ def _build_recommendation(report: dict) -> dict:
})
q = report.get("overall_qualified") or {}
target_net = q.get("net_avg_r")
# Legacy diagnostic: target/stop race vs the best fixed hold.
time_rows = [r for r in report.get("time_exit_sweep") or [] if r.get("net_avg_r") is not None]
best_hold = max(time_rows, key=lambda r: r["net_avg_r"], default=None)
sim_rows = {
p.get("policy"): p
for p in (report.get("portfolio_sim") or {}).get("policies", [])
}
hold_sim = sim_rows.get("hold")
if best_hold is not None and target_net is not None:
if best_hold["net_avg_r"] > target_net + _EXIT_SWITCH_THRESHOLD:
text = (
f"Legacy exit diagnostic: hold {best_hold['hold_days']} trading days with the initial stop "
f"({best_hold['net_avg_r']:+.2f}R net/trade vs {target_net:+.2f}R for the S/R target exit)."
)
target_sim = sim_rows.get("target")
if (
hold_sim is not None and target_sim is not None
and hold_sim.get("cagr_pct") is not None and target_sim.get("cagr_pct") is not None
):
text += (
f" The simulated book agrees: {hold_sim['cagr_pct']:+.1f}% vs "
f"{target_sim['cagr_pct']:+.1f}% CAGR at similar drawdown."
)
items.append({"topic": "exit", "text": text})
else:
items.append({
"topic": "exit",
"text": (
f"Legacy exit diagnostic: keep the S/R target exit ({target_net:+.2f}R net/trade) — "
"no fixed hold beats it by a meaningful margin."
),
})
# Nothing here reads time_exit_sweep any more. The hold-vs-target comparison
# is not reported (both are exits the production book replaced, so choosing
# between them cannot lead to an action), and the robustness check below no
# longer picks its basis from them either.
# Gate floors, judged under the hold exit (the ablation's Hold column).
ablation = {r["variant"]: r for r in report.get("gate_ablation") or []}
@@ -4025,33 +3996,32 @@ def _build_recommendation(report: dict) -> dict:
),
})
# Book vs benchmark.
book = hold_sim or sim_rows.get("target")
if book is not None and book.get("spy_return_pct") is not None:
edge = book["total_return_pct"] - book["spy_return_pct"]
# Book vs benchmark — read from the SAME production monitor row the page
# shows in its tiles. It used to read the hold/target policy sim, so the
# recommendation quoted a different portfolio return than the tile directly
# above it, against an identical SPY figure. Those policies are legacy
# diagnostics; the production book is the ATR trail.
if production_row is not None and production_row.get("spy_return_pct") is not None:
edge = production_row["total_return_pct"] - production_row["spy_return_pct"]
verdict = "beats" if edge > 0 else "LAGS"
items.append({
"topic": "benchmark",
"text": (
f"Book vs SPY: {verdict} buy-and-hold by {edge:+.1f} points "
f"({book['total_return_pct']:+.1f}% vs {book['spy_return_pct']:+.1f}%), "
f"max drawdown {book['max_drawdown_pct']:.1f}%."
f"({production_row['total_return_pct']:+.1f}% vs "
f"{production_row['spy_return_pct']:+.1f}%)."
),
})
# Robustness: does the edge survive without the biggest winners? Judged on
# the RECOMMENDED exit — outlier dependence under an exit we'd abandon
# would be the wrong warning.
hold_recommended = (
best_hold is not None and target_net is not None
and best_hold["net_avg_r"] > target_net + _EXIT_SWITCH_THRESHOLD
)
if hold_recommended and best_hold.get("net_avg_r_ex_top5") is not None:
trimmed = best_hold["net_avg_r_ex_top5"]
basis = f"under the recommended {best_hold['hold_days']}d hold"
else:
# Robustness: does the edge survive without the biggest winners?
#
# There is no ATR-trail equivalent of this number in the report — the only
# ex-top-5% figure is the gate-level target/stop grading. So it is reported
# on that basis and SAYS SO, rather than being dressed up as a verdict on the
# production book. It used to pick between "the recommended Nd hold" and "the
# S/R target exit", naming a rejected exit as recommended.
trimmed = q.get("net_avg_r_ex_top5")
basis = "under the S/R target exit"
basis = "gate-level grading, not the production ATR-trail book"
if trimmed is not None:
if trimmed > 0:
items.append({
@@ -4072,20 +4042,20 @@ def _build_recommendation(report: dict) -> dict:
),
})
if headline is None and hold_recommended:
cagr_note = (
f" (~{hold_sim['cagr_pct']:.0f}% CAGR simulated)"
if hold_sim is not None and hold_sim.get("cagr_pct") is not None
else ""
)
headline = (
f"Trade the qualified list long-only; hold {best_hold['hold_days']} trading days "
f"with the initial ATR stop{cagr_note}."
)
# No fallback headline. It used to recommend the fixed-hold exit whenever the
# portfolio monitor was missing, which meant a report without a production
# row advised an exit the production book had already replaced. A report that
# cannot describe the production baseline states no baseline.
return {
"headline": headline,
"items": items,
# Which monitor row every production/benchmark figure above was read
# from. The page defaults its lookback selector to this, so the tiles and
# the recommendation cannot open on different windows — they used to,
# because this preferred "all" while the UI defaulted to "3y".
"basis_lookback": (production_row or {}).get("lookback"),
"basis_lookback_label": (production_row or {}).get("lookback_label"),
"note": "Derived from this report's numbers on every run — the advice flips if the data does.",
}
@@ -4466,11 +4436,38 @@ async def run_and_store(
async def get_backtest_report(db: AsyncSession) -> dict | None:
"""Return the last cached backtest report, or None if never run."""
"""Return the last cached backtest report, or None if never run.
The recommendation is **re-derived from the cached report** rather than
served as stored. It is a pure function of the numbers already in the
report — the payload's own note says it is derived from them on every run —
so recomputing costs nothing and keeps one class of bug out:
A report cached by an older build carries that build's recommendation. After
a change to how the recommendation is sourced, the page would keep showing
the old one — quoting the legacy policy book, naming a rejected exit as
"recommended", and omitting ``basis_lookback``, which in turn let the
lookback selector default somewhere else. The result was the exact
tiles-disagree-with-recommendation contradiction this rebuild exists to
prevent, silently, until the next scheduled run happened to overwrite it.
Re-deriving means a corrected recommendation appears on the first page load
after deploy instead of after the next backtest.
"""
setting = await settings_store.get_setting(db, KEY_REPORT)
if setting is None:
return None
try:
return json.loads(setting.value)
report = json.loads(setting.value)
except (TypeError, ValueError):
return None
if not isinstance(report, dict):
return None
try:
report["recommendation"] = _build_recommendation(report)
except Exception:
# Fail closed: drop it rather than fall back to the stored one, which is
# precisely the stale derivation this rebuild is here to replace.
logger.exception("Could not rebuild the backtest recommendation; omitting it")
report.pop("recommendation", None)
return report
@@ -42,8 +42,19 @@ export function BacktestPanel() {
const monitor = report?.portfolio_monitor ?? null;
const activeStrategy =
selectedStrategy || monitor?.production_strategy || monitor?.strategies[0]?.strategy || '';
// Default to the window the recommendation was computed on, so the tiles and
// the recommendation never open showing different numbers. They used to: the
// backend preferred "all" while this defaulted to "3y". The 3y fallback is
// only for reports predating basis_lookback.
const basisLookback = report?.recommendation?.basis_lookback ?? null;
const activeLookback =
selectedLookback || (monitor?.lookbacks.some((l) => l.lookback === '3y') ? '3y' : monitor?.lookbacks[0]?.lookback) || '';
selectedLookback ||
(basisLookback && monitor?.lookbacks.some((l) => l.lookback === basisLookback)
? basisLookback
: monitor?.lookbacks.some((l) => l.lookback === '3y')
? '3y'
: monitor?.lookbacks[0]?.lookback) ||
'';
const monitorRun = useMemo(
() =>
monitor?.runs.find((row) => row.strategy === activeStrategy && row.lookback === activeLookback) ??
@@ -70,23 +81,30 @@ export function BacktestPanel() {
return (
<Section title="Is the strategy working?" hint="portfolio simulation of the promoted strategy vs S&P 500">
<div className="space-y-4">
<div className="flex flex-wrap items-start justify-between gap-3">
<Disclosure summary="How this is measured">
<p className="max-w-2xl text-xs text-gray-400">
The backtest replays the current config at the selected cadence at each point the setup is
rebuilt using only data up to that day (no lookahead) and the following ~30 trading days decide
its outcome then simulates one capital-constrained book against the S&P 500. Sentiment and
fundamentals are held neutral (no point-in-time history). ~6 months is roughly one market regime,
so read it as directional.
{/* Run status and the controls that start a new run, on one line. The
explainer sits BELOW this row rather than beside it — sharing a flex
row meant expanding it shoved every control down the page. */}
<div className="flex flex-wrap items-end justify-between gap-3">
<div className="min-w-0">
<p className="section-index">Last run</p>
{report ? (
<p className="mt-1 text-xs text-gray-400">
{timeAgo(report.generated_at)} · {report.tickers} tickers ·{' '}
{report.candidates} setups ({report.qualified} qualified) ·{' '}
{report.params.entry_cadence ?? 'weekly'},{' '}
{report.params.horizon_days}d horizon
{report.params.cost_per_side_pct != null && (
<> · net of {report.params.cost_per_side_pct}%/side</>
)}
{' · '}
<span className={report.params.is_production_target_model === false ? 'text-amber-300' : 'text-blue-300'}>
{report.params.target_model_label ?? 'Unknown (legacy report)'}
</span>
</p>
<p className="mt-2 max-w-2xl text-xs text-gray-400">
<strong className="text-gray-300">Live GTL</strong> is the exact target path the scanner and the
scheduled backtest use; <strong className="text-gray-300">Structural S/R</strong> is a comparison
arm sourcing targets from chart structure. <strong className="text-gray-300">Weekly</strong> steps
five sessions at a time and is what the server runs; <strong className="text-gray-300">Daily</strong>
{' '}is roughly 5× the replay work.
</p>
</Disclosure>
) : (
<p className="mt-1 text-xs text-gray-500">Never run</p>
)}
</div>
{/* flex-wrap is load-bearing: two dropdowns plus the button overflow a
narrow viewport otherwise. */}
@@ -112,11 +130,30 @@ export function BacktestPanel() {
/>
</div>
<Button onClick={() => run.mutate()} loading={run.isPending} className="shrink-0">
{run.isPending ? 'Starting…' : report ? 'Re-run backtest' : 'Run backtest'}
{run.isPending ? 'Starting…' : report ? 'Re-run' : 'Run backtest'}
</Button>
</div>
</div>
<div>
<Disclosure summary="How this is measured">
<p className="max-w-2xl text-xs text-gray-400">
The backtest replays the current config at the selected cadence at each point the setup is
rebuilt using only data up to that day (no lookahead) and the following ~30 trading days decide
its outcome then simulates one capital-constrained book against the S&P 500. Sentiment and
fundamentals are held neutral (no point-in-time history). ~6 months is roughly one market regime,
so read it as directional.
</p>
<p className="mt-2 max-w-2xl text-xs text-gray-400">
<strong className="text-gray-300">Live GTL</strong> is the exact target path the scanner and the
scheduled backtest use; <strong className="text-gray-300">Structural S/R</strong> is a comparison
arm sourcing targets from chart structure. <strong className="text-gray-300">Weekly</strong> steps
five sessions at a time and is what the server runs; <strong className="text-gray-300">Daily</strong>
{' '}is roughly 5× the replay work.
</p>
</Disclosure>
</div>
{/* Only surfaced for non-default choices — zero noise on the common path,
but a non-production selection still announces itself, which is what
the old always-amber cards were really for. */}
@@ -142,19 +179,6 @@ export function BacktestPanel() {
{report && (
<>
<p className="text-[11px] text-gray-500">
Ran {timeAgo(report.generated_at)} · {report.tickers} tickers · {report.candidates} setups
({report.qualified} qualified) · {report.params.entry_cadence ?? 'weekly'} cadence,
{' '}{report.params.horizon_days}-day horizon
{report.params.cost_per_side_pct != null && (
<> · net of {report.params.cost_per_side_pct}%/side costs</>
)}
{' '}· target model:{' '}
<span className={report.params.is_production_target_model === false ? 'text-amber-300' : 'text-blue-300'}>
{report.params.target_model_label ?? 'Unknown (legacy report)'}
</span>
</p>
<PortfolioMonitorPanel
monitor={monitor}
monitorRun={monitorRun}
@@ -162,6 +186,9 @@ export function BacktestPanel() {
activeLookback={activeLookback}
onStrategyChange={setSelectedStrategy}
onLookbackChange={setSelectedLookback}
basisLookback={basisLookback}
basisLookbackLabel={report.recommendation?.basis_lookback_label ?? null}
productionStrategy={monitor?.production_strategy ?? null}
/>
{report.recommendation && (
@@ -4,15 +4,14 @@ import type { BacktestRecommendation } from '../../lib/types';
/**
* The verdict, ahead of the tuning detail.
*
* All eight findings used to render as equal-weight bullets, so "does this
* strategy work" sat in the same visual register as "which cutoff scored best".
* `topic` splits them: the three that answer the question stay inline, the rest
* collapse.
* Two problems this solves. All eight findings used to render as equal-weight
* bullets, so "does this strategy work" sat in the same register as "which
* cutoff scored best". And the headline — which is a *description of the
* config*, not a verdict — was the loudest thing on the card while every actual
* finding was small grey text.
*
* No topic chips — every backend string already self-prefixes ("Gate: …",
* "Robustness: …"), so a chip would render "GATE │ Gate: …", and stripping the
* prefix would drop real information ("(3y)" carries the lookback, "Legacy"
* qualifies the diagnostic).
* So: findings first, each split into a label and its detail; the config
* description demoted to a footer where it belongs.
*/
const PRIMARY_TOPICS = new Set(['production', 'benchmark', 'robustness']);
@@ -25,6 +24,43 @@ function isWarning(text: string): boolean {
return text.includes('WARNING') || text.includes('LAGS');
}
/**
* Every backend string self-prefixes ("Gate: keep the R:R floor…"), so the
* prefix IS the label — no need for a chip that would just repeat it, and no
* need to reword anything server-side. Split on the first colon; if a string
* ever stops carrying one, it renders whole as detail.
*/
function splitLabel(text: string): { label: string | null; detail: string } {
const at = text.indexOf(': ');
if (at === -1 || at > 48) return { label: null, detail: text };
return { label: text.slice(0, at), detail: text.slice(at + 2) };
}
function Finding({ text, primary }: { text: string; primary: boolean }) {
const warn = isWarning(text);
const { label, detail } = splitLabel(text);
return (
<li className="flex flex-col gap-0.5 sm:flex-row sm:gap-3">
{label && (
<span
className={`shrink-0 text-[11px] font-semibold uppercase tracking-wider sm:w-44 sm:pt-0.5 ${
warn ? 'text-amber-400' : 'text-gray-500'
}`}
>
{label}
</span>
)}
<span
className={`${primary ? 'text-sm' : 'text-xs'} ${
warn ? 'text-amber-300' : primary ? 'text-gray-200' : 'text-gray-400'
}`}
>
{detail}
</span>
</li>
);
}
export function BacktestRecommendationCard({
recommendation,
}: {
@@ -45,30 +81,43 @@ export function BacktestRecommendationCard({
<div className="glass border border-blue-400/20 p-4">
<div className="flex flex-wrap items-center justify-between gap-2">
<p className="section-index">What this backtest recommends</p>
{warningCount > 0 && (
{/* No headline means the backend found no production monitor row, so
nothing here describes the production book. Zero keyword warnings
is then absence of data, not a clean bill of health — a green chip
beside "this report predates the portfolio monitor" would be a
success badge for missing data. */}
{!recommendation.headline ? (
<span className="rounded-full border border-white/15 bg-white/[0.05] px-2 py-0.5 text-[10px] font-semibold uppercase tracking-wider text-gray-400">
baseline unavailable
</span>
) : warningCount > 0 ? (
<span className="rounded-full border border-amber-400/40 bg-amber-400/10 px-2 py-0.5 text-[10px] font-semibold uppercase tracking-wider text-amber-300">
{warningCount} warning{warningCount > 1 ? 's' : ''}
</span>
) : (
<span className="rounded-full border border-emerald-400/30 bg-emerald-400/10 px-2 py-0.5 text-[10px] font-semibold uppercase tracking-wider text-emerald-300">
no warnings
</span>
)}
</div>
{recommendation.headline && (
<p className="mt-1.5 text-sm font-semibold text-gray-100">{recommendation.headline}</p>
)}
{primary.length > 0 && (
<ul className="mt-3 space-y-1.5 border-t border-white/[0.06] pt-3">
<ul className="mt-3 space-y-2.5">
{primary.map((item) => (
<li
key={item.topic + item.text}
className={`text-xs ${isWarning(item.text) ? 'text-amber-400' : 'text-gray-300'}`}
>
{item.text}
</li>
<Finding key={item.topic + item.text} text={item.text} primary />
))}
</ul>
)}
{/* The config description, demoted: it says what the strategy IS, which
is context for the findings above rather than a finding itself. */}
{recommendation.headline && (
<div className="mt-3 border-t border-white/[0.06] pt-3">
<p className="section-index">Configuration under test</p>
<p className="mt-1 text-xs leading-relaxed text-gray-500">{recommendation.headline}</p>
</div>
)}
{recommendation.note && (
<p className="mt-2 text-[11px] text-gray-600">{recommendation.note}</p>
)}
@@ -77,12 +126,10 @@ export function BacktestRecommendationCard({
{/* Outside the card body on purpose: Disclosure renders its own glass-sm
panel, so nesting it inside the bordered card double-frames it. */}
{secondary.length > 0 && (
<Disclosure summary={`Gate, exit and cutoff detail (${secondary.length})`}>
<ul className="space-y-1.5">
<Disclosure summary={`Gate and cutoff detail (${secondary.length})`}>
<ul className="space-y-2">
{secondary.map((item) => (
<li key={item.topic + item.text} className="text-xs text-gray-400">
{item.text}
</li>
<Finding key={item.topic + item.text} text={item.text} primary={false} />
))}
</ul>
</Disclosure>
@@ -30,6 +30,9 @@ export function PortfolioMonitorPanel({
activeLookback,
onStrategyChange,
onLookbackChange,
basisLookback = null,
basisLookbackLabel = null,
productionStrategy = null,
}: {
monitor: BacktestPortfolioMonitor | null | undefined;
monitorRun: BacktestPortfolioMonitorRun | null | undefined;
@@ -37,6 +40,10 @@ export function PortfolioMonitorPanel({
activeLookback: string;
onStrategyChange: (v: string) => void;
onLookbackChange: (v: string) => void;
/** The window the recommendation below was computed on. */
basisLookback?: string | null;
basisLookbackLabel?: string | null;
productionStrategy?: string | null;
}) {
if (!monitor || !monitorRun) {
return (
@@ -70,7 +77,10 @@ export function PortfolioMonitorPanel({
onChange={onStrategyChange}
options={monitor.strategies.map((s) => ({
value: s.strategy,
label: `${s.is_production ? 'Production: ' : ''}${s.label}`,
// "Production: " prefix dropped — a bullet costs one character
// instead of twelve, and the full config is spelled out under
// the chart anyway.
label: `${s.is_production ? '● ' : ''}${s.label}`,
}))}
/>
</div>
@@ -87,6 +97,21 @@ export function PortfolioMonitorPanel({
</div>
</div>
{/* The recommendation below is baked into the report and cannot follow a
dropdown. On load the two agree by construction; say so plainly the
moment a selection moves off that basis. */}
{((basisLookback && activeLookback !== basisLookback) ||
(productionStrategy && activeStrategy !== productionStrategy)) && (
<p className="text-[11px] text-amber-300/80">
Showing{' '}
{productionStrategy && activeStrategy !== productionStrategy
? 'a comparison strategy'
: 'a different window'}
. The recommendation below is computed on the production strategy over{' '}
{basisLookbackLabel ?? basisLookback} these tiles will not match it.
</p>
)}
{/* Tier 1 — what the book returned. */}
<div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-5">
<StatTile
@@ -97,7 +122,12 @@ export function PortfolioMonitorPanel({
/>
<StatTile label="CAGR" value={fmtSignedPct(monitorRun.cagr_pct)} valueClass={rColor(monitorRun.cagr_pct)} />
<StatTile label="Max Drawdown" value={fmtDrawdown(monitorRun.max_drawdown_pct)} valueClass="text-amber-400" />
<StatTile label="Sharpe" value={fmtRatio(monitorRun.sharpe)} />
<StatTile
label="EV / trade"
value={fmtSignedMoney(monitorRun.avg_trade_pnl)}
valueClass={rColor(monitorRun.avg_trade_pnl)}
title="Average realized P&L per closed trade. Scales with position size, so it carries no quality band."
/>
<StatTile label="Trades" value={String(monitorRun.trades)} sub={`${fmtPct(monitorRun.win_rate)} win rate`} />
</div>
@@ -109,39 +139,49 @@ export function PortfolioMonitorPanel({
</p>
) : (
<div className="space-y-2">
<div className="flex flex-wrap items-baseline justify-between gap-2">
<p className="section-index">Risk-adjusted quality</p>
<p className="text-[11px] text-gray-600">
Bands are set stricter than textbook ranges this universe is today's
survivors replayed backward, which flatters every ratio.
</p>
</div>
<div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-5">
<StatTile
size="sm"
label="Sharpe"
value={fmtRatio(monitorRun.sharpe)}
metric="sharpe"
raw={monitorRun.sharpe}
title="Return per unit of total volatility (annualized). Penalizes upside swings as well as downside."
/>
<StatTile
label="Sortino"
value={fmtRatio(monitorRun.sortino)}
title="Return per unit of downside deviation (annualized)."
metric="sortino"
raw={monitorRun.sortino}
title="Return per unit of downside deviation (annualized). Punishes losing days only, unlike Sharpe."
/>
<StatTile
size="sm"
label="Calmar (MAR)"
value={fmtRatio(monitorRun.calmar)}
title="CAGR divided by maximum drawdown."
metric="calmar"
raw={monitorRun.calmar}
title="CAGR divided by maximum drawdown — return earned per unit of worst-case pain."
/>
<StatTile
size="sm"
label="Gain / Pain"
value={fmtRatio(monitorRun.gain_to_pain)}
title="Sum of monthly returns divided by the absolute sum of the negative ones."
metric="gain_to_pain"
raw={monitorRun.gain_to_pain}
title="Sum of monthly returns divided by the absolute sum of the negative ones (Schwager)."
/>
<StatTile
size="sm"
label="Profit Factor ($)"
value={fmtRatio(monitorRun.profit_factor)}
metric="profit_factor"
raw={monitorRun.profit_factor}
title="Gross winning dollars divided by gross losing dollars, across closed trades."
/>
<StatTile
size="sm"
label="EV / trade"
value={fmtSignedMoney(monitorRun.avg_trade_pnl)}
valueClass={rColor(monitorRun.avg_trade_pnl)}
title="Average realized P&L per closed trade."
/>
</div>
</div>
)}
+6 -1
View File
@@ -86,7 +86,12 @@ export function Dropdown({
onClick={() => setOpen((v) => !v)}
className="input-glass flex w-full items-center justify-between gap-2 px-3 py-1.5 text-left text-sm"
>
<span className={selected ? 'text-gray-200' : 'text-gray-500'}>
{/* truncate, not wrap: a long option name used to push the trigger to
three lines and shove the whole control row out of alignment. */}
<span
className={`truncate ${selected ? 'text-gray-200' : 'text-gray-500'}`}
title={selected ? selected.label : undefined}
>
{selected ? selected.label : placeholder}
</span>
<svg
+48 -12
View File
@@ -1,11 +1,21 @@
import {
BAND_STYLE,
bandTicks,
classifyMetric,
meterFraction,
} from '../../lib/metricBands';
/**
* One labelled metric. Lifted from the byte-identical `Stat` that lived in both
* BacktestPanel and MyTradesPanel.
* One labelled metric.
*
* `size` is the hierarchy lever: `md` (default) is the headline look those two
* panels already had; `sm` marks a metric as supporting detail, which is what
* keeps a second row of ratios from reading as equally important as the returns
* above it.
* Optionally carries a quality meter: pass `metric` (a key in METRIC_BANDS) and
* the numeric `raw` value. The meter is the answer to "2.72 — is that good?" —
* a track showing where the value sits, ticks at the band edges, and the band
* word. Colour never travels alone; the word is always rendered beside it.
*
* Every tile is the same size. Hierarchy comes from grouping and section
* labels, not from shrinking one row — two sizes read as inconsistent rather
* than as a deliberate ranking.
*/
export function StatTile({
label,
@@ -13,7 +23,8 @@ export function StatTile({
valueClass = 'text-gray-100',
sub,
title,
size = 'md',
metric,
raw,
}: {
label: string;
value: string;
@@ -21,14 +32,39 @@ export function StatTile({
sub?: string;
/** Native tooltip — how the metric is defined. */
title?: string;
size?: 'md' | 'sm';
/** Key into METRIC_BANDS; enables the quality meter. */
metric?: string;
/** Numeric value the meter reads (the formatted `value` is display-only). */
raw?: number | null;
}) {
const pad = size === 'sm' ? 'p-3' : 'p-4';
const text = size === 'sm' ? 'text-lg' : 'text-2xl';
const band = metric ? classifyMetric(metric, raw) : null;
const style = band ? BAND_STYLE[band] : null;
return (
<div className={`glass ${pad}`} title={title}>
<div className="glass flex flex-col p-4" title={title}>
<p className="section-index">{label}</p>
<p className={`num mt-1.5 ${text} font-semibold ${valueClass}`}>{value}</p>
<p className={`num mt-1.5 text-2xl font-semibold ${valueClass}`}>{value}</p>
{style && metric && (
<div className="mt-2.5">
<div className="relative h-1.5 overflow-hidden rounded-full bg-white/[0.07]">
<div
className={`h-full rounded-full ${style.fill}`}
style={{ width: `${meterFraction(metric, raw) * 100}%` }}
/>
{/* Band edges — where "fair" becomes "good", and so on. */}
{bandTicks(metric).map((t) => (
<span
key={t}
className="absolute top-0 h-full w-px bg-black/50"
style={{ left: `${t * 100}%` }}
/>
))}
</div>
<p className={`mt-1.5 text-[11px] font-medium ${style.text}`}>{style.label}</p>
</div>
)}
{sub && <p className="mt-1 text-xs text-gray-500">{sub}</p>}
</div>
);
+77
View File
@@ -0,0 +1,77 @@
/**
* Quality bands for the risk-adjusted metrics.
*
* A tile reading "Sortino 2.72" answers nothing on its own. These bands turn
* each ratio into weak / fair / good / strong so the tile says whether the
* number is any good.
*
* The bands are deliberately STRICTER than the textbook ranges. This backtest
* replays today's ~512 tracked tickers backward, so every name that failed or
* was acquired inside the window is missing and every ratio here is flattered.
* Standard thresholds would print "strong" on numbers survivorship inflated.
* Treat a band as a claim about this book relative to itself, not a claim that
* the live strategy will reproduce it.
*
* Edges are lower-inclusive: a value exactly on an edge takes the higher band.
*/
export type BandName = 'weak' | 'fair' | 'good' | 'strong';
export interface MetricBand {
/** Lower edges for fair / good / strong. Below the first edge is weak. */
edges: [number, number, number];
/** Where the meter track ends. Values above clamp to full. */
max: number;
}
export const METRIC_BANDS: Record<string, MetricBand> = {
sharpe: { edges: [0.8, 1.5, 2.5], max: 3.5 },
sortino: { edges: [1.2, 2.0, 3.0], max: 4.0 },
calmar: { edges: [0.5, 1.0, 2.5], max: 3.5 },
gain_to_pain: { edges: [1.0, 1.5, 2.5], max: 3.5 },
profit_factor: { edges: [1.3, 1.8, 2.5], max: 3.5 },
};
const BAND_ORDER: BandName[] = ['weak', 'fair', 'good', 'strong'];
export function classifyMetric(
key: keyof typeof METRIC_BANDS | string,
value: number | null | undefined,
): BandName | null {
const band = METRIC_BANDS[key];
if (!band || value === null || value === undefined || !Number.isFinite(value)) {
return null;
}
const passed = band.edges.filter((edge) => value >= edge).length;
return BAND_ORDER[passed];
}
/** Fraction of the meter track a value fills, clamped to 0..1. */
export function meterFraction(
key: keyof typeof METRIC_BANDS | string,
value: number | null | undefined,
): number {
const band = METRIC_BANDS[key];
if (!band || value === null || value === undefined || !Number.isFinite(value)) {
return 0;
}
return Math.max(0, Math.min(1, value / band.max));
}
/** Band edges as track fractions, for drawing the tick marks. */
export function bandTicks(key: keyof typeof METRIC_BANDS | string): number[] {
const band = METRIC_BANDS[key];
if (!band) return [];
return band.edges.map((edge) => edge / band.max);
}
/**
* Status colours, not the categorical palette — these encode state, so they are
* reserved and always paired with the band word rather than standing alone.
*/
export const BAND_STYLE: Record<BandName, { fill: string; text: string; label: string }> = {
weak: { fill: 'bg-red-400/70', text: 'text-red-400', label: 'weak' },
fair: { fill: 'bg-amber-400/70', text: 'text-amber-400', label: 'fair' },
good: { fill: 'bg-emerald-400/70', text: 'text-emerald-400', label: 'good' },
strong: { fill: 'bg-emerald-300/80', text: 'text-emerald-300', label: 'strong' },
};
+7
View File
@@ -336,6 +336,13 @@ export interface BacktestCurvePoint {
export interface BacktestRecommendation {
headline: string | null;
items: { topic: string; text: string }[];
/**
* The monitor lookback every production/benchmark figure was read from. The
* page defaults its selector to this so the tiles and the recommendation
* cannot open on different windows. Absent on reports predating the field.
*/
basis_lookback?: string | null;
basis_lookback_label?: string | null;
note?: string;
}
+116 -8
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import json
import math
from datetime import date, timedelta
from types import SimpleNamespace
@@ -1272,32 +1273,63 @@ def test_build_recommendation_reads_the_report():
{"min_momentum_percentile": 60.0, "net_avg_r": 0.05, "total": 300},
{"min_momentum_percentile": 0.0, "net_avg_r": -0.12, "total": 1000},
],
# Legacy policy book. Its numbers are deliberately DIFFERENT from the
# production monitor's below, so sourcing the benchmark line from here
# again would fail the assertion rather than pass unnoticed.
"portfolio_sim": {"policies": [
{"policy": "target", "cagr_pct": 23.7, "total_return_pct": 134.8,
"spy_return_pct": 95.9, "max_drawdown_pct": 20.7},
{"policy": "hold", "cagr_pct": 31.9, "total_return_pct": 203.6,
"spy_return_pct": 95.9, "max_drawdown_pct": 21.2},
]},
"portfolio_monitor": {
"production_strategy": "prod",
"runs": [{
"strategy": "prod", "lookback": "all", "lookback_label": "All history",
"cagr_pct": 40.0, "sharpe": 1.72, "max_drawdown_pct": 17.7,
"total_return_pct": 297.8, "spy_return_pct": 101.9,
}, {
# A second window with DIFFERENT numbers. Without it the "all"
# preference is untested and a lookback mix-up cannot fail.
"strategy": "prod", "lookback": "3y", "lookback_label": "3y",
"cagr_pct": 47.9, "sharpe": 1.96, "max_drawdown_pct": 17.3,
"total_return_pct": 220.9, "spy_return_pct": 71.5,
}],
},
}
rec = bt._build_recommendation(report)
by_topic: dict[str, list[str]] = {}
for item in rec["items"]:
by_topic.setdefault(item["topic"], []).append(item["text"])
assert rec["headline"] is not None and "hold 30" in rec["headline"]
assert any("hold 30 trading days" in t for t in by_topic["exit"])
assert rec["headline"] is not None and "Production baseline" in rec["headline"]
# The hold-vs-target comparison is gone: both are exits the production book
# replaced, so a recommendation between them cannot lead to an action.
assert "exit" not in by_topic
# Benchmark must quote the SAME row the page's tiles show, not the policy sim.
assert "+297.8%" in by_topic["benchmark"][0]
assert "203.6" not in by_topic["benchmark"][0]
gate_texts = " | ".join(by_topic["gate"])
assert "confidence floor adds nothing" in gate_texts
assert "keep the R:R floor" in gate_texts
assert "keep the NEUTRAL exclusion" in gate_texts
assert "80" in by_topic["cutoff"][0]
assert "beats" in by_topic["benchmark"][0]
# robustness is judged under the RECOMMENDED exit (the 30d hold), not the
# target model the recommendation advises abandoning
assert any(
"not a handful of outliers" in t and "under the recommended 30d hold" in t
for t in by_topic["robustness"]
)
# Every production figure comes from ONE window, and the report says which,
# so the page can default its selector to the same one.
assert rec["basis_lookback"] == "all"
assert rec["basis_lookback_label"] == "All history"
assert "+40.0%" in by_topic["production"][0]
assert "47.9" not in by_topic["production"][0] # the 3y row must not leak in
assert "220.9" not in by_topic["benchmark"][0]
# Robustness names its real basis. It used to claim "under the recommended
# 30d hold" — nothing recommends that exit; production is the ATR trail.
robustness = by_topic["robustness"][0]
assert "not a handful of outliers" in robustness
assert "gate-level grading" in robustness
assert "recommended" not in robustness
def test_build_recommendation_flags_outlier_dependence():
@@ -1810,3 +1842,79 @@ class TestPortfolioQualityMetrics:
for key in ("sortino", "gain_to_pain", "profit_factor"):
assert key in sim
assert sim["sortino"] is None
def test_build_recommendation_states_no_baseline_without_a_production_row():
"""A report with no portfolio monitor cannot describe the production book.
It used to fall back to recommending the fixed-hold exit — advice for a model
the production book had already replaced."""
report = {
"overall_qualified": {"net_avg_r": 0.13, "net_avg_r_ex_top5": 0.05},
"time_exit_sweep": [{"hold_days": 30, "net_avg_r": 0.50, "net_avg_r_ex_top5": 0.21}],
"portfolio_sim": {"policies": [
{"policy": "hold", "cagr_pct": 31.9, "total_return_pct": 203.6,
"spy_return_pct": 95.9, "max_drawdown_pct": 21.2},
]},
}
rec = bt._build_recommendation(report)
topics = {item["topic"] for item in rec["items"]}
assert rec["headline"] is None
# Nothing may be sourced from the legacy policy book.
assert "benchmark" not in topics
assert "exit" not in topics
async def test_cached_report_recommendation_is_rebuilt_on_read(session):
"""A report cached by an older build carries that build's recommendation.
Served verbatim, the page would show the legacy wording and no
basis_lookback — which let the lookback selector default elsewhere, putting
3y tiles beside an all-history recommendation with no warning. This is the
shape of the report sitting in production right now.
"""
from app.services.admin_service import update_setting
stale = {
"generated_at": "2026-08-12T05:00:00+00:00",
"tickers": 512, "candidates": 100, "qualified": 10,
"params": {"horizon_days": 30},
"overall_qualified": {"net_avg_r": 0.13, "net_avg_r_ex_top5": 0.20},
"portfolio_sim": {"policies": [
{"policy": "hold", "cagr_pct": 31.9, "total_return_pct": 175.0,
"spy_return_pct": 101.9, "max_drawdown_pct": 23.7},
]},
"portfolio_monitor": {
"production_strategy": "prod",
"runs": [{
"strategy": "prod", "lookback": "all", "lookback_label": "All history",
"cagr_pct": 40.0, "sharpe": 1.72, "max_drawdown_pct": 17.7,
"total_return_pct": 297.8, "spy_return_pct": 101.9,
}],
},
# What the old build stored: sourced from the policy book, and naming an
# exit the production book replaced.
"recommendation": {
"headline": "Trade the qualified list long-only; hold 30 trading days.",
"items": [
{"topic": "benchmark", "text": "Book vs SPY: beats buy-and-hold by "
"+73.1 points (+175.0% vs +101.9%)."},
{"topic": "robustness", "text": "Robustness: expectancy survives removing "
"the top 5% of winners (+0.20R net/trade "
"under the recommended 30d hold)."},
],
"note": "stale",
},
}
await update_setting(session, bt.KEY_REPORT, json.dumps(stale))
report = await bt.get_backtest_report(session)
assert report is not None
rec = report["recommendation"]
# Rebuilt: the basis is published, so the page cannot default elsewhere.
assert rec["basis_lookback"] == "all"
texts = " | ".join(i["text"] for i in rec["items"])
# ...and it quotes the production book, not the policy sim it used to.
assert "+297.8%" in texts and "175.0" not in texts
assert "recommended 30d hold" not in texts
assert "Production baseline" in (rec["headline"] or "")