diff --git a/README.md b/README.md index 93eb298..3bce2c0 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ Once a day (default 07:00). Steps run **in dependency order**, each consuming th 1. **OHLCV** — fetch the latest daily bars for every tracked ticker (Alpaca); new tickers backfill ~5 years. 2. **Sentiment** — fetch sentiment for the names that matter and are stale (> 5 days): top-pick feeders (residual-momentum leaders with a tradeable long setup), the watchlist, and open paper trades, plus a top-N-by-composite discovery net. Runs *before* the scan so the scan sees fresh sentiment. -3. **R:R Scan** — recompute S/R zones, the 5-dimension scores and long/short setups (ATR stops, S/R targets) for every ticker, and attach each ticker's residual 12‑1 momentum activation percentile. -4. **Outcome Eval** — resolve setups that hit target/stop or expired (default 30 trading days) and auto-close paper trades per the exit policy (default: hold 30 trading days with the initial stop — the backtest-validated exit). +3. **R:R Scan** — recompute S/R zones, the 5-dimension scores and long/short setups (ATR stops, S/R targets) for every ticker, and attach each ticker's residual 12‑1 momentum activation percentile plus the promoted 80/20 production rank. +4. **Outcome Eval** — resolve setups that hit target/stop or expired (default 30 trading days) and auto-close paper trades per the exit policy (default: 3x ATR trail with a 30-trading-day max hold). 5. **Market Regime** — recompute the regime index (breadth/trend). 6. **Regime Monitor** — observational early-warning snapshot (VIX, credit spreads via FRED); feeds nothing else. @@ -34,7 +34,7 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we 1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize). 2. **Setups** — the scanner builds long/short setups with ATR stops and S/R targets, then adds a confidence score, conflict flags and a target reach-probability. 3. **Activation gate** — a setup *qualifies* only if it clears the R:R floor **and** ranks in the top residual-momentum percentile of the universe (the validated edge is long-only; the confidence floor was ablated to zero effect and defaults off). -4. **Top pick** — the highest residual-momentum qualified setup; highlighted on the Dashboard and labelled on the ticker page. +4. **Top pick** — qualified setups are ordered by the production rank: 80% residual momentum percentile + 20% 6-month realized-volatility percentile. The #1 is highlighted on the Dashboard and labelled on the ticker page. ## Strategy Status — What's Validated and What Isn't @@ -42,7 +42,7 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we | Component | Verdict | Evidence | |---|---|---| -| **Residual 12-1 cross-sectional momentum** (the activation gate, long-only) | **Production ranking — in-sample edge** | Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable | +| **Residual 12-1 cross-sectional momentum** (the activation gate, long-only) | **Production gate — in-sample edge** | Promoted July 2026 after the portfolio variant beat raw 80 on CAGR, Sharpe and drawdown. Raw 12-1 remains a fallback only when benchmark data is unavailable | | S/R setup engine (ATR stops, S/R targets, reach-probability) | **Filter/execution context, not the exit** | R:R/room-to-run still earns its keep as a filter, but S/R targets underperform the time exit. The probability model is display-only | | Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) | | LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet | @@ -54,23 +54,34 @@ Caveats on the momentum result: in-sample, roughly one market regime, costs/slip ### Current production baseline -Use this as a regression guardrail for future strategy changes, not as a return promise. Backtest run: 506 tickers, weekly cadence, 30-trading-day horizon, 2022-06-28 → 2026-07-01, 0.1% per-side costs, price-only SPY benchmark. +Use this as a regression guardrail for future strategy changes, not as a return promise. Backtest run: local production SQLite snapshot, 506 tickers, weekly cadence, 30-trading-day horizon, 2022-06-28 → 2026-07-01, 0.1% per-side costs, price-only SPY benchmark. | Item | Current baseline | |---|---| -| Strategy version | `residual_momentum_12_1_rr_time_v2` | +| Strategy version | `residual_highvol_80_20_atr_trail3_v1` | | Production gate | Long-only, residual 12-1 momentum percentile >= 80, R:R floor on, NEUTRAL excluded, confidence floor effectively off | -| Exit | Hold 30 trading days with the initial ATR stop | -| Qualified setups | 1,810 | -| Qualified net expectancy | +0.16R per setup | -| Profit factor | 1.27 | -| Portfolio CAGR | +40.4% | -| Portfolio total return | +289.4% vs SPY +95.9% | -| Max drawdown | -26.1% | -| Sharpe | 1.52 daily, annualized | -| Robustness | 30d hold remains +0.16R net/trade after removing the top 5% winners | +| Production rank | 80% residual momentum percentile + 20% 6-month realized-volatility percentile | +| Exit | Initial ATR stop plus 3x ATR trailing stop, max 30 trading days | +| Portfolio CAGR | +44.4% | +| Portfolio total return | +336.6% vs SPY +95.7% | +| Max drawdown | -23.8% | +| Sharpe | 1.72 daily, annualized | +| Trades | 376 | +| Average hold | 14.7 trading days | -Nearest challengers from the same run: legacy raw 80 was weaker (+33.8% CAGR, -28.8% max drawdown, Sharpe 1.32); raw 90 was close but had lower Sharpe and worse drawdown (+40.4% CAGR, -27.6% max drawdown, Sharpe 1.49); residual 80 / max 15 removed book-full skips but did not improve CAGR, drawdown, Sharpe or closed trades. +Promotion evidence from the same snapshot: + +| Candidate | CAGR | Max DD | Sharpe | Trades | Read | +|---|---:|---:|---:|---:|---| +| Legacy residual 80 + 30d hold | +34.8% | -24.4% | 1.51 | 339 | Previous production baseline | +| Residual/high-vol 80/20 + 30d hold | +39.2% | -23.9% | 1.55 | 345 | Better entry rank, slightly lower drawdown | +| Residual/high-vol 80/20 + 3x ATR trail | +44.4% | -23.8% | 1.72 | 376 | Promoted: better CAGR, Sharpe, and drawdown | +| Pure high-vol 80 + 30d hold | +37.7% | -37.6% | 1.22 | 491 | Rejected: standalone volatility was too volatile | +| Low-vol 80 + 30d hold | +0.4% | -23.1% | 0.09 | 257 | Rejected: no useful edge | + +The conclusion is not "trade high volatility alone." Keep residual momentum as the entry gate, use realized volatility only as a small ranking tilt, and add the ATR trail as defensive exit discipline. + +Live-ranking note: the backtest ranks residual momentum and volatility inside each weekly setup-candidate cross-section. The live scanner computes the same 80/20 formula across the current ticker universe before scanning so every generated setup carries a stable ticker-level rank. That is the production approximation; reconcile it later only if candidate-only post-scan ranking proves materially different. ### The iron rule for strategy changes @@ -84,15 +95,16 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m ### Highest-value next experiments (in order) -1. **Raw 90 challenger** — keep comparing raw 12-1 momentum at cutoff 90 against production residual 80; promote only if it beats residual production on Sharpe and drawdown without a meaningful CAGR hit. -2. **Capacity check** — keep only the residual 80 / max 15 portfolio row as a guardrail; max 20 and raw max 15 added no information in the July 2026 run. -3. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only. -4. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. (Deeper history was considered and declined.) +1. **Forward monitor the promoted strategy** — the production UI now behaves like a portfolio monitor for the current strategy, with selectable lookbacks and SPY comparison. +2. **Trailing-stop sensitivity** — locally compare 2.5x, 3x, and 3.5x ATR trails before changing the promoted 3x default. +3. **Capacity check** — retest residual/high-vol 80/20 with a max-15 weekly book cap; promote only if it improves drawdown or trade quality without costing too much CAGR. +4. **Signal context snapshots** — accumulate point-in-time composite/sentiment/fundamental context for every new setup so the discretionary overlay can be tested forward-only. +5. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. (Deeper history was considered and declined.) ## Key Use Cases -- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists qualified setups ranked by residual momentum with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop. -- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed by the active exit policy (default: 30 trading days with the initial stop), and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge. +- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists residual-gated qualified setups ranked by the production 80/20 residual/high-vol score, with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop. +- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed by the active exit policy (default: 3x ATR trail with a 30-trading-day max hold), and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge. ## Stack @@ -123,7 +135,7 @@ Corollaries: never let an unvalidated score gate setups; the outcome evaluator m - Risk:Reward scanner — long and short setups, ATR-based stops, S/R-based targets, configurable R:R threshold (default 1.5:1) - Activation gate — qualifies setups on a residual-momentum percentile floor plus an R:R floor (validated long-only edge) - Recommendation layer — directional confidence, conflict detection, per-target reach-probability -- Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: hold 30 trading days with the initial stop; trailing / target-stop selectable), realized track record + outcome evaluation +- Paper trading — take a setup, mark-to-market vs. latest close, auto-close per the exit policy (default: 3x ATR trail with a 30-trading-day max hold; time / percent-trailing / target-stop selectable), realized track record + outcome evaluation - Market-regime index + FRED early-warning monitor (VIX, credit spreads); weekly backtest + manual event study - Telegram alerts (e.g. regime-quadrant changes) - User-curated watchlist (cap: 20), enriched with composite score, R:R and S/R summary @@ -280,12 +292,16 @@ python scripts/create_backtest_snapshot.py \ ```bash # macOS/Linux -python scripts/run_backtest_snapshot.py backtest_snapshots/prod.sqlite --workers 8 +python scripts/run_backtest_snapshot.py backtest_snapshots/prod.sqlite --workers 6 # Windows PowerShell -.venv\Scripts\python.exe scripts\run_backtest_snapshot.py backtest_snapshots\prod.sqlite --workers 12 --allow-spawn +.venv\Scripts\python.exe scripts\run_backtest_snapshot.py backtest_snapshots\prod.sqlite --workers 6 --allow-spawn ``` +On an 8-thread machine, `--workers 6` is a good starting point: it leaves a +couple of threads for Windows, the shell, and browser/UI work while still using +most of the CPU. + The runner writes `reports/backtest-.json` and prints the headline metrics. Keep the SSH tunnel open only while creating the snapshot; the backtest run itself is local/offline. `backtest_snapshots/` and generated backtest reports diff --git a/alembic/versions/017_add_trade_setup_strategy_rank.py b/alembic/versions/017_add_trade_setup_strategy_rank.py new file mode 100644 index 0000000..788341f --- /dev/null +++ b/alembic/versions/017_add_trade_setup_strategy_rank.py @@ -0,0 +1,27 @@ +"""Add production strategy rank fields to trade setups. + +Revision ID: 017_add_trade_setup_strategy_rank +Revises: 016_add_signal_context_snapshots +Create Date: 2026-07-03 20:15:00.000000 +""" + +from __future__ import annotations + +from alembic import op +import sqlalchemy as sa + + +revision = "017_add_trade_setup_strategy_rank" +down_revision = "016_add_signal_context_snapshots" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column("trade_setups", sa.Column("strategy_rank", sa.Float(), nullable=True)) + op.add_column("trade_setups", sa.Column("volatility_percentile", sa.Float(), nullable=True)) + + +def downgrade() -> None: + op.drop_column("trade_setups", "volatility_percentile") + op.drop_column("trade_setups", "strategy_rank") diff --git a/app/models/paper_trade.py b/app/models/paper_trade.py index 5700ec0..a533c5b 100644 --- a/app/models/paper_trade.py +++ b/app/models/paper_trade.py @@ -34,5 +34,5 @@ class PaperTrade(Base): ) close_price: Mapped[float | None] = mapped_column(Float, nullable=True) closed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) - # How the trade was closed: "trailing" | "stop" | "target" | "manual". + # How the trade was closed: "time" | "trailing" | "stop" | "target" | "manual". close_reason: Mapped[str | None] = mapped_column(String(10), nullable=True) diff --git a/app/models/trade_setup.py b/app/models/trade_setup.py index a7cebb9..f389b4a 100644 --- a/app/models/trade_setup.py +++ b/app/models/trade_setup.py @@ -30,6 +30,10 @@ class TradeSetup(Base): # time. Since July 2026 this is residual 12-1 momentum when benchmark data is # available, with raw 12-1 as a fallback. momentum_percentile: Mapped[float | None] = mapped_column(Float, nullable=True) + # Production ordering score. July 2026 promotion: residual momentum remains + # the gate, while this rank blends residual momentum with realized volatility. + strategy_rank: Mapped[float | None] = mapped_column(Float, nullable=True) + volatility_percentile: Mapped[float | None] = mapped_column(Float, nullable=True) targets_json: Mapped[str | None] = mapped_column(Text, nullable=True) conflict_flags_json: Mapped[str | None] = mapped_column(Text, nullable=True) recommended_action: Mapped[str | None] = mapped_column(String(20), nullable=True) diff --git a/app/routers/paper_trades.py b/app/routers/paper_trades.py index c84ec30..c13e554 100644 --- a/app/routers/paper_trades.py +++ b/app/routers/paper_trades.py @@ -62,7 +62,11 @@ async def write_exit_policy( ) -> APIEnvelope: """Change the auto-exit policy (admin).""" data = await paper_trade_service.set_exit_policy( - db, mode=body.mode, trailing_pct=body.trailing_pct, hold_days=body.hold_days + db, + mode=body.mode, + trailing_pct=body.trailing_pct, + atr_multiplier=body.atr_multiplier, + hold_days=body.hold_days, ) return APIEnvelope(status="success", data=data) diff --git a/app/schemas/paper_trade.py b/app/schemas/paper_trade.py index 02ba810..31e0bfe 100644 --- a/app/schemas/paper_trade.py +++ b/app/schemas/paper_trade.py @@ -22,8 +22,9 @@ class PaperTradeClose(BaseModel): class ExitPolicyUpdate(BaseModel): """Auto-exit policy for open paper trades.""" - mode: str | None = Field(default=None, pattern=r"^(time|trailing|target)$") + mode: str | None = Field(default=None, pattern=r"^(time|trailing|atr_trailing|target)$") trailing_pct: float | None = Field(default=None, ge=0.5, le=90) + atr_multiplier: float | None = Field(default=None, ge=0.5, le=10) hold_days: int | None = Field(default=None, ge=2, le=250) diff --git a/app/schemas/trade_setup.py b/app/schemas/trade_setup.py index 604efd1..975c5c8 100644 --- a/app/schemas/trade_setup.py +++ b/app/schemas/trade_setup.py @@ -57,5 +57,7 @@ class TradeSetupResponse(BaseModel): evaluated_at: datetime | None = None current_price: float | None = None momentum_percentile: float | None = None + strategy_rank: float | None = None + volatility_percentile: float | None = None context_as_of: TradeSetupContextAsOfResponse | None = None recommendation_summary: RecommendationSummaryResponse | None = None diff --git a/app/services/backtest_service.py b/app/services/backtest_service.py index 37f1d1f..fa0121b 100644 --- a/app/services/backtest_service.py +++ b/app/services/backtest_service.py @@ -44,6 +44,7 @@ from app.models.ticker import Ticker from app.services import settings_store from app.services.admin_service import get_activation_config, update_setting from app.services.indicator_service import _extract_ohlcv, compute_atr +from app.services.momentum_service import compute_realized_vol_6m from app.services.outcome_service import ( OUTCOME_AMBIGUOUS, OUTCOME_STOP_HIT, @@ -97,6 +98,13 @@ MIN_RELIABLE_PERIODS = 12 # min non-overlapping windows before a signal's IC is PRODUCTION_PERCENTILE_KEY = "activation_momentum_percentile" RAW_PERCENTILE_KEY = "momentum_percentile" RESIDUAL_PERCENTILE_KEY = "residual_momentum_percentile" +VOL_PERCENTILE_KEY = "vol_6m_percentile" +LOW_VOL_PERCENTILE_KEY = "low_vol_6m_percentile" +RESIDUAL_LOW_VOL_BLEND_KEY = "residual_low_vol_blend_score" +RESIDUAL_HIGH_VOL_BLEND_90_10_KEY = "residual_high_vol_blend_90_10_score" +RESIDUAL_HIGH_VOL_BLEND_80_20_KEY = "residual_high_vol_blend_80_20_score" +RESIDUAL_HIGH_VOL_BLEND_KEY = "residual_high_vol_blend_score" +RESIDUAL_HIGH_VOL_BLEND_60_40_KEY = "residual_high_vol_blend_60_40_score" def _wrap_levels(level_dicts: list[dict]) -> list[Any]: @@ -322,6 +330,7 @@ def _replay_ticker( residual_momentum = _residual_momentum_12_1( dates, closes, len(window) - 1, benchmark_closes ) + vol_6m = _realized_vol_6m(closes, len(window) - 1) for s in _window_setups(window, config, activation): outcome, outcome_date = evaluate_setup_against_bars( @@ -365,6 +374,7 @@ def _replay_ticker( "best_prob": s["best_prob"], "momentum": s["momentum"], "residual_momentum": residual_momentum, + "vol_6m": vol_6m, "meets_core": s["meets_core"], # Gate fields the ablation recomputes floors from — without them # every candidate looks NEUTRAL and the ablation rows collapse. @@ -551,6 +561,11 @@ def _residual_momentum_12_1( return sum(stock_rets[k] - beta * market_rets[k] for k in range(len(stock_rets))) +def _realized_vol_6m(closes: list[float], i: int) -> float | None: + """126-trading-day realized daily volatility at point-in-time index ``i``.""" + return compute_realized_vol_6m(closes[: i + 1]) + + def _signal_values( dates: list[date], closes: list[float], @@ -588,16 +603,9 @@ def _signal_values( high_52w = max(highs[i - 251 : i + 1]) if high_52w > 0: out["high_52w"] = closes[i] / high_52w - if i - 126 >= 0: - rets = [ - closes[k] / closes[k - 1] - 1.0 - for k in range(i - 125, i + 1) - if closes[k - 1] > 0 - ] - if len(rets) >= 2: - mean = sum(rets) / len(rets) - var = sum((x - mean) ** 2 for x in rets) / (len(rets) - 1) - out["vol_6m"] = math.sqrt(var) + vol_6m = _realized_vol_6m(closes, i) + if vol_6m is not None: + out["vol_6m"] = vol_6m return out @@ -878,6 +886,14 @@ def _assign_residual_momentum_percentiles(candidates: list[dict]) -> None: ) +def _assign_low_volatility_percentiles(candidates: list[dict]) -> None: + """Per ISO week, attach volatility ranks where 100 = lowest 6-month vol.""" + _assign_signal_percentiles(candidates, "vol_6m", VOL_PERCENTILE_KEY) + for c in candidates: + raw = c.get(VOL_PERCENTILE_KEY) + c[LOW_VOL_PERCENTILE_KEY] = (100.0 - raw) if raw is not None else None + + def _assign_activation_momentum_percentiles(candidates: list[dict]) -> None: """Production activation rank: residual 12-1 when available, raw fallback. @@ -892,6 +908,52 @@ def _assign_activation_momentum_percentiles(candidates: list[dict]) -> None: ) +def _assign_weighted_blend( + candidates: list[dict], + output_key: str, + primary_key: str, + primary_weight: float, + secondary_key: str, +) -> None: + secondary_weight = 1.0 - primary_weight + for c in candidates: + primary = c.get(primary_key) + secondary = c.get(secondary_key) + c[output_key] = ( + primary * primary_weight + secondary * secondary_weight + if primary is not None and secondary is not None + else None + ) + + +def _assign_residual_low_vol_blend(candidates: list[dict]) -> None: + """Research rank: mostly residual momentum, with lower-vol names preferred.""" + _assign_weighted_blend( + candidates, + RESIDUAL_LOW_VOL_BLEND_KEY, + PRODUCTION_PERCENTILE_KEY, + 0.7, + LOW_VOL_PERCENTILE_KEY, + ) + + +def _assign_residual_high_vol_blend(candidates: list[dict]) -> None: + """Research ranks: residual momentum blended with higher-vol preference.""" + for output_key, residual_weight in ( + (RESIDUAL_HIGH_VOL_BLEND_90_10_KEY, 0.9), + (RESIDUAL_HIGH_VOL_BLEND_80_20_KEY, 0.8), + (RESIDUAL_HIGH_VOL_BLEND_KEY, 0.7), + (RESIDUAL_HIGH_VOL_BLEND_60_40_KEY, 0.6), + ): + _assign_weighted_blend( + candidates, + output_key, + PRODUCTION_PERCENTILE_KEY, + residual_weight, + VOL_PERCENTILE_KEY, + ) + + def _momentum_qualifies(cand: dict, threshold: float) -> bool: """Whether a candidate clears the floors (meets_core) and the momentum gate. Threshold 0 disables the momentum gate (floors only). The gate is long-only: @@ -1012,6 +1074,8 @@ def _simulate_portfolio( ranking_key: str = PRODUCTION_PERCENTILE_KEY, max_positions: int = SIM_MAX_POSITIONS, risk_per_trade: float = SIM_RISK_PER_TRADE, + start_date: date | None = None, + include_curve: bool = False, ) -> dict | None: """Replay the qualified setups as ONE capital-constrained book and report portfolio economics from the daily equity curve (return, CAGR, drawdown, @@ -1020,9 +1084,10 @@ def _simulate_portfolio( ``exit_policy``: "target" races the S/R target against the stop with a timeout at ``hold_days``; "hold" keeps only the initial stop and exits at - the ``hold_days``-th close. Stops fill at the worse of stop or open (gaps - modeled); positions still open at the end are closed at their last mark. - Returns None when there is nothing to trade. + the ``hold_days``-th close. Research exits add price-derived early exits: + "sma50", "low20", "technical40", and "atr_trail3". Stops fill at the + worse of stop or open (gaps modeled); positions still open at the end are + closed at their last mark. Returns None when there is nothing to trade. """ if qualified_fn is None: def _default_qualified(c: dict) -> bool: @@ -1031,12 +1096,16 @@ def _simulate_portfolio( qualified_fn = _default_qualified entries_by_ord: dict[int, list[dict]] = defaultdict(list) + start_ord = start_date.toordinal() if start_date is not None else None for c in candidates: if not qualified_fn(c) or c.get("direction") != "long": continue + entry_ord = date.fromisoformat(c["date"]).toordinal() + if start_ord is not None and entry_ord < start_ord: + continue if not c.get("entry") or not c.get("stop"): continue - entries_by_ord[date.fromisoformat(c["date"]).toordinal()].append(c) + entries_by_ord[entry_ord].append(c) if not entries_by_ord: return None @@ -1045,7 +1114,7 @@ def _simulate_portfolio( sym: {o: i for i, o in enumerate(cols[0])} for sym, cols in prices.items() } - first_ord = min(entries_by_ord) + first_ord = start_ord if start_ord is not None else min(entries_by_ord) calendar = sorted({o for cols in prices.values() for o in cols[0] if o >= first_ord}) if not calendar: return None @@ -1055,6 +1124,8 @@ def _simulate_portfolio( curve: list[tuple[int, float]] = [] trades: list[dict] = [] skipped_full = 0 + technical_cache: dict[tuple[str, int], float | None] = {} + atr_cache: dict[tuple[str, int], float | None] = {} def _bar(sym: str, o: int): idx = index_of.get(sym, {}).get(o) @@ -1062,16 +1133,68 @@ def _simulate_portfolio( return None cols = prices[sym] return SimpleNamespace( - open=cols[1][idx], high=cols[2][idx], low=cols[3][idx], close=cols[4][idx] + idx=idx, + open=cols[1][idx], + high=cols[2][idx], + low=cols[3][idx], + close=cols[4][idx], ) + def _sma(sym: str, idx: int, lookback: int) -> float | None: + if idx + 1 < lookback: + return None + closes = prices[sym][4] + return sum(closes[idx - lookback + 1 : idx + 1]) / lookback + + def _prior_low(sym: str, idx: int, lookback: int) -> float | None: + if idx < lookback: + return None + lows = prices[sym][3] + return min(lows[idx - lookback : idx]) + + def _technical_score(sym: str, idx: int) -> float | None: + key = (sym, idx) + if key in technical_cache: + return technical_cache[key] + if idx + 1 < MIN_LOOKBACK: + technical_cache[key] = None + return None + cols = prices[sym] + try: + score = compute_technical_from_arrays( + cols[2][: idx + 1], + cols[3][: idx + 1], + cols[4][: idx + 1], + cols[5][: idx + 1], + )[0] + technical_cache[key] = float(score) if score is not None else None + except Exception: + technical_cache[key] = None + return technical_cache[key] + + def _atr(sym: str, idx: int) -> float | None: + key = (sym, idx) + if key in atr_cache: + return atr_cache[key] + cols = prices[sym] + try: + value = compute_atr( + cols[2][: idx + 1], + cols[3][: idx + 1], + cols[4][: idx + 1], + )["atr"] + atr_cache[key] = float(value) if value and value > 0 else None + except Exception: + atr_cache[key] = None + return atr_cache[key] + def _close_trade(sym: str, fill: float, reason: str) -> None: nonlocal cash pos = positions.pop(sym) proceeds = pos["shares"] * fill cost = proceeds * COST_PER_SIDE cash += proceeds - cost - risk = pos["entry"] - pos["stop"] + risk = pos["entry"] - pos["initial_stop"] trades.append({ "pnl": proceeds - pos["shares"] * pos["entry"] - cost - pos["entry_cost"], "r": (fill - pos["entry"]) / risk if risk > 0 else 0.0, @@ -1094,13 +1217,41 @@ def _simulate_portfolio( if bar.low <= pos["stop"]: # Same-bar stop+target resolves as the loss (conservative, like # the evaluator); gap through the stop fills at the open. - _close_trade(sym, min(pos["stop"], bar.open), "stop") + reason = ( + "trailing_stop" + if pos["stop"] > pos["initial_stop"] + 1e-9 + else "stop" + ) + _close_trade(sym, min(pos["stop"], bar.open), reason) continue if exit_policy == "target" and pos["target"] and bar.high >= pos["target"]: _close_trade(sym, pos["target"], "target") continue + if exit_policy == "sma50": + sma = _sma(sym, bar.idx, 50) + if sma is not None and bar.close < sma: + _close_trade(sym, bar.close, "sma50") + continue + elif exit_policy == "low20": + prior_low = _prior_low(sym, bar.idx, 20) + if prior_low is not None and bar.close < prior_low: + _close_trade(sym, bar.close, "low20") + continue + elif exit_policy == "technical40": + technical = _technical_score(sym, bar.idx) + if technical is not None and technical < 40.0: + _close_trade(sym, bar.close, "technical40") + continue if pos["bars_held"] >= hold_days: _close_trade(sym, bar.close, "time") + continue + if exit_policy == "atr_trail3": + pos["highest_close"] = max(pos["highest_close"], bar.close) + atr = _atr(sym, bar.idx) + if atr is not None: + next_stop = pos["highest_close"] - 3.0 * atr + if next_stop < bar.close: + pos["stop"] = max(pos["stop"], next_stop) # 2) entries at today's close, best momentum first equity = _marked_equity() @@ -1132,11 +1283,13 @@ def _simulate_portfolio( positions[sym] = { "shares": shares, "entry": entry, + "initial_stop": stop, "stop": stop, "target": float(c["target"]) if c.get("target") else None, "entry_cost": entry_cost, "bars_held": 0, "last_close": entry, + "highest_close": entry, } equity = _marked_equity() @@ -1198,6 +1351,10 @@ def _simulate_portfolio( pnls = [t["pnl"] for t in trades] wins = sum(1 for p in pnls if p > 0) + reason_counts = { + reason: sum(1 for t in trades if t["reason"] == reason) + for reason in sorted({t["reason"] for t in trades}) + } spy_pct = None if spy_closes: from app.services.benchmark_service import benchmark_return_pct @@ -1206,7 +1363,37 @@ def _simulate_portfolio( spy_closes, date.fromordinal(calendar[0]), date.fromordinal(calendar[-1]) ) - return { + curve_payload: list[dict] | None = None + benchmark_payload: list[dict] | None = None + if include_curve: + curve_base = curve[0][1] if curve else SIM_STARTING_CAPITAL + curve_payload = [ + { + "date": date.fromordinal(o).isoformat(), + "equity": round(eq, 2), + "return_pct": round((eq / curve_base - 1.0) * 100.0, 2) + if curve_base > 0 + else None, + } + for o, eq in curve + ] + if spy_closes: + benchmark_payload = [] + base_spy = None + for o, _ in curve: + d = date.fromordinal(o) + close = spy_closes.get(d) + if close is None or close <= 0: + continue + if base_spy is None: + base_spy = close + benchmark_payload.append({ + "date": d.isoformat(), + "equity": round(SIM_STARTING_CAPITAL * close / base_spy, 2), + "return_pct": round((close / base_spy - 1.0) * 100.0, 2), + }) + + result = { "starting_capital": SIM_STARTING_CAPITAL, "final_equity": round(final_equity, 2), "total_return_pct": round(total_return_pct, 1), @@ -1223,12 +1410,18 @@ def _simulate_portfolio( "avg_hold_days": ( round(sum(t["hold"] for t in trades) / len(trades), 1) if trades else None ), + "exit_reasons": reason_counts, "skipped_book_full": skipped_full, "spy_return_pct": round(spy_pct, 1) if spy_pct is not None else None, "yearly_returns": yearly, "start_date": date.fromordinal(calendar[0]).isoformat(), "end_date": date.fromordinal(calendar[-1]).isoformat(), } + if curve_payload is not None: + result["equity_curve"] = curve_payload + if benchmark_payload is not None: + result["benchmark_curve"] = benchmark_payload + return result STRATEGY_VARIANTS: tuple[dict, ...] = ( @@ -1268,6 +1461,162 @@ STRATEGY_VARIANTS: tuple[dict, ...] = ( "risk_per_trade": 0.01, "risk_scale": None, }, + { + "variant": "residual80_lowvol50_fixed10", + "label": "Residual 80 + low-vol 50 / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ( + (PRODUCTION_PERCENTILE_KEY, 80.0), + (LOW_VOL_PERCENTILE_KEY, 50.0), + ), + "ranking_key": PRODUCTION_PERCENTILE_KEY, + "ranking": "residual+low_vol_filter", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_lowvol70_fixed10", + "label": "Residual 80 + low-vol 70 / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ( + (PRODUCTION_PERCENTILE_KEY, 80.0), + (LOW_VOL_PERCENTILE_KEY, 70.0), + ), + "ranking_key": PRODUCTION_PERCENTILE_KEY, + "ranking": "residual+low_vol_filter", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_lowvol_blend_fixed10", + "label": "Residual 80 + low-vol blend / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ((PRODUCTION_PERCENTILE_KEY, 80.0),), + "ranking_key": RESIDUAL_LOW_VOL_BLEND_KEY, + "ranking": "residual_low_vol_blend", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol50_fixed10", + "label": "Residual 80 + high-vol 50 / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ( + (PRODUCTION_PERCENTILE_KEY, 80.0), + (VOL_PERCENTILE_KEY, 50.0), + ), + "ranking_key": PRODUCTION_PERCENTILE_KEY, + "ranking": "residual+high_vol_filter", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol70_fixed10", + "label": "Residual 80 + high-vol 70 / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ( + (PRODUCTION_PERCENTILE_KEY, 80.0), + (VOL_PERCENTILE_KEY, 70.0), + ), + "ranking_key": PRODUCTION_PERCENTILE_KEY, + "ranking": "residual+high_vol_filter", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol_blend90_10_fixed10", + "label": "Residual 80 + high-vol 90/10 blend / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ((PRODUCTION_PERCENTILE_KEY, 80.0),), + "ranking_key": RESIDUAL_HIGH_VOL_BLEND_90_10_KEY, + "ranking": "residual_high_vol_blend_90_10", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol_blend80_20_fixed10", + "label": "Residual 80 + high-vol 80/20 blend / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ((PRODUCTION_PERCENTILE_KEY, 80.0),), + "ranking_key": RESIDUAL_HIGH_VOL_BLEND_80_20_KEY, + "ranking": "residual_high_vol_blend_80_20", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol_blend_fixed10", + "label": "Residual 80 + high-vol 70/30 blend / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ((PRODUCTION_PERCENTILE_KEY, 80.0),), + "ranking_key": RESIDUAL_HIGH_VOL_BLEND_KEY, + "ranking": "residual_high_vol_blend_70_30", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "residual80_highvol_blend60_40_fixed10", + "label": "Residual 80 + high-vol 60/40 blend / max 10", + "percentile_key": PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ((PRODUCTION_PERCENTILE_KEY, 80.0),), + "ranking_key": RESIDUAL_HIGH_VOL_BLEND_60_40_KEY, + "ranking": "residual_high_vol_blend_60_40", + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "highvol70_fixed10", + "label": "High-vol 70 / max 10", + "percentile_key": VOL_PERCENTILE_KEY, + "cutoff": 70.0, + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "highvol80_fixed10", + "label": "High-vol 80 / max 10", + "percentile_key": VOL_PERCENTILE_KEY, + "cutoff": 80.0, + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "highvol90_fixed10", + "label": "High-vol 90 / max 10", + "percentile_key": VOL_PERCENTILE_KEY, + "cutoff": 90.0, + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, + { + "variant": "lowvol80_fixed10", + "label": "Low-vol 80 / max 10", + "percentile_key": LOW_VOL_PERCENTILE_KEY, + "cutoff": 80.0, + "max_positions": 10, + "risk_per_trade": 0.01, + "risk_scale": None, + }, ) @@ -1284,6 +1633,21 @@ def _qualifies_by_percentile(cand: dict, percentile_key: str, threshold: float) return pct is not None and pct >= threshold +def _variant_filters(cfg: dict) -> tuple[tuple[str, float], ...]: + filters = cfg.get("filters") + if filters is None: + return ((str(cfg["percentile_key"]), float(cfg["cutoff"])),) + return tuple((str(key), float(cutoff)) for key, cutoff in filters) + + +def _qualifies_strategy_variant(cand: dict, cfg: dict) -> bool: + """Research-only variant qualification with optional secondary gates.""" + return all( + _qualifies_by_percentile(cand, key, cutoff) + for key, cutoff in _variant_filters(cfg) + ) + + def _strategy_variant_sims( candidates: list[dict], prices: dict[str, tuple], @@ -1296,24 +1660,37 @@ def _strategy_variant_sims( for cfg in STRATEGY_VARIANTS: percentile_key = str(cfg["percentile_key"]) cutoff = float(cfg["cutoff"]) + ranking_key = str(cfg.get("ranking_key") or percentile_key) + filters = [ + {"percentile_key": key, "cutoff": threshold} + for key, threshold in _variant_filters(cfg) + ] sim = _simulate_portfolio( candidates, prices, _spy_closes, "hold", hold_days, - qualified_fn=lambda c, pk=percentile_key, th=cutoff: _qualifies_by_percentile(c, pk, th), - ranking_key=percentile_key, + qualified_fn=lambda c, config=cfg: _qualifies_strategy_variant(c, config), + ranking_key=ranking_key, max_positions=int(cfg["max_positions"]), risk_per_trade=float(cfg["risk_per_trade"]), ) if sim is None: continue + default_ranking = "residual" + if percentile_key == RAW_PERCENTILE_KEY: + default_ranking = "raw" + elif percentile_key == VOL_PERCENTILE_KEY: + default_ranking = "high_vol" + elif percentile_key == LOW_VOL_PERCENTILE_KEY: + default_ranking = "low_vol" rows.append({ "variant": cfg["variant"], "label": cfg["label"], - "ranking": "raw" if percentile_key == RAW_PERCENTILE_KEY else "residual", + "ranking": cfg.get("ranking", default_ranking), "cutoff": cutoff, + "filters": filters, "max_positions": int(cfg["max_positions"]), "risk_per_trade_pct": round(float(cfg["risk_per_trade"]) * 100, 2), "risk_scale": cfg["risk_scale"], @@ -1322,12 +1699,234 @@ def _strategy_variant_sims( return rows +EXIT_ENTRY_VARIANT = "residual80_highvol_blend80_20_fixed10" +EXIT_POLICY_VARIANTS: tuple[dict, ...] = ( + { + "exit_policy": "hold", + "label": "80/20 entry + 30d hold / initial stop", + "description": "Baseline: keep the initial ATR stop and exit at 30 trading days.", + }, + { + "exit_policy": "sma50", + "label": "80/20 entry + SMA50 break", + "description": "Exit at the close when price closes below its 50-day SMA.", + }, + { + "exit_policy": "low20", + "label": "80/20 entry + 20-day low break", + "description": "Exit at the close when price closes below the prior 20-day low.", + }, + { + "exit_policy": "technical40", + "label": "80/20 entry + technical score < 40", + "description": "Exit at the close when the price-derived technical score deteriorates below 40.", + }, + { + "exit_policy": "atr_trail3", + "label": "80/20 entry + 3x ATR trailing stop", + "description": "Trail the stop upward to highest close minus 3 ATR, evaluated from prior data.", + }, +) + + +PORTFOLIO_MONITOR_LOOKBACKS: tuple[dict, ...] = ( + {"lookback": "6m", "label": "6 months", "days": 183}, + {"lookback": "1y", "label": "1 year", "days": 365}, + {"lookback": "3y", "label": "3 years", "days": 365 * 3}, + {"lookback": "5y", "label": "5 years", "days": 365 * 5}, + {"lookback": "all", "label": "All history", "days": None}, +) + +PRODUCTION_PORTFOLIO_STRATEGY = "residual80_highvol80_20_atr3" +PORTFOLIO_MONITOR_STRATEGIES: tuple[dict, ...] = ( + { + "strategy": "legacy_residual80_hold", + "label": "Legacy residual 80 + 30d hold", + "description": "Previous production baseline: residual momentum gate/rank with 30-trading-day hold.", + "entry_variant": "production_residual_80_fixed10", + "exit_policy": "hold", + }, + { + "strategy": "residual80_highvol80_20_hold", + "label": "Residual/high-vol 80/20 + 30d hold", + "description": "Promoted entry candidate before adding the ATR trailing exit.", + "entry_variant": "residual80_highvol_blend80_20_fixed10", + "exit_policy": "hold", + }, + { + "strategy": PRODUCTION_PORTFOLIO_STRATEGY, + "label": "Production: residual/high-vol 80/20 + 3x ATR trail", + "description": "Residual gate, 80/20 residual/high-vol rank, 3x ATR trailing stop.", + "entry_variant": "residual80_highvol_blend80_20_fixed10", + "exit_policy": "atr_trail3", + "is_production": True, + }, +) + + +def _entry_variant_config(variant: str) -> dict | None: + return next((cfg for cfg in STRATEGY_VARIANTS if cfg["variant"] == variant), None) + + +def _exit_policy_sims( + candidates: list[dict], + prices: dict[str, tuple], + _spy_closes: dict[date, float] | None, + hold_days: int, +) -> list[dict]: + """Research-only exit variants over the promoted entry challenger.""" + entry_cfg = _entry_variant_config(EXIT_ENTRY_VARIANT) + if entry_cfg is None: + return [] + + rows: list[dict] = [] + ranking_key = str(entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]) + for cfg in EXIT_POLICY_VARIANTS: + sim = _simulate_portfolio( + candidates, + prices, + _spy_closes, + str(cfg["exit_policy"]), + hold_days, + qualified_fn=lambda c, config=entry_cfg: _qualifies_strategy_variant(c, config), + ranking_key=ranking_key, + max_positions=int(entry_cfg["max_positions"]), + risk_per_trade=float(entry_cfg["risk_per_trade"]), + ) + if sim is None: + continue + rows.append({ + "entry_variant": EXIT_ENTRY_VARIANT, + "exit_policy": cfg["exit_policy"], + "label": cfg["label"], + "description": cfg["description"], + "hold_days": hold_days, + **sim, + }) + return rows + + +def _lookback_start(max_ord: int | None, days: int | None) -> date | None: + if max_ord is None or days is None: + return None + return date.fromordinal(max_ord - days) + + +def _portfolio_monitor( + candidates: list[dict], + prices: dict[str, tuple], + _spy_closes: dict[date, float] | None, + hold_days: int, +) -> dict: + latest_ord = max((max(cols[0]) for cols in prices.values() if cols[0]), default=None) + rows: list[dict] = [] + for strategy in PORTFOLIO_MONITOR_STRATEGIES: + entry_cfg = _entry_variant_config(str(strategy["entry_variant"])) + if entry_cfg is None: + continue + ranking_key = str(entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]) + for lookback in PORTFOLIO_MONITOR_LOOKBACKS: + start = _lookback_start(latest_ord, lookback["days"]) + sim = _simulate_portfolio( + candidates, + prices, + _spy_closes, + str(strategy["exit_policy"]), + hold_days, + qualified_fn=lambda c, config=entry_cfg: _qualifies_strategy_variant(c, config), + ranking_key=ranking_key, + max_positions=int(entry_cfg["max_positions"]), + risk_per_trade=float(entry_cfg["risk_per_trade"]), + start_date=start, + include_curve=True, + ) + if sim is None: + continue + rows.append({ + "strategy": strategy["strategy"], + "label": strategy["label"], + "description": strategy["description"], + "is_production": bool(strategy.get("is_production")), + "entry_variant": strategy["entry_variant"], + "exit_policy": strategy["exit_policy"], + "lookback": lookback["lookback"], + "lookback_label": lookback["label"], + **sim, + }) + return { + "production_strategy": PRODUCTION_PORTFOLIO_STRATEGY, + "strategies": [ + { + "strategy": s["strategy"], + "label": s["label"], + "description": s["description"], + "is_production": bool(s.get("is_production")), + } + for s in PORTFOLIO_MONITOR_STRATEGIES + ], + "lookbacks": [ + {"lookback": l["lookback"], "label": l["label"]} + for l in PORTFOLIO_MONITOR_LOOKBACKS + ], + "runs": rows, + "note": ( + "Portfolio monitor runs supported named strategies across cached lookbacks. " + "Local snapshot backtests remain the research surface for broad variant sweeps." + ), + } + + def _pct_loss(base: float | None, candidate: float | None) -> float | None: if base is None or candidate is None or base <= 0: return None return (base - candidate) / base +def _is_promotion_candidate( + row: dict, + *, + base_sharpe: float | None, + base_dd: float | None, + base_cagr: float | None, +) -> bool: + if ( + base_sharpe is None or base_dd is None or base_cagr is None + or row.get("sharpe") is None + or row.get("cagr_pct") is None + or row.get("max_drawdown_pct") is None + ): + return False + cagr_loss = _pct_loss(base_cagr, row.get("cagr_pct")) + return ( + row["sharpe"] > base_sharpe + and row["max_drawdown_pct"] <= base_dd + and cagr_loss is not None and cagr_loss < 0.10 + ) + + +def _best_research_row( + rows: list[dict], + *, + base_sharpe: float | None, + base_dd: float | None, + base_cagr: float | None, +) -> tuple[dict | None, bool]: + candidates = [ + row for row in rows + if _is_promotion_candidate( + row, base_sharpe=base_sharpe, base_dd=base_dd, base_cagr=base_cagr + ) + ] + if candidates: + return max(candidates, key=_sharpe_key), True + return max(rows, key=_sharpe_key, default=None), False + + +def _sharpe_key(row: dict) -> float: + sharpe = row.get("sharpe") + return float(sharpe) if sharpe is not None else -999.0 + + def _build_research_recommendation(report: dict) -> dict: """Advisory rules for the remaining research variants after residual promotion.""" variants = { @@ -1372,7 +1971,7 @@ def _build_research_recommendation(report: dict) -> dict: v for key, v in variants.items() if key.startswith("raw_90_") and v.get("risk_scale") is None ] - raw_90 = max(raw_90s, key=lambda v: v.get("sharpe") or -999, default=None) + raw_90 = max(raw_90s, key=_sharpe_key, default=None) if ( raw_90 and base_sharpe is not None and base_dd is not None and base_cagr is not None and raw_90.get("sharpe") is not None and raw_90.get("cagr_pct") is not None @@ -1397,6 +1996,54 @@ def _build_research_recommendation(report: dict) -> dict: ), }) + low_vol_rows = [ + v for key, v in variants.items() + if (key.startswith("residual80_lowvol") or key.startswith("lowvol")) + and v.get("risk_scale") is None + ] + low_vol, candidate = _best_research_row( + low_vol_rows, base_sharpe=base_sharpe, base_dd=base_dd, base_cagr=base_cagr + ) + if ( + low_vol and base_sharpe is not None and base_dd is not None and base_cagr is not None + and low_vol.get("sharpe") is not None and low_vol.get("cagr_pct") is not None + and low_vol.get("max_drawdown_pct") is not None + ): + items.append({ + "topic": "low_vol_overlay", + "candidate": candidate, + "text": ( + f"Low-vol overlay {'is a promotion candidate' if candidate else 'stays research-only'}: " + f"{low_vol['label']} Sharpe {low_vol['sharpe']:.2f} vs {base_sharpe:.2f}, " + f"drawdown {low_vol['max_drawdown_pct']:.1f}% vs {base_dd:.1f}%, " + f"CAGR {low_vol.get('cagr_pct'):+.1f}% vs {base_cagr:+.1f}%." + ), + }) + + high_vol_rows = [ + v for key, v in variants.items() + if (key.startswith("residual80_highvol") or key.startswith("highvol")) + and v.get("risk_scale") is None + ] + high_vol, candidate = _best_research_row( + high_vol_rows, base_sharpe=base_sharpe, base_dd=base_dd, base_cagr=base_cagr + ) + if ( + high_vol and base_sharpe is not None and base_dd is not None and base_cagr is not None + and high_vol.get("sharpe") is not None and high_vol.get("cagr_pct") is not None + and high_vol.get("max_drawdown_pct") is not None + ): + items.append({ + "topic": "high_vol_overlay", + "candidate": candidate, + "text": ( + f"High-vol overlay {'is a promotion candidate' if candidate else 'stays research-only'}: " + f"{high_vol['label']} Sharpe {high_vol['sharpe']:.2f} vs {base_sharpe:.2f}, " + f"drawdown {high_vol['max_drawdown_pct']:.1f}% vs {base_dd:.1f}%, " + f"CAGR {high_vol.get('cagr_pct'):+.1f}% vs {base_cagr:+.1f}%." + ), + }) + return { "items": items, "note": ( @@ -1422,10 +2069,42 @@ def _build_recommendation(report: dict) -> dict: run, so if the data flips, the advice flips. Rules are deliberately simple and transparent; thresholds are module constants above.""" items: list[dict] = [] + headline = None + monitor = report.get("portfolio_monitor") or {} + production_strategy = monitor.get("production_strategy") + production_rows = [ + row for row in monitor.get("runs", []) + if row.get("strategy") == production_strategy + ] + production_row = ( + next((row for row in production_rows if row.get("lookback") == "all"), None) + or next((row for row in production_rows if row.get("lookback") == "3y"), None) + or (production_rows[0] if production_rows else None) + ) + if production_row is not None: + headline = ( + "Production baseline: residual/high-vol 80/20 entry rank with a " + "3x ATR trailing exit and 30-trading-day max hold." + ) + if ( + production_row.get("cagr_pct") is not None + and production_row.get("sharpe") is not None + and production_row.get("max_drawdown_pct") is not None + ): + items.append({ + "topic": "production", + "text": ( + f"Production monitor ({production_row.get('lookback_label', 'selected window')}): " + f"{production_row.get('cagr_pct'):+.1f}% CAGR, " + f"Sharpe {production_row.get('sharpe'):.2f}, " + f"max drawdown -{production_row.get('max_drawdown_pct', 0):.1f}%." + ), + }) + q = report.get("overall_qualified") or {} target_net = q.get("net_avg_r") - # Exit policy: the production target/stop race vs the best fixed hold. + # Legacy diagnostic: target/stop race vs the best fixed hold. time_rows = [r for r in report.get("time_exit_sweep") or [] if r.get("net_avg_r") is not None] best_hold = max(time_rows, key=lambda r: r["net_avg_r"], default=None) sim_rows = { @@ -1436,7 +2115,7 @@ def _build_recommendation(report: dict) -> dict: if best_hold is not None and target_net is not None: if best_hold["net_avg_r"] > target_net + _EXIT_SWITCH_THRESHOLD: text = ( - f"Exit: hold {best_hold['hold_days']} trading days with the initial stop " + f"Legacy exit diagnostic: hold {best_hold['hold_days']} trading days with the initial stop " f"({best_hold['net_avg_r']:+.2f}R net/trade vs {target_net:+.2f}R for the S/R target exit)." ) target_sim = sim_rows.get("target") @@ -1453,7 +2132,7 @@ def _build_recommendation(report: dict) -> dict: items.append({ "topic": "exit", "text": ( - f"Exit: keep the S/R target exit ({target_net:+.2f}R net/trade) — " + f"Legacy exit diagnostic: keep the S/R target exit ({target_net:+.2f}R net/trade) — " "no fixed hold beats it by a meaningful margin." ), }) @@ -1551,8 +2230,7 @@ def _build_recommendation(report: dict) -> dict: ), }) - headline = None - if hold_recommended: + if headline is None and hold_recommended: cagr_note = ( f" (~{hold_sim['cagr_pct']:.0f}% CAGR simulated)" if hold_sim is not None and hold_sim.get("cagr_pct") is not None @@ -1672,7 +2350,10 @@ async def run_backtest( # (raw 12-1 fallback only when benchmark data is unavailable). _assign_momentum_percentiles(candidates) _assign_residual_momentum_percentiles(candidates) + _assign_low_volatility_percentiles(candidates) _assign_activation_momentum_percentiles(candidates) + _assign_residual_low_vol_blend(candidates) + _assign_residual_high_vol_blend(candidates) current_min_pct = float(activation.get("min_momentum_percentile", 80.0)) for c in candidates: c["qualified"] = _momentum_qualifies(c, current_min_pct) @@ -1695,17 +2376,14 @@ async def run_backtest( hold_horizon = max(TIME_EXIT_DAYS) sim_policies: list[dict] = [] strategy_variant_rows: list[dict] = [] + exit_policy_rows: list[dict] = [] + portfolio_monitor_report: dict | None = None try: qual_symbols = sorted({ c["symbol"] for c in candidates if c.get("qualified") - or any( - _qualifies_by_percentile( - c, str(cfg["percentile_key"]), float(cfg["cutoff"]) - ) - for cfg in STRATEGY_VARIANTS - ) + or any(_qualifies_strategy_variant(c, cfg) for cfg in STRATEGY_VARIANTS) }) price_columns: dict[str, tuple] = {} for sym in qual_symbols: @@ -1734,6 +2412,12 @@ async def run_backtest( strategy_variant_rows = _strategy_variant_sims( candidates, price_columns, spy_closes, hold_horizon ) + exit_policy_rows = _exit_policy_sims( + candidates, price_columns, spy_closes, hold_horizon + ) + portfolio_monitor_report = _portfolio_monitor( + candidates, price_columns, spy_closes, hold_horizon + ) except Exception: logger.exception("Portfolio simulation failed") @@ -1795,9 +2479,19 @@ async def run_backtest( "note": ( "Research-only hold-to-horizon portfolio variants. Production now " "uses residual 12-1 momentum at cutoff 80; the remaining rows compare " - "the legacy raw rank, raw cutoff 90, and one max-15 capacity check." + "the legacy raw rank, raw cutoff 90, one max-15 capacity check, and " + "volatility overlays." ), }, + "exit_policy_variants": { + "variants": exit_policy_rows, + "note": ( + "Research-only exit policies over the residual/high-vol 80/20 entry " + "candidate. Every row uses the same entry qualification/ranking and " + "changes only the exit discipline." + ), + }, + "portfolio_monitor": portfolio_monitor_report, "signal_eval": _signal_evaluation(collected), "signal_eval_note": ( "Cross-sectional rank-IC of price-only signals vs the forward " diff --git a/app/services/momentum_service.py b/app/services/momentum_service.py index 37f89ca..727a91a 100644 --- a/app/services/momentum_service.py +++ b/app/services/momentum_service.py @@ -122,15 +122,96 @@ async def compute_momentum_percentiles(db: AsyncSession) -> dict[str, float]: if value is not None: values[ticker.symbol] = value - ranked = sorted(values, key=lambda s: values[s]) - n = len(ranked) - percentiles = { - sym: round((rank / (n - 1) * 100.0) if n > 1 else 100.0, 2) - for rank, sym in enumerate(ranked) - } + percentiles = _percentiles(values) logger.info(json.dumps({ "event": "momentum_ranked", "signal": "residual_12_1" if using_residual else "raw_12_1_fallback", - "tickers": n, + "tickers": len(percentiles), })) return percentiles + + +def compute_realized_vol_6m(closes: list[float]) -> float | None: + """126-trading-day realized daily volatility. Higher = more volatile.""" + if len(closes) < 127: + return None + rets = [ + closes[k] / closes[k - 1] - 1.0 + for k in range(len(closes) - 126, len(closes)) + if closes[k - 1] > 0 + ] + if len(rets) < 2: + return None + mean = sum(rets) / len(rets) + var = sum((x - mean) ** 2 for x in rets) / (len(rets) - 1) + return var ** 0.5 + + +def _percentiles(values: dict[str, float]) -> dict[str, float]: + ranked = sorted(values, key=lambda s: values[s]) + n = len(ranked) + return { + sym: round((rank / (n - 1) * 100.0) if n > 1 else 100.0, 2) + for rank, sym in enumerate(ranked) + } + + +async def compute_activation_ranks(db: AsyncSession) -> dict[str, dict[str, float | None]]: + """Compute production activation ranks for the live scanner. + + ``momentum_percentile`` remains the residual/raw 12-1 gate. ``strategy_rank`` + is the promoted production ordering score: 80% activation momentum rank plus + 20% 6-month realized-volatility percentile. Live ranks are universe-wide + before scanning; the research backtest ranked each weekly setup-candidate + cross-section, so this is the deliberate production approximation. + """ + result = await db.execute(select(Ticker).order_by(Ticker.symbol)) + tickers = list(result.scalars().all()) + + benchmark_closes = await _load_activation_benchmark(db) + using_residual = len(benchmark_closes) >= _MOM_LOOKBACK + + momentum_values: dict[str, float] = {} + vol_values: dict[str, float] = {} + for ticker in tickers: + try: + records = await query_ohlcv(db, ticker.symbol) + except Exception: + logger.exception("Activation rank fetch failed for %s", ticker.symbol) + continue + closes = [float(r.close) for r in records] + momentum = ( + compute_residual_12_1_momentum([r.date for r in records], closes, benchmark_closes) + if using_residual + else compute_12_1_momentum(closes) + ) + if momentum is not None: + momentum_values[ticker.symbol] = momentum + vol = compute_realized_vol_6m(closes) + if vol is not None: + vol_values[ticker.symbol] = vol + + momentum_percentiles = _percentiles(momentum_values) + vol_percentiles = _percentiles(vol_values) + symbols = set(momentum_percentiles) | set(vol_percentiles) + ranks: dict[str, dict[str, float | None]] = {} + for sym in symbols: + momentum_pct = momentum_percentiles.get(sym) + vol_pct = vol_percentiles.get(sym) + strategy_rank = ( + round(momentum_pct * 0.8 + vol_pct * 0.2, 2) + if momentum_pct is not None and vol_pct is not None + else momentum_pct + ) + ranks[sym] = { + "momentum_percentile": momentum_pct, + "volatility_percentile": vol_pct, + "strategy_rank": strategy_rank, + } + + logger.info(json.dumps({ + "event": "activation_ranked", + "signal": "residual_12_1_plus_vol_80_20" if using_residual else "raw_12_1_plus_vol_80_20", + "tickers": len(ranks), + })) + return ranks diff --git a/app/services/paper_trade_service.py b/app/services/paper_trade_service.py index 46f6d0c..fbdb502 100644 --- a/app/services/paper_trade_service.py +++ b/app/services/paper_trade_service.py @@ -12,6 +12,7 @@ from app.models.ohlcv import OHLCVRecord from app.models.paper_trade import PaperTrade from app.models.ticker import Ticker from app.services import benchmark_service, settings_store +from app.services.indicator_service import compute_atr from app.services.outcome_service import ( OUTCOME_AMBIGUOUS, OUTCOME_STOP_HIT, @@ -20,24 +21,25 @@ from app.services.outcome_service import ( evaluate_setup_against_bars, ) -# Exit policy for OPEN paper trades (auto-close). "time" holds a fixed number of -# trading days with the initial stop and exits at that day's close — the exit the -# July 2026 backtest validated (the classic momentum hold-and-re-rank); "trailing" -# rides a trailing stop; "target" closes at the setup's stop/target. Stored in -# SystemSetting so it's tunable + transparent in the UI. +# Exit policy for OPEN paper trades (auto-close). Production defaults to the +# July 2026 promoted strategy: initial stop + 3x ATR trailing stop, with a max +# 30-trading-day hold. The older percent trail and target/stop modes remain +# selectable for comparison. Stored in SystemSetting so it's tunable and visible. KEY_EXIT_MODE = "paper_exit_mode" KEY_TRAILING_PCT = "paper_trailing_pct" +KEY_ATR_MULTIPLIER = "paper_atr_multiplier" KEY_HOLD_DAYS = "paper_hold_days" -DEFAULT_EXIT_MODE = "time" +DEFAULT_EXIT_MODE = "atr_trailing" DEFAULT_TRAILING_PCT = 12.0 +DEFAULT_ATR_MULTIPLIER = 3.0 DEFAULT_HOLD_DAYS = 30 -_VALID_EXIT_MODES = ("time", "trailing", "target") +_VALID_EXIT_MODES = ("time", "trailing", "atr_trailing", "target") async def get_exit_policy(db: AsyncSession) -> dict: """Active auto-exit policy: - {'mode': 'time'|'trailing'|'target', 'trailing_pct': float, 'hold_days': int}.""" + {'mode': 'time'|'trailing'|'atr_trailing'|'target', ...}.""" mode = (await settings_store.get_value(db, KEY_EXIT_MODE, DEFAULT_EXIT_MODE)).strip().lower() if mode not in _VALID_EXIT_MODES: mode = DEFAULT_EXIT_MODE @@ -47,13 +49,24 @@ async def get_exit_policy(db: AsyncSession) -> dict: except (TypeError, ValueError): pct = DEFAULT_TRAILING_PCT pct = max(0.5, min(90.0, pct)) + raw_atr = await settings_store.get_value(db, KEY_ATR_MULTIPLIER, str(DEFAULT_ATR_MULTIPLIER)) + try: + atr_multiplier = float(raw_atr) + except (TypeError, ValueError): + atr_multiplier = DEFAULT_ATR_MULTIPLIER + atr_multiplier = max(0.5, min(10.0, atr_multiplier)) raw_days = await settings_store.get_value(db, KEY_HOLD_DAYS, str(DEFAULT_HOLD_DAYS)) try: hold_days = int(float(raw_days)) except (TypeError, ValueError): hold_days = DEFAULT_HOLD_DAYS hold_days = max(2, min(250, hold_days)) - return {"mode": mode, "trailing_pct": pct, "hold_days": hold_days} + return { + "mode": mode, + "trailing_pct": pct, + "atr_multiplier": atr_multiplier, + "hold_days": hold_days, + } async def set_exit_policy( @@ -61,18 +74,23 @@ async def set_exit_policy( *, mode: str | None = None, trailing_pct: float | None = None, + atr_multiplier: float | None = None, hold_days: int | None = None, ) -> dict: """Persist the auto-exit policy (admin). Validates inputs.""" if mode is not None: mode = mode.strip().lower() if mode not in _VALID_EXIT_MODES: - raise ValidationError("mode must be 'time', 'trailing' or 'target'") + raise ValidationError("mode must be 'time', 'trailing', 'atr_trailing' or 'target'") await settings_store.upsert_setting(db, KEY_EXIT_MODE, mode) if trailing_pct is not None: if not 0.5 <= float(trailing_pct) <= 90.0: raise ValidationError("trailing_pct must be between 0.5 and 90") await settings_store.upsert_setting(db, KEY_TRAILING_PCT, str(float(trailing_pct))) + if atr_multiplier is not None: + if not 0.5 <= float(atr_multiplier) <= 10.0: + raise ValidationError("atr_multiplier must be between 0.5 and 10") + await settings_store.upsert_setting(db, KEY_ATR_MULTIPLIER, str(float(atr_multiplier))) if hold_days is not None: if not 2 <= int(hold_days) <= 250: raise ValidationError("hold_days must be between 2 and 250") @@ -163,6 +181,107 @@ def _trailing_close( return None +def _atr_from_rows(rows: list[tuple], idx: int) -> float | None: + try: + result = compute_atr( + [float(r[2]) for r in rows[: idx + 1]], + [float(r[3]) for r in rows[: idx + 1]], + [float(r[4]) for r in rows[: idx + 1]], + ) + except Exception: + return None + atr = result.get("atr") + return float(atr) if atr and atr > 0 else None + + +def _atr_trailing_level( + direction: str, + entry: float, + init_stop: float, + atr_multiplier: float, + rows: list[tuple], + opened_on: date, +) -> float: + """Current ATR trailing stop level after all available post-entry closes.""" + long = direction == "long" + stop = float(init_stop) + anchor = float(entry) + for idx, (d, _, _, _, close) in enumerate(rows): + if d <= opened_on: + continue + close = float(close) + atr = _atr_from_rows(rows, idx) + if long: + anchor = max(anchor, close) + if atr is not None: + next_stop = anchor - atr_multiplier * atr + if next_stop < close: + stop = max(stop, next_stop) + else: + anchor = min(anchor, close) + if atr is not None: + next_stop = anchor + atr_multiplier * atr + if next_stop > close: + stop = min(stop, next_stop) + return stop + + +def _atr_trailing_close( + direction: str, + entry: float, + init_stop: float, + atr_multiplier: float, + hold_days: int, + rows: list[tuple], + opened_on: date, +) -> tuple[float, date, str] | None: + """Initial stop + ATR trailing stop + max hold, matching the portfolio sim. + + Stop checks happen before the same day's trailing update, so a newly ratcheted + stop becomes active on the next bar. Gaps through the stop fill at the open. + """ + long = direction == "long" + stop = float(init_stop) + anchor = float(entry) + bars_held = 0 + for idx, (d, open_, high, low, close) in enumerate(rows): + if d <= opened_on: + continue + open_ = float(open_) + high = float(high) + low = float(low) + close = float(close) + bars_held += 1 + + if long: + if low <= stop: + reason = "trailing" if stop > init_stop + 1e-9 else "stop" + return min(stop, open_), d, reason + else: + if high >= stop: + reason = "trailing" if stop < init_stop - 1e-9 else "stop" + return max(stop, open_), d, reason + + if bars_held >= hold_days: + return close, d, "time" + + atr = _atr_from_rows(rows, idx) + if long: + anchor = max(anchor, close) + if atr is not None: + next_stop = anchor - atr_multiplier * atr + if next_stop < close: + stop = max(stop, next_stop) + else: + anchor = min(anchor, close) + if atr is not None: + next_stop = anchor + atr_multiplier * atr + if next_stop > close: + stop = min(stop, next_stop) + + return None + + async def create_trade( db: AsyncSession, user_id: int, @@ -273,7 +392,8 @@ async def list_trades( # makes a provider call). benchmark_closes = await benchmark_service.load_benchmark_closes(db) - # Current trailing-stop level + distance for open trades (when trailing is active). + # Current trailing-stop level + distance for open trades (when a trailing + # policy is active). policy = await get_exit_policy(db) trailing_info: dict[int, tuple[float, float | None]] = {} if policy["mode"] == "trailing": @@ -294,6 +414,33 @@ async def list_trades( if cur: dist = ((cur - level) / cur * 100.0) if long else ((level - cur) / cur * 100.0) trailing_info[t.id] = (level, dist) + elif policy["mode"] == "atr_trailing": + atr_multiplier = float(policy["atr_multiplier"]) + for t, _ in rows: + if t.status != "open": + continue + bars_result = await db.execute( + select( + OHLCVRecord.date, OHLCVRecord.open, OHLCVRecord.high, + OHLCVRecord.low, OHLCVRecord.close, + ) + .where(OHLCVRecord.ticker_id == t.ticker_id) + .order_by(OHLCVRecord.date.asc()) + ) + level = _atr_trailing_level( + t.direction, + t.entry_price, + t.stop_loss, + atr_multiplier, + bars_result.all(), + t.opened_at.date(), + ) + cur = prices.get(t.ticker_id) + dist = None + if cur: + long = t.direction == "long" + dist = ((cur - level) / cur * 100.0) if long else ((level - cur) / cur * 100.0) + trailing_info[t.id] = (level, dist) return [ _to_dict(t, sym, prices.get(t.ticker_id), benchmark_closes, trailing_info.get(t.id)) @@ -337,10 +484,11 @@ async def close_trade( async def resolve_open_trades(db: AsyncSession) -> int: """Auto-close open trades per the active exit policy, from the daily bars. - Walks the bars after each trade's open. 'time' closes at the initial stop or - the hold_days-th close; 'trailing' at the trailing/initial stop; 'target' at - the setup's target or stop (same logic as the outcome evaluator). Trades that - have hit nothing stay open. Returns the count closed. + Walks the bars after each trade's open. 'atr_trailing' closes at the initial + stop, a 3x-ATR-style trailing stop, or the hold_days-th close; 'time' closes + at the initial stop or the hold_days-th close; 'trailing' uses the legacy + percent trail; 'target' uses the setup's target or stop. Trades that have hit + nothing stay open. Returns the count closed. """ result = await db.execute(select(PaperTrade).where(PaperTrade.status == "open")) open_trades = list(result.scalars().all()) @@ -350,6 +498,7 @@ async def resolve_open_trades(db: AsyncSession) -> int: policy = await get_exit_policy(db) mode = policy["mode"] trail_frac = policy["trailing_pct"] / 100.0 + atr_multiplier = float(policy["atr_multiplier"]) hold_days = policy["hold_days"] closed = 0 @@ -365,13 +514,13 @@ async def resolve_open_trades(db: AsyncSession) -> int: ) .order_by(OHLCVRecord.date.asc()) ) - rows = bars_result.all() - bars = [Bar(date=d, high=h, low=lo) for d, _, h, lo, _ in rows] + post_rows = bars_result.all() + bars = [Bar(date=d, high=h, low=lo) for d, _, h, lo, _ in post_rows] if not bars: continue if mode == "time": - hit = _time_close(trade.direction, trade.stop_loss, hold_days, rows) + hit = _time_close(trade.direction, trade.stop_loss, hold_days, post_rows) if hit is None: continue # neither the stop nor the hold horizon reached yet close_price, close_date, reason = hit @@ -380,6 +529,27 @@ async def resolve_open_trades(db: AsyncSession) -> int: if hit is None: continue # neither the trailing nor the initial stop reached yet close_price, close_date, reason = hit + elif mode == "atr_trailing": + all_bars_result = await db.execute( + select( + OHLCVRecord.date, OHLCVRecord.open, OHLCVRecord.high, + OHLCVRecord.low, OHLCVRecord.close, + ) + .where(OHLCVRecord.ticker_id == trade.ticker_id) + .order_by(OHLCVRecord.date.asc()) + ) + hit = _atr_trailing_close( + trade.direction, + trade.entry_price, + trade.stop_loss, + atr_multiplier, + hold_days, + all_bars_result.all(), + trade.opened_at.date(), + ) + if hit is None: + continue + close_price, close_date, reason = hit else: # max_bars beyond the data so a still-open trade returns undecided (not "expired"). outcome, outcome_date = evaluate_setup_against_bars( diff --git a/app/services/rr_scanner_service.py b/app/services/rr_scanner_service.py index e4705a8..25168dc 100644 --- a/app/services/rr_scanner_service.py +++ b/app/services/rr_scanner_service.py @@ -36,7 +36,7 @@ from app.services.recommendation_service import ( logger = logging.getLogger(__name__) -STRATEGY_VERSION = "residual_momentum_12_1_rr_time_v2" +STRATEGY_VERSION = "residual_highvol_80_20_atr_trail3_v1" async def _get_ticker(db: AsyncSession, symbol: str) -> Ticker: @@ -288,6 +288,21 @@ async def _create_signal_context_snapshots( "composite_score": float(comp.score) if comp else float(setup.composite_score), "composite_is_stale": bool(comp.is_stale) if comp else None, "composite_computed_at": comp.computed_at if comp else None, + "momentum_percentile": ( + float(setup.momentum_percentile) + if setup.momentum_percentile is not None + else None + ), + "volatility_percentile": ( + float(setup.volatility_percentile) + if setup.volatility_percentile is not None + else None + ), + "strategy_rank": ( + float(setup.strategy_rank) + if setup.strategy_rank is not None + else None + ), "dimensions": dims.get(setup.ticker_id, {}), } sentiment_context = ( @@ -348,12 +363,15 @@ async def scan_ticker( rr_threshold: float = 1.5, atr_multiplier: float = 1.5, momentum_percentile: float | None = None, + strategy_rank: float | None = None, + volatility_percentile: float | None = None, ) -> list[TradeSetup]: """Scan a single ticker for trade setups meeting the R:R threshold. ``momentum_percentile`` is the ticker's residual 12-1 momentum activation rank across the universe (computed by the caller), stored on each setup so - the activation gate can select the top slice.""" + the activation gate can select the top slice. ``strategy_rank`` is the + production ordering score used for top-pick ranking.""" ticker = await _get_ticker(db, symbol) records = await query_ohlcv(db, symbol) @@ -441,6 +459,8 @@ async def scan_ticker( composite_score=round(composite_score, 4), detected_at=now, momentum_percentile=momentum_percentile, + strategy_rank=strategy_rank, + volatility_percentile=volatility_percentile, )) if levels_below: @@ -475,6 +495,8 @@ async def scan_ticker( composite_score=round(composite_score, 4), detected_at=now, momentum_percentile=momentum_percentile, + strategy_rank=strategy_rank, + volatility_percentile=volatility_percentile, )) available_directions = {s.direction for s in setups} @@ -525,16 +547,17 @@ async def scan_all_tickers( tickers = list(result.scalars().all()) total = len(tickers) - # Rank the universe by residual 12-1 momentum up front so each new setup - # carries its activation percentile. Best-effort; the ranker falls back to - # raw 12-1 momentum only if benchmark data is unavailable. + # Rank the universe up front so each new setup carries both the residual + # activation gate percentile and the promoted production ordering score. + # Best-effort; the ranker falls back to raw 12-1 momentum only if benchmark + # data is unavailable. try: from app.services import momentum_service - percentiles = await momentum_service.compute_momentum_percentiles(db) + ranks = await momentum_service.compute_activation_ranks(db) except Exception: - logger.exception("Momentum ranking refresh failed") - percentiles = {} + logger.exception("Activation ranking refresh failed") + ranks = {} all_setups: list[TradeSetup] = [] for index, ticker in enumerate(tickers): @@ -555,7 +578,9 @@ async def scan_all_tickers( setups = await scan_ticker( db, ticker.symbol, rr_threshold, atr_multiplier, - momentum_percentile=percentiles.get(ticker.symbol), + momentum_percentile=(ranks.get(ticker.symbol) or {}).get("momentum_percentile"), + strategy_rank=(ranks.get(ticker.symbol) or {}).get("strategy_rank"), + volatility_percentile=(ranks.get(ticker.symbol) or {}).get("volatility_percentile"), ) all_setups.extend(setups) except Exception: @@ -605,6 +630,8 @@ async def get_trade_setups( latest_rows = list(latest_by_key.values()) latest_rows.sort( key=lambda row: ( + row[0].strategy_rank if row[0].strategy_rank is not None else -1.0, + row[0].momentum_percentile if row[0].momentum_percentile is not None else -1.0, row[0].confidence_score if row[0].confidence_score is not None else -1.0, row[0].rr_ratio, row[0].composite_score, @@ -632,6 +659,8 @@ async def get_trade_setups( ] rows_out.sort( key=lambda row: ( + row["strategy_rank"] if row["strategy_rank"] is not None else -1.0, + row["momentum_percentile"] if row["momentum_percentile"] is not None else -1.0, row["confidence_score"] if row["confidence_score"] is not None else -1.0, row["rr_ratio"], row["composite_score"], @@ -757,5 +786,7 @@ def _trade_setup_to_dict(setup: TradeSetup, symbol: str, price_context: dict | N "evaluated_at": setup.evaluated_at, "current_price": current_price, "momentum_percentile": setup.momentum_percentile, + "strategy_rank": setup.strategy_rank, + "volatility_percentile": setup.volatility_percentile, "context_as_of": context_as_of, } diff --git a/app/services/watchlist_service.py b/app/services/watchlist_service.py index e3e8266..296d59e 100644 --- a/app/services/watchlist_service.py +++ b/app/services/watchlist_service.py @@ -173,9 +173,10 @@ async def _enrich_entry( "dimensions": dims, "rr_ratio": setup.rr_ratio if setup else None, "rr_direction": setup.direction if setup else None, - # Residual 12-1 activation percentile (the top-pick selector); ticker-level, - # so any of the ticker's setups carries the same value. + # Residual 12-1 activation percentile gates qualification; strategy_rank + # is the promoted top-pick ordering score. "momentum_percentile": setup.momentum_percentile if setup else None, + "strategy_rank": setup.strategy_rank if setup else None, "sr_levels": sr_levels, "last_close": last_close, "change_pct": change_pct, diff --git a/frontend/src/components/admin/ExitPolicySettings.tsx b/frontend/src/components/admin/ExitPolicySettings.tsx index 93eb87e..6f8013a 100644 --- a/frontend/src/components/admin/ExitPolicySettings.tsx +++ b/frontend/src/components/admin/ExitPolicySettings.tsx @@ -6,14 +6,16 @@ import { SkeletonCard } from '../ui/Skeleton'; export function ExitPolicySettings() { const { data, isLoading } = useExitPolicy(); const update = useUpdateExitPolicy(); - const [mode, setMode] = useState('time'); + const [mode, setMode] = useState('atr_trailing'); const [pct, setPct] = useState(12); + const [atrMultiplier, setAtrMultiplier] = useState(3); const [holdDays, setHoldDays] = useState(30); useEffect(() => { if (data) { setMode(data.mode); setPct(data.trailing_pct); + setAtrMultiplier(data.atr_multiplier ?? 3); setHoldDays(data.hold_days ?? 30); } }, [data]); @@ -26,14 +28,15 @@ export function ExitPolicySettings() {

Paper-Trade Exit

How open paper trades auto-close (in the nightly/intraday outcome job).{' '} - Hold keeps the initial stop and exits at the Nth trading - day's close — the backtest-validated exit (classic momentum: hold ~a month, re-rank);{' '} - Trailing rides a trailing stop;{' '} + ATR trail is the promoted production exit: initial stop, + ATR trailing stop, and a max N-trading-day hold;{' '} + Hold keeps only the initial stop until the Nth trading + day's close; Percent trail is the older trailing mode;{' '} Target / stop closes at the setup's target or stop. The setup's initial stop is always the floor.

-
+
@@ -55,10 +59,24 @@ export function ExitPolicySettings() { step={1} value={holdDays} onChange={(e) => setHoldDays(Number(e.target.value))} - disabled={mode !== 'time'} + disabled={mode !== 'time' && mode !== 'atr_trailing'} className="w-full input-glass px-3 py-2 text-sm disabled:opacity-50" /> - Backtest optimum: 30 (its evaluation horizon). + Production max hold: 30 trading days. + +
diff --git a/frontend/src/components/dashboard/OpenTradesPanel.tsx b/frontend/src/components/dashboard/OpenTradesPanel.tsx index b0a65ee..5fbf62f 100644 --- a/frontend/src/components/dashboard/OpenTradesPanel.tsx +++ b/frontend/src/components/dashboard/OpenTradesPanel.tsx @@ -24,9 +24,13 @@ export function OpenTradesPanel() { const close = useClosePaperTrade(); const exitLabel = policy - ? policy.mode === 'trailing' - ? `auto-exit: trailing ${Math.round(policy.trailing_pct)}%` - : 'auto-exit: target/stop' + ? policy.mode === 'atr_trailing' + ? `auto-exit: ${(policy.atr_multiplier ?? 3).toFixed(1)}x ATR trail / ${policy.hold_days}d max` + : policy.mode === 'trailing' + ? `auto-exit: trailing ${Math.round(policy.trailing_pct)}%` + : policy.mode === 'time' + ? `auto-exit: ${policy.hold_days}d hold` + : 'auto-exit: target/stop' : null; const totals = useMemo(() => { diff --git a/frontend/src/components/signals/BacktestPanel.tsx b/frontend/src/components/signals/BacktestPanel.tsx index 58d1034..d75d779 100644 --- a/frontend/src/components/signals/BacktestPanel.tsx +++ b/frontend/src/components/signals/BacktestPanel.tsx @@ -1,3 +1,4 @@ +import { useMemo, useState } from 'react'; import { useMutation, useQueryClient } from '@tanstack/react-query'; import { useBacktestReport } from '../../hooks/useMarketRegime'; import { triggerJob } from '../../api/admin'; @@ -6,7 +7,13 @@ import { Callout } from '../ui/Callout'; import { Disclosure } from '../ui/Disclosure'; import { Section } from '../ui/Section'; import { useToast } from '../ui/Toast'; -import type { BacktestBucket, BacktestPortfolioPolicy, BacktestStrategyVariant } from '../../lib/types'; +import type { + BacktestBucket, + BacktestCurvePoint, + BacktestPortfolioMonitorRun, + BacktestPortfolioPolicy, + BacktestStrategyVariant, +} from '../../lib/types'; function fmtR(v: number | null | undefined): string { if (v === null || v === undefined) return '—'; @@ -23,6 +30,9 @@ function fmtSignedPct(v: number | null | undefined): string { if (v === null || v === undefined) return '—'; return `${v > 0 ? '+' : ''}${v.toFixed(1)}%`; } +function fmtDrawdown(v: number | null | undefined): string { + return v === null || v === undefined ? '—' : `-${Math.abs(v).toFixed(1)}%`; +} function fmtDays(v: number | null | undefined): string { return v === null || v === undefined ? '—' : `${v.toFixed(1)}d`; } @@ -120,16 +130,109 @@ function BucketRow({ label, b }: { label: string; b: BacktestBucket }) { ); } +function curvePath( + points: BacktestCurvePoint[], + min: number, + max: number, + w: number, + h: number, + pad: number, + startMs: number, + endMs: number, +): string { + if (points.length < 2) return ''; + const span = Math.max(max - min, 1); + const timeSpan = Math.max(endMs - startMs, 1); + return points + .map((p, i) => { + const t = new Date(p.date).getTime(); + const x = pad + ((t - startMs) / timeSpan) * (w - pad * 2); + const value = p.return_pct ?? 0; + const y = pad + (1 - (value - min) / span) * (h - pad * 2); + return `${i === 0 ? 'M' : 'L'}${x.toFixed(1)},${y.toFixed(1)}`; + }) + .join(' '); +} + +function EquityCurveChart({ run }: { run: BacktestPortfolioMonitorRun }) { + const portfolio = run.equity_curve ?? []; + const benchmark = run.benchmark_curve ?? []; + const values = [...portfolio, ...benchmark] + .map((p) => p.return_pct) + .filter((v): v is number => v !== null && v !== undefined); + if (portfolio.length < 2 || values.length === 0) { + return No equity curve points for this selection.; + } + + const min = Math.min(0, ...values); + const max = Math.max(0, ...values); + const times = [...portfolio, ...benchmark] + .map((p) => new Date(p.date).getTime()) + .filter((v) => Number.isFinite(v)); + if (times.length === 0) { + return No dated equity curve points for this selection.; + } + const startMs = Math.min(...times); + const endMs = Math.max(...times); + const w = 720; + const h = 240; + const pad = 28; + const portfolioPath = curvePath(portfolio, min, max, w, h, pad, startMs, endMs); + const benchmarkPath = curvePath(benchmark, min, max, w, h, pad, startMs, endMs); + const lastPortfolio = portfolio[portfolio.length - 1]?.return_pct ?? null; + const lastBenchmark = benchmark[benchmark.length - 1]?.return_pct ?? run.spy_return_pct; + + return ( +
+
+
+

{run.label}

+

{run.start_date} - {run.end_date}

+
+
+ Portfolio {fmtSignedPct(lastPortfolio)} + S&P 500 {fmtSignedPct(lastBenchmark)} +
+
+ + + + {benchmarkPath && ( + + )} + + {fmtSignedPct(max)} + {fmtSignedPct(min)} + +
+ ); +} + export function BacktestPanel() { const { data: report, isLoading } = useBacktestReport(); const queryClient = useQueryClient(); const toast = useToast(); + const [selectedStrategy, setSelectedStrategy] = useState(''); + const [selectedLookback, setSelectedLookback] = useState(''); const bestTimeAvgR = report?.time_exit_sweep && report.time_exit_sweep.length > 0 ? Math.max(...report.time_exit_sweep.map((r) => netOrGross(r) ?? -Infinity)) : null; const sim = report?.portfolio_sim ?? null; + const monitor = report?.portfolio_monitor ?? null; + const activeStrategy = + selectedStrategy || monitor?.production_strategy || monitor?.strategies[0]?.strategy || ''; + const activeLookback = + selectedLookback || (monitor?.lookbacks.some((l) => l.lookback === '3y') ? '3y' : monitor?.lookbacks[0]?.lookback) || ''; + const monitorRun = useMemo( + () => + monitor?.runs.find((row) => row.strategy === activeStrategy && row.lookback === activeLookback) ?? + monitor?.runs.find((row) => row.strategy === activeStrategy) ?? + monitor?.runs[0] ?? + null, + [monitor, activeStrategy, activeLookback], + ); const run = useMutation({ mutationFn: () => triggerJob('backtest'), @@ -182,6 +285,58 @@ export function BacktestPanel() { )}

+ {monitor && monitorRun && ( +
+
+
+

Portfolio monitor

+

+ Cached portfolio simulation for supported strategies, compared with S&P 500. +

+
+
+ + +
+
+ +
+ + + + + +
+ + + {monitor.note &&

{monitor.note}

} +
+ )} + {report.recommendation && report.recommendation.items.length > 0 && (

What this backtest recommends

@@ -487,8 +642,8 @@ export function BacktestPanel() { Metric {sim.policies.map((p) => ( - - {POLICY_LABELS[p.policy] ?? p.policy} + + {POLICY_LABELS[p.policy ?? ''] ?? p.policy ?? 'Policy'} ))} @@ -523,7 +678,7 @@ export function BacktestPanel() { {label} {sim.policies.map((p) => ( - + {fmt(p)} ))} diff --git a/frontend/src/lib/qualification.ts b/frontend/src/lib/qualification.ts index 8a53c49..289118d 100644 --- a/frontend/src/lib/qualification.ts +++ b/frontend/src/lib/qualification.ts @@ -75,7 +75,9 @@ export function topPickSymbol( if (all.length === 0) return null; const qualified = activation ? all.filter((t) => qualifiesSetup(t, activation)) : []; const top = [...qualified].sort( - (a, b) => (b.momentum_percentile ?? -Infinity) - (a.momentum_percentile ?? -Infinity), + (a, b) => + (b.strategy_rank ?? b.momentum_percentile ?? -Infinity) - + (a.strategy_rank ?? a.momentum_percentile ?? -Infinity), )[0]; return top?.symbol ?? null; } diff --git a/frontend/src/lib/types.ts b/frontend/src/lib/types.ts index 6146f89..7e61de0 100644 --- a/frontend/src/lib/types.ts +++ b/frontend/src/lib/types.ts @@ -20,6 +20,7 @@ export interface WatchlistEntry { rr_ratio: number | null; rr_direction: string | null; momentum_percentile: number | null; + strategy_rank: number | null; sr_levels: SRLevelSummary[]; last_close: number | null; change_pct: number | null; @@ -141,6 +142,8 @@ export interface TradeSetup { evaluated_at: string | null; current_price: number | null; momentum_percentile?: number | null; + strategy_rank?: number | null; + volatility_percentile?: number | null; context_as_of?: TradeSetupContextAsOf | null; recommendation_summary?: RecommendationSummary; } @@ -227,8 +230,9 @@ export interface PaperTrade { } export interface ExitPolicy { - mode: 'time' | 'trailing' | 'target'; + mode: 'time' | 'trailing' | 'atr_trailing' | 'target'; trailing_pct: number; + atr_multiplier: number; hold_days: number; } @@ -276,7 +280,7 @@ export interface BacktestTimeExitRow { } export interface BacktestPortfolioPolicy { - policy: string; + policy?: string; starting_capital: number; final_equity: number; total_return_pct: number; @@ -294,10 +298,19 @@ export interface BacktestPortfolioPolicy { skipped_book_full: number; spy_return_pct: number | null; yearly_returns?: { year: number; return_pct: number | null }[]; + exit_reasons?: Record; + equity_curve?: BacktestCurvePoint[]; + benchmark_curve?: BacktestCurvePoint[]; start_date: string; end_date: string; } +export interface BacktestCurvePoint { + date: string; + equity: number; + return_pct: number | null; +} + export interface BacktestRecommendation { headline: string | null; items: { topic: string; text: string }[]; @@ -337,6 +350,25 @@ export interface BacktestStrategyVariants { note?: string; } +export interface BacktestPortfolioMonitorRun extends BacktestPortfolioPolicy { + strategy: string; + label: string; + description: string; + is_production: boolean; + entry_variant: string; + exit_policy: string; + lookback: string; + lookback_label: string; +} + +export interface BacktestPortfolioMonitor { + production_strategy: string; + strategies: { strategy: string; label: string; description: string; is_production: boolean }[]; + lookbacks: { lookback: string; label: string }[]; + runs: BacktestPortfolioMonitorRun[]; + note?: string; +} + export interface BacktestGateAblationRow extends BacktestBucket { variant: string; // The same variant graded under the hold-to-horizon time exit. @@ -378,6 +410,8 @@ export interface BacktestReport { time_exit_sweep?: BacktestTimeExitRow[]; portfolio_sim?: BacktestPortfolioSim; strategy_variants?: BacktestStrategyVariants; + exit_policy_variants?: { variants: BacktestStrategyVariant[]; note?: string }; + portfolio_monitor?: BacktestPortfolioMonitor | null; recommendation?: BacktestRecommendation; research_recommendation?: BacktestResearchRecommendation; signal_eval?: BacktestSignalEvalRow[]; diff --git a/frontend/src/pages/DashboardPage.tsx b/frontend/src/pages/DashboardPage.tsx index 05f310a..b2eebfa 100644 --- a/frontend/src/pages/DashboardPage.tsx +++ b/frontend/src/pages/DashboardPage.tsx @@ -76,10 +76,16 @@ export default function DashboardPage() { [trades.data, activation.data], ); - // Rank only actionable/qualified setups by residual 12-1 momentum percentile. + // Rank only actionable/qualified setups by the production strategy score. + // Residual momentum still gates qualification; strategy_rank is the promoted + // 80/20 residual-momentum/high-vol ordering score when available. const topSetups: TradeSetup[] = useMemo(() => { return [...qualifiedSetups] - .sort((a, b) => (b.momentum_percentile ?? -Infinity) - (a.momentum_percentile ?? -Infinity)) + .sort( + (a, b) => + (b.strategy_rank ?? b.momentum_percentile ?? -Infinity) - + (a.strategy_rank ?? a.momentum_percentile ?? -Infinity), + ) .slice(0, 5); }, [qualifiedSetups]); @@ -194,7 +200,7 @@ export default function DashboardPage() {
{trades.isLoading && } {trades.isError && Failed to load setups} @@ -211,7 +217,8 @@ export default function DashboardPage() { Entry R:R Target Prob - Residual Mom. + Prod. Rank + Residual Action @@ -252,6 +259,9 @@ export default function DashboardPage() { })()} + {setup.strategy_rank != null ? `${Math.round(setup.strategy_rank)}%ile` : '—'} + + {setup.momentum_percentile != null ? `${Math.round(setup.momentum_percentile)}%ile` : '—'} @@ -264,7 +274,7 @@ export default function DashboardPage() {
- Momentum = ticker's 12-1 month rank across the universe (higher = stronger) + Production rank = 80% residual momentum + 20% realized volatility; residual still gates qualification. All setups → diff --git a/frontend/src/pages/TickerDetailPage.tsx b/frontend/src/pages/TickerDetailPage.tsx index b01c889..6461339 100644 --- a/frontend/src/pages/TickerDetailPage.tsx +++ b/frontend/src/pages/TickerDetailPage.tsx @@ -296,7 +296,7 @@ export default function TickerDetailPage() { )} {hasOpenTrade && ( diff --git a/scripts/run_backtest_snapshot.py b/scripts/run_backtest_snapshot.py index 3e68185..d084ef9 100644 --- a/scripts/run_backtest_snapshot.py +++ b/scripts/run_backtest_snapshot.py @@ -34,7 +34,12 @@ def _parse_args() -> argparse.Namespace: default=None, help="JSON report path. Defaults to reports/backtest-.json.", ) - parser.add_argument("--workers", type=int, default=None, help="Override backtest worker count.") + parser.add_argument( + "--workers", + type=int, + default=None, + help="Override backtest worker count. On an 8-thread machine, 6 is a good starting point.", + ) parser.add_argument( "--allow-spawn", action="store_true", @@ -53,6 +58,10 @@ def _pct(value: Any) -> str: return "-" if value is None else f"{float(value):+.1f}%" +def _drawdown_pct(value: Any) -> str: + return "-" if value is None else f"-{abs(float(value)):.1f}%" + + def _r(value: Any) -> str: return "-" if value is None else f"{float(value):+.2f}R" @@ -79,10 +88,46 @@ def _print_summary(report: dict) -> None: print(f" 30d hold total R: {_r(hold_30.get('total_r'))}") if hold_policy: print(f" hold CAGR: {_pct(hold_policy.get('cagr_pct'))}") - print(f" hold max drawdown: {_pct(hold_policy.get('max_drawdown_pct'))}") + print(f" hold max drawdown: {_drawdown_pct(hold_policy.get('max_drawdown_pct'))}") print(f" hold Sharpe: {hold_policy.get('sharpe')}") print(f" hold trades: {hold_policy.get('trades')}") + variants = list((report.get("strategy_variants") or {}).get("variants") or []) + ranked = sorted( + (v for v in variants if v.get("sharpe") is not None), + key=lambda v: v.get("sharpe"), + reverse=True, + ) + if ranked: + print(" top strategy variants by Sharpe:") + for row in ranked[:5]: + print( + " " + f"{row.get('variant')}: " + f"Sharpe {row.get('sharpe')}, " + f"CAGR {_pct(row.get('cagr_pct'))}, " + f"DD {_drawdown_pct(row.get('max_drawdown_pct'))}, " + f"trades {row.get('trades')}" + ) + + exits = list((report.get("exit_policy_variants") or {}).get("variants") or []) + ranked_exits = sorted( + (v for v in exits if v.get("sharpe") is not None), + key=lambda v: v.get("sharpe"), + reverse=True, + ) + if ranked_exits: + print(" exit policies for 80/20 entry by Sharpe:") + for row in ranked_exits[:5]: + print( + " " + f"{row.get('exit_policy')}: " + f"Sharpe {row.get('sharpe')}, " + f"CAGR {_pct(row.get('cagr_pct'))}, " + f"DD {_drawdown_pct(row.get('max_drawdown_pct'))}, " + f"trades {row.get('trades')}" + ) + async def _main() -> None: args = _parse_args() diff --git a/tests/unit/test_backtest_service.py b/tests/unit/test_backtest_service.py index 67104b0..abde56f 100644 --- a/tests/unit/test_backtest_service.py +++ b/tests/unit/test_backtest_service.py @@ -130,6 +130,45 @@ def test_activation_percentile_prefers_residual_with_raw_fallback(): assert cands[1][bt.PRODUCTION_PERCENTILE_KEY] == 70.0 +def test_low_volatility_percentile_prefers_lower_realized_vol(): + cands = [ + {"iso_week": (2026, 1), "vol_6m": 0.04}, + {"iso_week": (2026, 1), "vol_6m": 0.01}, + {"iso_week": (2026, 1), "vol_6m": 0.02}, + ] + + bt._assign_low_volatility_percentiles(cands) + + assert cands[1][bt.LOW_VOL_PERCENTILE_KEY] == 100.0 + assert cands[2][bt.LOW_VOL_PERCENTILE_KEY] == 50.0 + assert cands[0][bt.LOW_VOL_PERCENTILE_KEY] == 0.0 + + +def test_residual_low_vol_blend_is_research_only_rank(): + cands = [{ + bt.PRODUCTION_PERCENTILE_KEY: 80.0, + bt.LOW_VOL_PERCENTILE_KEY: 60.0, + }] + + bt._assign_residual_low_vol_blend(cands) + + assert cands[0][bt.RESIDUAL_LOW_VOL_BLEND_KEY] == 74.0 + + +def test_residual_high_vol_blend_is_research_only_rank(): + cands = [{ + bt.PRODUCTION_PERCENTILE_KEY: 80.0, + bt.VOL_PERCENTILE_KEY: 60.0, + }] + + bt._assign_residual_high_vol_blend(cands) + + assert cands[0][bt.RESIDUAL_HIGH_VOL_BLEND_90_10_KEY] == 78.0 + assert cands[0][bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY] == 76.0 + assert cands[0][bt.RESIDUAL_HIGH_VOL_BLEND_KEY] == 74.0 + assert cands[0][bt.RESIDUAL_HIGH_VOL_BLEND_60_40_KEY] == 72.0 + + def test_strategy_variants_keep_only_current_research_candidates(): variants = {cfg["variant"]: cfg for cfg in bt.STRATEGY_VARIANTS} @@ -142,9 +181,43 @@ def test_strategy_variants_keep_only_current_research_candidates(): assert variants["production_residual_80_fixed10"]["percentile_key"] == bt.PRODUCTION_PERCENTILE_KEY assert variants["legacy_raw_80_fixed10"]["percentile_key"] == bt.RAW_PERCENTILE_KEY assert variants["residual_80_fixed15"]["max_positions"] == 15 + assert variants["residual80_lowvol50_fixed10"]["filters"] == ( + (bt.PRODUCTION_PERCENTILE_KEY, 80.0), + (bt.LOW_VOL_PERCENTILE_KEY, 50.0), + ) + assert variants["residual80_lowvol_blend_fixed10"]["ranking_key"] == bt.RESIDUAL_LOW_VOL_BLEND_KEY + assert variants["residual80_highvol50_fixed10"]["filters"] == ( + (bt.PRODUCTION_PERCENTILE_KEY, 80.0), + (bt.VOL_PERCENTILE_KEY, 50.0), + ) + assert variants["residual80_highvol_blend90_10_fixed10"]["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_90_10_KEY + assert variants["residual80_highvol_blend80_20_fixed10"]["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY + assert variants["residual80_highvol_blend_fixed10"]["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_KEY + assert variants["residual80_highvol_blend60_40_fixed10"]["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_60_40_KEY + assert variants["highvol80_fixed10"]["percentile_key"] == bt.VOL_PERCENTILE_KEY + assert variants["lowvol80_fixed10"]["percentile_key"] == bt.LOW_VOL_PERCENTILE_KEY assert all(cfg["risk_scale"] is None for cfg in bt.STRATEGY_VARIANTS) +def test_low_vol_strategy_variant_applies_secondary_filter(): + cfg = { + "percentile_key": bt.PRODUCTION_PERCENTILE_KEY, + "cutoff": 80.0, + "filters": ( + (bt.PRODUCTION_PERCENTILE_KEY, 80.0), + (bt.LOW_VOL_PERCENTILE_KEY, 70.0), + ), + } + base = { + "meets_core": True, + "direction": "long", + bt.PRODUCTION_PERCENTILE_KEY: 85.0, + } + + assert bt._qualifies_strategy_variant({**base, bt.LOW_VOL_PERCENTILE_KEY: 75.0}, cfg) + assert not bt._qualifies_strategy_variant({**base, bt.LOW_VOL_PERCENTILE_KEY: 65.0}, cfg) + + def test_strategy_variant_sims_emit_fixed_variants_without_mutating_qualified(monkeypatch): cands = [{ "qualified": False, @@ -153,6 +226,13 @@ def test_strategy_variant_sims_emit_fixed_variants_without_mutating_qualified(mo "momentum_percentile": 90.0, "residual_momentum_percentile": 91.0, "activation_momentum_percentile": 91.0, + "low_vol_6m_percentile": 80.0, + "residual_low_vol_blend_score": 87.7, + "vol_6m_percentile": 20.0, + "residual_high_vol_blend_90_10_score": 83.9, + "residual_high_vol_blend_80_20_score": 76.8, + "residual_high_vol_blend_score": 69.7, + "residual_high_vol_blend_60_40_score": 62.6, }] calls = [] @@ -187,10 +267,57 @@ def test_strategy_variant_sims_emit_fixed_variants_without_mutating_qualified(mo assert all(call["exit_policy"] == "hold" for call in calls) assert any(call["ranking_key"] == bt.PRODUCTION_PERCENTILE_KEY for call in calls) assert any(call["ranking_key"] == bt.RAW_PERCENTILE_KEY for call in calls) + assert any(call["ranking_key"] == bt.RESIDUAL_LOW_VOL_BLEND_KEY for call in calls) + assert any(call["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_90_10_KEY for call in calls) + assert any(call["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY for call in calls) + assert any(call["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_KEY for call in calls) + assert any(call["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_60_40_KEY for call in calls) + assert any(call["ranking_key"] == bt.VOL_PERCENTILE_KEY for call in calls) + assert any(call["ranking_key"] == bt.LOW_VOL_PERCENTILE_KEY for call in calls) assert any(call["max_positions"] == 15 for call in calls) assert cands[0]["qualified"] is False +def test_exit_policy_sims_use_80_20_entry_variant(monkeypatch): + calls = [] + + def fake_sim(candidates, prices, spy_closes, exit_policy, hold_days, **kwargs): + calls.append({"exit_policy": exit_policy, "hold_days": hold_days, **kwargs}) + return { + "starting_capital": bt.SIM_STARTING_CAPITAL, + "final_equity": 11_000.0, + "total_return_pct": 10.0, + "cagr_pct": 9.0, + "max_drawdown_pct": 5.0, + "sharpe": 1.1, + "trades": 1, + "win_rate": 100.0, + "avg_trade_pnl": 100.0, + "best_trade_r": 1.0, + "worst_trade_r": 1.0, + "best_trade_pnl": 100.0, + "worst_trade_pnl": 100.0, + "avg_hold_days": 30.0, + "exit_reasons": {exit_policy: 1}, + "skipped_book_full": 0, + "spy_return_pct": 1.0, + "yearly_returns": [], + "start_date": "2026-01-01", + "end_date": "2026-02-01", + } + + monkeypatch.setattr(bt, "_simulate_portfolio", fake_sim) + + rows = bt._exit_policy_sims([], {}, {}, 30) + + assert [r["exit_policy"] for r in rows] == [ + cfg["exit_policy"] for cfg in bt.EXIT_POLICY_VARIANTS + ] + assert all(r["entry_variant"] == bt.EXIT_ENTRY_VARIANT for r in rows) + assert all(call["ranking_key"] == bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY for call in calls) + assert all(call["exit_policy"] != "target" for call in calls) + + def test_build_research_recommendation_applies_promotion_rules(): report = { "strategy_variants": {"variants": [ @@ -200,6 +327,10 @@ def test_build_research_recommendation_applies_promotion_rules(): "max_drawdown_pct": 20.0, "cagr_pct": 32.0, "skipped_book_full": 0}, {"variant": "raw_90_fixed10", "label": "Cutoff 90", "sharpe": 1.25, "max_drawdown_pct": 19.0, "cagr_pct": 28.0}, + {"variant": "residual80_highvol_blend_fixed10", "label": "High-vol 70/30", + "sharpe": 1.68, "max_drawdown_pct": 21.0, "cagr_pct": 43.0}, + {"variant": "residual80_highvol_blend80_20_fixed10", "label": "High-vol 80/20", + "sharpe": 1.55, "max_drawdown_pct": 19.0, "cagr_pct": 39.0}, ]}, } @@ -210,6 +341,8 @@ def test_build_research_recommendation_applies_promotion_rules(): assert "not needed yet" in by_topic["capacity_15"]["text"] assert by_topic["cutoff_90"]["candidate"] is False assert "Cutoff 90" in by_topic["cutoff_90"]["text"] + assert by_topic["high_vol_overlay"]["candidate"] is True + assert "High-vol 80/20" in by_topic["high_vol_overlay"]["text"] class TestStopFillR: @@ -442,6 +575,32 @@ class TestSimulatePortfolio: assert sim["trades"] == 1 assert sim["worst_trade_r"] == pytest.approx(-2.0) # (90 − 100) / 5 + def test_sma50_policy_exits_on_close_break(self): + closes = [100.0] * 56 + [90.0, 91.0] + prices = {"AAA": _sim_prices(self.ORD, closes)} + entry_ord = self.ORD + 55 + cand = _sim_cand("AAA", entry_ord, entry=100.0, stop=80.0, target=130.0) + + sim = bt._simulate_portfolio([cand], prices, None, "sma50", 30) + + assert sim is not None + assert sim["trades"] == 1 + assert sim["exit_reasons"] == {"sma50": 1} + assert sim["worst_trade_r"] == pytest.approx(-0.5) + + def test_low20_policy_exits_on_prior_low_break(self): + closes = [100.0] * 26 + [95.0, 96.0] + prices = {"AAA": _sim_prices(self.ORD, closes)} + entry_ord = self.ORD + 25 + cand = _sim_cand("AAA", entry_ord, entry=100.0, stop=80.0, target=130.0) + + sim = bt._simulate_portfolio([cand], prices, None, "low20", 30) + + assert sim is not None + assert sim["trades"] == 1 + assert sim["exit_reasons"] == {"low20": 1} + assert sim["worst_trade_r"] == pytest.approx(-0.25) + def test_nothing_qualified_returns_none(self): assert bt._simulate_portfolio([], {}, None, "hold", 30) is None @@ -547,6 +706,26 @@ def test_build_recommendation_flags_outlier_dependence(): assert robustness and "WARNING" in robustness[0] +def test_build_recommendation_prefers_production_monitor_headline(): + rec = bt._build_recommendation({ + "portfolio_monitor": { + "production_strategy": bt.PRODUCTION_PORTFOLIO_STRATEGY, + "runs": [{ + "strategy": bt.PRODUCTION_PORTFOLIO_STRATEGY, + "lookback": "all", + "lookback_label": "All history", + "cagr_pct": 44.4, + "sharpe": 1.72, + "max_drawdown_pct": 23.8, + }], + }, + "overall_qualified": {}, + }) + assert rec["headline"] is not None + assert "3x ATR trailing exit" in rec["headline"] + assert any(item["topic"] == "production" for item in rec["items"]) + + def test_window_setups_too_short_returns_empty(): assert bt._window_setups([], {}, {}) == [] @@ -607,7 +786,7 @@ async def test_run_backtest_smoke(session): for key in ( "overall_qualified", "overall_all", "by_direction", "sweep", "gate_ablation", "time_exit_sweep", "portfolio_sim", "strategy_variants", - "recommendation", "research_recommendation", + "exit_policy_variants", "portfolio_monitor", "recommendation", "research_recommendation", ): assert key in report # the oscillating series should yield at least some resolved setups @@ -633,6 +812,8 @@ async def test_run_backtest_smoke(session): assert isinstance(report["portfolio_sim"]["policies"], list) assert report["portfolio_sim"]["params"]["max_positions"] == bt.SIM_MAX_POSITIONS assert isinstance(report["strategy_variants"]["variants"], list) + assert isinstance(report["exit_policy_variants"]["variants"], list) + assert report["portfolio_monitor"] is None or isinstance(report["portfolio_monitor"]["runs"], list) # sweep: lowering the momentum-percentile cutoff can only add qualifiers sweep = sorted(report["sweep"], key=lambda r: r["min_momentum_percentile"], reverse=True) diff --git a/tests/unit/test_paper_trade_service.py b/tests/unit/test_paper_trade_service.py index 87fd90f..f5e65b1 100644 --- a/tests/unit/test_paper_trade_service.py +++ b/tests/unit/test_paper_trade_service.py @@ -204,13 +204,48 @@ class TestTrailingClose: assert svc._trailing_close("long", 100.0, 95.0, 0.12, bars) is None +class TestAtrTrailingClose: + def test_long_uses_ratchet_on_next_bar(self, monkeypatch): + monkeypatch.setattr(svc, "compute_atr", lambda *_args, **_kwargs: {"atr": 5.0}) + rows = [ + _r(date(2026, 1, 1), 100, 100, 100, 100), + _r(date(2026, 1, 2), 115, 121, 114, 120), + _r(date(2026, 1, 3), 106, 107, 103, 104), + ] + hit = svc._atr_trailing_close( + "long", 100.0, 95.0, 3.0, 30, rows, date(2026, 1, 1) + ) + assert hit is not None + price, when, reason = hit + assert price == pytest.approx(105.0) + assert when == date(2026, 1, 3) + assert reason == "trailing" + + def test_max_hold_still_closes(self, monkeypatch): + monkeypatch.setattr(svc, "compute_atr", lambda *_args, **_kwargs: {"atr": 50.0}) + rows = [ + _r(date(2026, 1, 1), 100, 100, 100, 100), + _r(date(2026, 1, 2), 101, 102, 100, 101), + _r(date(2026, 1, 3), 102, 103, 101, 102), + ] + assert svc._atr_trailing_close( + "long", 100.0, 95.0, 3.0, 2, rows, date(2026, 1, 1) + ) == (102.0, date(2026, 1, 3), "time") + + async def test_exit_policy_defaults_and_round_trip(session): - # Default: the backtest-validated hold-to-horizon exit. + # Default: the promoted production exit. assert await svc.get_exit_policy(session) == { - "mode": "time", "trailing_pct": 12.0, "hold_days": 30, + "mode": "atr_trailing", "trailing_pct": 12.0, + "atr_multiplier": 3.0, "hold_days": 30, + } + updated = await svc.set_exit_policy( + session, mode="target", trailing_pct=15.0, atr_multiplier=2.5, hold_days=21 + ) + assert updated == { + "mode": "target", "trailing_pct": 15.0, + "atr_multiplier": 2.5, "hold_days": 21, } - updated = await svc.set_exit_policy(session, mode="target", trailing_pct=15.0, hold_days=21) - assert updated == {"mode": "target", "trailing_pct": 15.0, "hold_days": 21} assert (await svc.get_exit_policy(session))["mode"] == "target" @@ -219,6 +254,8 @@ async def test_exit_policy_rejects_bad_input(session): await svc.set_exit_policy(session, mode="bogus") with pytest.raises(ValidationError): await svc.set_exit_policy(session, trailing_pct=200.0) + with pytest.raises(ValidationError): + await svc.set_exit_policy(session, atr_multiplier=20.0) with pytest.raises(ValidationError): await svc.set_exit_policy(session, hold_days=1) @@ -284,6 +321,18 @@ async def test_resolve_trailing_closes_with_reason(session): assert closed[0]["close_reason"] == "trailing" +async def test_resolve_atr_trailing_closes_with_reason(session, monkeypatch): + monkeypatch.setattr(svc, "compute_atr", lambda *_args, **_kwargs: {"atr": 5.0}) + await svc.set_exit_policy(session, mode="atr_trailing", atr_multiplier=3.0) + tid = await _seed(session, "AAA", close=100.0) + await _add_open_trade(session, tid, "long", entry=100.0, shares=10, days_ago=10) + await _add_bars(session, tid, [(121, 114), (107, 101)], start=date.today()) + assert await svc.resolve_open_trades(session) == 1 + closed = await svc.list_trades(session, 1, status="closed") + assert closed[0]["close_reason"] == "trailing" + assert closed[0]["current_price"] == pytest.approx(102.5) + + async def test_manual_close_sets_reason(session): await _seed(session, "AAA", close=112.0) trade = await svc.create_trade(session, 1, symbol="AAA", direction="long", @@ -300,3 +349,14 @@ async def test_list_open_exposes_trailing_stop(session): row = (await svc.list_trades(session, 1, status="open"))[0] assert row["trailing_stop"] == pytest.approx(110.0) # 125 * (1 - 0.12) assert row["trailing_distance_pct"] is not None + + +async def test_list_open_exposes_atr_trailing_stop(session, monkeypatch): + monkeypatch.setattr(svc, "compute_atr", lambda *_args, **_kwargs: {"atr": 5.0}) + await svc.set_exit_policy(session, mode="atr_trailing", atr_multiplier=3.0) + tid = await _seed(session, "AAA", close=120.0) + await _add_open_trade(session, tid, "long", entry=100.0, shares=10, days_ago=10) + await _add_bars(session, tid, [(125, 118)], start=date.today()) + row = (await svc.list_trades(session, 1, status="open"))[0] + assert row["trailing_stop"] == pytest.approx(106.5) # latest close 121.5 - 3 * 5 + assert row["trailing_distance_pct"] is not None