Compare commits
2 Commits
da0bb3367e
...
29b1a9a28c
| Author | SHA1 | Date | |
|---|---|---|---|
 dennisthiessen
|
29b1a9a28c | ||
|
dennisthiessen
|
84ce7c5c26 |
@@ -36,6 +36,40 @@ Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (we
|
||||
3. **Activation gate** — a setup *qualifies* only if it clears the R:R and confidence floors **and** ranks in the top momentum percentile of the universe (the validated edge is long-only momentum).
|
||||
4. **Top pick** — the highest-momentum qualified setup; highlighted on the Dashboard and labelled on the ticker page.
|
||||
|
||||
## Strategy Status — What's Validated and What Isn't
|
||||
|
||||
**Read this before touching scoring, gating, or setup logic.** The platform measures itself — a weekly-replay backtest plus a factor rank-IC harness (`app/services/backtest_service.py`) — and the verdicts below come from those reports (June 2026, ~5 years of OHLCV), not from opinion.
|
||||
|
||||
| Component | Verdict | Evidence |
|
||||
|---|---|---|
|
||||
| **12-1 cross-sectional momentum** (the activation gate, long-only) | **The only demonstrated edge — in-sample** | Qualified setups ≈ **+0.25R** avg vs ≈ −0.05R all-setups baseline; the percentile sweep is cleanly monotonic (cutoff 50 → +0.14R, 70 → +0.21R, 80 → +0.25R). Rank-IC ≈ 0.05, t ≈ 1.6 — right sign and size for the classic factor, **not yet statistically significant** |
|
||||
| S/R setup engine (ATR stops, S/R targets, reach-probability) | **No selection edge — execution/timing only** | ≈ breakeven (+0.01R) before the momentum gate. The probability model is honest (calibrated) but does not discriminate winners |
|
||||
| Composite score + 5 dimensions | **Display/ranking only** | Sub-scores are hand-built heuristics; none has a measured IC. Note: the "momentum" *dimension* is 5/20-day ROC — NOT the validated 12-1 factor (that lives in `momentum_service`) |
|
||||
| LLM sentiment | Display + a bounded composite adjustment (± weight × 100 pts around neutral 50) | Deliberately kept out of the setup engine; no point-in-time history to validate against yet |
|
||||
| Fundamentals | Feeds composite + confidence only | Latest values only, no history — same limitation |
|
||||
| Short setups | **Excluded while the momentum gate is active** | Backtest showed shorts fight the trend and drag expectancy |
|
||||
| Expected-value gate (removed June 2026) | Degenerate — do not resurrect | Structurally favored distant lottery targets; selected *worse*-than-random setups |
|
||||
|
||||
Caveats on the momentum result: in-sample, roughly one market regime, no transaction costs or slippage modeled, and the factor is beta-heavy (6-month volatility posted the top IC — that's beta, not alpha). The **out-of-sample proof is the forward paper-trade record**: Signals → Track Record compares live qualified expectancy against the backtest.
|
||||
|
||||
### The iron rule for strategy changes
|
||||
|
||||
A signal earns its way into selection **only** through the factor harness:
|
||||
|
||||
1. Add it as a point-in-time function of past bars in `_signal_values()` (`backtest_service.py`).
|
||||
2. Run the backtest (Admin → Jobs, or the weekly run) and read the **Signal edge** table (Signals → Track Record).
|
||||
3. Wire it into the gate or ranking **only if** |mean IC| ≳ 0.03 with a consistent sign and `reliable: true` (≥ 12 non-overlapping windows).
|
||||
|
||||
Corollaries: never let an unvalidated score gate setups; the outcome evaluator must keep scoring **all** setups (unqualified ones are the control group); LLM output stays display-only in the quant path.
|
||||
|
||||
### Highest-value next experiments (in order)
|
||||
|
||||
1. **Volatility-scaled momentum** — add `mom_12_1 / vol_6m` to `_signal_values`; risk-adjusted momentum typically beats raw and dampens momentum crashes.
|
||||
2. **Regime filter on the gate** — momentum crashes cluster in post-bear rebounds; `market_regime_service` already computes the SPY 50/200 trend, so test "qualify only in Risk-On" in the backtest before wiring it live.
|
||||
3. **Cost haircut in the backtest** — subtract a fixed per-trade cost (e.g. 0.1% per side) in the outcome aggregation so expectancy is net; a thin edge must survive costs.
|
||||
4. **More breadth, not more history** — widening the ranked universe (e.g. `nasdaq_all`) strengthens each week's cross-section and the IC t-stat, even if only the top slice is traded. (Deeper history was considered and declined.)
|
||||
5. **Exit tuning with the existing sweeps** — the report already sweeps fixed take-profits and trailing stops against the S/R-target model; momentum's edge lives in the right tail, so wide trailing exits (already the paper-trade default) tend to beat nearby S/R targets. Also worth testing: a pure time-based exit (hold ~1 month, re-rank) instead of the 30-day target/stop race.
|
||||
|
||||
## Key Use Cases
|
||||
|
||||
- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists qualified setups ranked by momentum with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop.
|
||||
@@ -188,9 +222,10 @@ npm run preview # Preview the production build locally
|
||||
# Backend tests (in-memory SQLite — no PostgreSQL needed)
|
||||
pytest tests/ -v
|
||||
|
||||
# Frontend tests
|
||||
# Frontend: there is no test suite — `npm test` calls vitest, which is not
|
||||
# installed. The frontend check is the full TypeScript build:
|
||||
cd frontend
|
||||
npm test
|
||||
npm run build
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
@@ -230,70 +265,68 @@ Configure in `.env` (copy from `.env.example`):
|
||||
|
||||
## Production Deployment (Debian 12)
|
||||
|
||||
**Ongoing deploys are automated.** Every push to `main` triggers the Gitea Actions pipeline (`.gitea/workflows/deploy.yml`): lint → test → rsync to the server → `pip install` → `alembic upgrade head` → restart `signalplatform.service` → health check. There is no manual deploy step; the steps below are only for provisioning a new server.
|
||||
|
||||
### 1. Install dependencies
|
||||
|
||||
```bash
|
||||
sudo apt update && sudo apt install -y python3.12 python3.12-venv postgresql nginx nodejs npm
|
||||
sudo apt update && sudo apt install -y python3.12 python3.12-venv postgresql nginx rsync
|
||||
```
|
||||
|
||||
### 2. Create service user
|
||||
### 2. Create the deploy user
|
||||
|
||||
The pipeline connects over SSH as this user; it owns the app directory and needs passwordless permission to restart the service:
|
||||
|
||||
```bash
|
||||
sudo useradd -r -s /usr/sbin/nologin stockdata
|
||||
sudo useradd -m deploy
|
||||
sudo mkdir -p /opt/signalplatform
|
||||
sudo chown deploy:deploy /opt/signalplatform
|
||||
echo 'deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart signalplatform.service' | sudo tee /etc/sudoers.d/deploy-restart
|
||||
```
|
||||
|
||||
### 3. Deploy application
|
||||
### 3. Configure the pipeline (Gitea repo settings)
|
||||
|
||||
Variables: `DEPLOY_HOST`, `DEPLOY_USER` (`deploy`), `DEPLOY_PATH` (`/opt/signalplatform`), `SSH_KNOWN_HOSTS` (host fingerprint), `SSH_PORT`. Secret: `SSH_PRIVATE_KEY` (matching the deploy user's authorized key).
|
||||
|
||||
### 4. Configure the app
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/stock-data-backend
|
||||
# Copy project files to /opt/stock-data-backend
|
||||
cd /opt/stock-data-backend
|
||||
python3.12 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install .
|
||||
```
|
||||
|
||||
### 4. Configure
|
||||
|
||||
```bash
|
||||
sudo cp .env.example /opt/stock-data-backend/.env
|
||||
sudo chown stockdata:stockdata /opt/stock-data-backend/.env
|
||||
# After the first pipeline run has synced the files:
|
||||
cp /opt/signalplatform/.env.example /opt/signalplatform/.env
|
||||
# Edit .env with production values (strong JWT_SECRET, real API keys, etc.)
|
||||
```
|
||||
|
||||
`.env` is excluded from the rsync, so it survives every deploy.
|
||||
|
||||
### 5. Database
|
||||
|
||||
Either trigger the workflow manually (workflow_dispatch) with `run_setup_db: true` — the deploy then runs `deploy/setup_db.sh` instead of plain migrations — or run it once by hand:
|
||||
|
||||
```bash
|
||||
DB_NAME=stock_data_backend DB_USER=stock_backend DB_PASS=strong_password ./deploy/setup_db.sh
|
||||
```
|
||||
|
||||
### 6. Build frontend
|
||||
### 6. Systemd service
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
### 7. Systemd service
|
||||
|
||||
```bash
|
||||
sudo cp deploy/stock-data-backend.service /etc/systemd/system/
|
||||
sudo cp deploy/signalplatform.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now stock-data-backend
|
||||
sudo systemctl enable --now signalplatform
|
||||
```
|
||||
|
||||
### 8. Nginx reverse proxy
|
||||
The unit runs uvicorn on `127.0.0.1:8998` as the `deploy` user, with `WorkingDirectory=/opt/signalplatform`.
|
||||
|
||||
### 7. Nginx reverse proxy
|
||||
|
||||
```bash
|
||||
sudo cp deploy/nginx.conf /etc/nginx/sites-available/stock-data-backend
|
||||
sudo ln -s /etc/nginx/sites-available/stock-data-backend /etc/nginx/sites-enabled/
|
||||
sudo cp deploy/nginx.conf /etc/nginx/sites-available/signalplatform
|
||||
sudo ln -s /etc/nginx/sites-available/signalplatform /etc/nginx/sites-enabled/
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
Nginx serves the frontend static files from `frontend/dist/` and proxies `/api/v1/` to the backend.
|
||||
Nginx serves the frontend static files from `frontend/dist/` (built on the CI runner and rsynced) and proxies `/api/v1/` to the backend.
|
||||
|
||||
### 9. SSL (recommended)
|
||||
### 8. SSL (recommended)
|
||||
|
||||
```bash
|
||||
sudo apt install certbot python3-certbot-nginx
|
||||
@@ -359,3 +392,62 @@ tests/
|
||||
├── unit/ # Unit tests
|
||||
└── property/ # Property-based tests (Hypothesis)
|
||||
```
|
||||
|
||||
## Maintainer Guide
|
||||
|
||||
Context for whoever — human or AI — continues this work. The owner pushes straight to `main` on a self-hosted Gitea remote (no PRs) and deploys manually.
|
||||
|
||||
### Invariants — do not break these
|
||||
|
||||
- **`app/services/qualification.py` is mirrored in `frontend/src/lib/qualification.ts`.** Any gate change must land in both, or the UI's "qualified" flags silently disagree with the server.
|
||||
- **Live scan and backtest share the same pure functions.** The backtest replays production logic through DB-free functions (`compute_technical_from_arrays`, `compute_momentum_from_closes`, `detect_sr_levels`, the recommendation helpers). New strategy logic must stay in pure functions consumed by both paths, or the backtest stops measuring what production actually does.
|
||||
- **One S/R model app-wide:** `sr_service.detect_sr_levels` + `cluster_sr_zones` (2% tolerance) feed the chart, alerts, and target generation identically.
|
||||
- **The outcome evaluator evaluates ALL setups**, not just qualified ones — unqualified setups are the control group that makes the Track Record meaningful.
|
||||
- **`SystemSetting` access goes through `app/services/settings_store.py`** — don't query the model directly.
|
||||
- **Time-series data gets a real table** (see `benchmark_prices`, `regime_snapshots`); `SystemSetting` JSON is only for config and cached reports.
|
||||
- Style: surgical changes, minimal new files; extend existing services rather than adding parallel ones.
|
||||
|
||||
### Where the strategy lives
|
||||
|
||||
| Concern | File |
|
||||
|---|---|
|
||||
| Composite + 5 dimension scores, weights | `app/services/scoring_service.py` |
|
||||
| 12-1 momentum ranking (the validated factor) | `app/services/momentum_service.py` |
|
||||
| Setup construction (ATR stop, S/R targets) | `app/services/rr_scanner_service.py` |
|
||||
| Confidence, targets, reach-probability, action | `app/services/recommendation_service.py` |
|
||||
| Activation gate predicate (mirrored in TS) | `app/services/qualification.py` |
|
||||
| Gate defaults / admin config | `app/services/admin_service.py` (`ACTIVATION_DEFAULTS`) |
|
||||
| Backtest + factor rank-IC harness ("Signal edge") | `app/services/backtest_service.py` |
|
||||
| Outcome resolution (target/stop/expired/ambiguous) | `app/services/outcome_service.py` |
|
||||
| Paper trades + trailing auto-exit | `app/services/paper_trade_service.py` |
|
||||
| S/R detection & zone clustering | `app/services/sr_service.py` |
|
||||
| SPY benchmark for paper-trade alpha | `app/services/benchmark_service.py` |
|
||||
| Pipelines & job registration | `app/scheduler.py` |
|
||||
|
||||
### Verifying changes
|
||||
|
||||
```bash
|
||||
pytest tests/ -q # backend; in-memory SQLite, no Postgres needed
|
||||
cd frontend && npm run build # full tsc check — this IS the frontend "test"
|
||||
```
|
||||
|
||||
- `npm test` in `frontend/` is dead (vitest isn't installed; there are no frontend test files). Use `npm run build`.
|
||||
- Backend tests that exercise services which `commit()` need a plain session fixture, not the rolling-back `db_session` — copy the pattern in `tests/unit/test_rr_scanner_integration.py`.
|
||||
- `ruff` reports ~11 pre-existing errors in old test files; those are not regressions.
|
||||
|
||||
### Deploying
|
||||
|
||||
Automated by Gitea Actions (`.gitea/workflows/deploy.yml`) on every push to `main`: **lint** (`ruff check app/`) → **test** (pytest; `alembic upgrade head` validated against a real Postgres 16 service; frontend `npm ci && npm run build`) → **deploy** (frontend built on the runner, rsync to the server excluding `.env`, `pip install -e .`, `alembic upgrade head`, restart `signalplatform.service`, health check on `127.0.0.1:8998`). Deploys are serialized by a concurrency group so overlapping pushes can't race.
|
||||
|
||||
Practical consequences:
|
||||
|
||||
- **A `ruff` error in `app/` or any failing backend test blocks the deploy.** (CI lints only `app/`, so the pre-existing ruff noise in old test files doesn't.)
|
||||
- **Migrations run automatically on deploy** — no manual `alembic` step. A migration that only works on SQLite will fail CI against Postgres, by design.
|
||||
- Pushing to `main` **is** deploying to production — there is no separate release step.
|
||||
- After a gate or scanner change ships, trigger an R:R scan (Admin → Jobs) so live setups pick up new fields.
|
||||
|
||||
### Roadmap (agreed June 2026)
|
||||
|
||||
1. **Forward paper-test the momentum book** — the out-of-sample proof the backtest can't give. Watch Signals → Track Record (live vs backtest).
|
||||
2. **Full IBKR integration** — read real positions, overlay entries/stops on charts, alert on holdings' score deterioration. (Paper trading, the lighter alternative, is done.)
|
||||
3. Strategy experiments in the order listed under **Strategy Status** above — each one goes through the factor harness first.
|
||||
|
||||
@@ -44,6 +44,7 @@ from app.services.outcome_service import (
|
||||
)
|
||||
from app.services.price_service import query_ohlcv
|
||||
from app.services.qualification import (
|
||||
HIGH_CONVICTION_ACTIONS,
|
||||
best_target_probability,
|
||||
setup_qualifies,
|
||||
)
|
||||
@@ -304,6 +305,47 @@ def _trailing_exits(
|
||||
return result
|
||||
|
||||
|
||||
def _time_exits(
|
||||
direction: str, entry: float, stop: float, forward: list, horizons
|
||||
) -> dict[int, float]:
|
||||
"""Realized R per hold-N-days exit, in one pass over the post-entry bars.
|
||||
|
||||
The initial stop stays active (fill at the stop level → −1R); otherwise the
|
||||
trade exits at the day-N close (the last available close when history ends
|
||||
early). No target, no trailing — the classic momentum implementation: buy,
|
||||
hold ~N days, re-rank. Same conservative bar logic as ``_tp_primitives``: a
|
||||
bar that pierces the stop is a loss before that bar's close counts.
|
||||
"""
|
||||
long = direction == "long"
|
||||
risk = abs(entry - stop) / entry if entry else 0.0
|
||||
if risk <= 0:
|
||||
return {int(n): 0.0 for n in horizons}
|
||||
bars = forward[: max(int(n) for n in horizons)]
|
||||
if not bars:
|
||||
return {int(n): 0.0 for n in horizons}
|
||||
|
||||
stop_day: int | None = None # 1-based trading day the stop was pierced
|
||||
closes: list[float] = []
|
||||
for i, r in enumerate(bars):
|
||||
if (r.low <= stop) if long else (r.high >= stop):
|
||||
stop_day = i + 1
|
||||
break
|
||||
closes.append(r.close)
|
||||
|
||||
result: dict[int, float] = {}
|
||||
for h in horizons:
|
||||
n = int(h)
|
||||
if stop_day is not None and stop_day <= n:
|
||||
result[n] = -1.0
|
||||
else:
|
||||
# closes can't be empty here: an empty closes means the stop hit on
|
||||
# day 1, which the branch above catches for every n >= 1.
|
||||
c = closes[min(n, len(closes)) - 1]
|
||||
move = (c - entry) / entry if long else (entry - c) / entry
|
||||
result[n] = move / risk
|
||||
return result
|
||||
|
||||
|
||||
def _replay_ticker(symbol: str, records: list, config: dict, activation: dict) -> list[dict]:
|
||||
"""Walk one ticker's history weekly, building setups and their realized outcomes."""
|
||||
candidates: list[dict] = []
|
||||
@@ -337,6 +379,9 @@ def _replay_ticker(symbol: str, records: list, config: dict, activation: dict) -
|
||||
trail_r = _trailing_exits(
|
||||
s["direction"], s["entry"], s["stop"], TRAIL_LEVELS, forward, HORIZON
|
||||
)
|
||||
time_r = _time_exits(
|
||||
s["direction"], s["entry"], s["stop"], forward, TIME_EXIT_DAYS
|
||||
)
|
||||
iso = records[i].date.isocalendar()
|
||||
candidates.append({
|
||||
"symbol": symbol,
|
||||
@@ -357,6 +402,7 @@ def _replay_ticker(symbol: str, records: list, config: dict, activation: dict) -
|
||||
"mfe_pct": mfe_pct,
|
||||
"tp_close_pct": tp_close_pct,
|
||||
"trail_r": trail_r,
|
||||
"time_r": time_r,
|
||||
})
|
||||
return candidates
|
||||
|
||||
@@ -367,6 +413,7 @@ def _bucket_stats(cands: list[dict]) -> dict:
|
||||
expired = sum(1 for c in cands if c["outcome"] not in (OUTCOME_TARGET_HIT, OUTCOME_STOP_HIT, OUTCOME_AMBIGUOUS))
|
||||
decided = wins + losses
|
||||
rs = [c["realized_r"] for c in cands]
|
||||
net_rs = [c["realized_r"] - _cost_r(c) for c in cands]
|
||||
return {
|
||||
"total": len(cands),
|
||||
"wins": wins,
|
||||
@@ -375,6 +422,8 @@ def _bucket_stats(cands: list[dict]) -> dict:
|
||||
"hit_rate": round(wins / decided * 100, 1) if decided else None,
|
||||
"avg_r": round(sum(rs) / len(rs), 3) if rs else None,
|
||||
"total_r": round(sum(rs), 2) if rs else None,
|
||||
"net_avg_r": round(sum(net_rs) / len(net_rs), 3) if net_rs else None,
|
||||
"net_total_r": round(sum(net_rs), 2) if net_rs else None,
|
||||
}
|
||||
|
||||
|
||||
@@ -382,10 +431,26 @@ def _bucket_stats(cands: list[dict]) -> dict:
|
||||
# Extended into the tail so the avg-R peak/plateau is visible (it's where letting
|
||||
# winners run stops paying). Note: this model ignores the setup's S/R target —
|
||||
# it's a standalone fixed-% exit; exiting at the target is the target model.
|
||||
TP_LEVELS = (0.04, 0.06, 0.08, 0.10, 0.12, 0.15, 0.20, 0.25, 0.30)
|
||||
TP_LEVELS = (0.04, 0.06, 0.08, 0.10, 0.12, 0.15, 0.20, 0.25, 0.30, 0.40, 0.50)
|
||||
|
||||
# Trailing-stop widths (give-back from the peak) swept for the trailing exit model.
|
||||
TRAIL_LEVELS = (0.03, 0.05, 0.07, 0.10, 0.15, 0.20)
|
||||
TRAIL_LEVELS = (0.03, 0.05, 0.07, 0.10, 0.15, 0.20, 0.25, 0.30)
|
||||
|
||||
# Hold-N-days exits (initial stop stays active, exit at the day-N close) — the
|
||||
# classic cross-sectional momentum implementation: buy, hold ~a month, re-rank.
|
||||
TIME_EXIT_DAYS = (5, 10, 21, 30)
|
||||
|
||||
# Assumed transaction cost per side as a fraction of notional (commission +
|
||||
# slippage). Aggregates report gross and net side by side; net subtracts a full
|
||||
# round trip, converted into R via the setup's stop distance (the 1R unit).
|
||||
COST_PER_SIDE = 0.001
|
||||
|
||||
|
||||
def _cost_r(cand: dict) -> float:
|
||||
"""Round-trip transaction cost in R units: two sides over the 1R stop
|
||||
distance. 0 when the candidate carries no usable risk_pct."""
|
||||
risk = cand.get("risk_pct") or 0.0
|
||||
return (2.0 * COST_PER_SIDE) / risk if risk > 0 else 0.0
|
||||
|
||||
|
||||
def _take_profit_bucket(cands: list[dict], tp: float) -> dict:
|
||||
@@ -394,18 +459,21 @@ def _take_profit_bucket(cands: list[dict], tp: float) -> dict:
|
||||
Results are in R (gain% / risk%) so they're comparable to the target model.
|
||||
``hit_rate`` here = share that reached +tp before the stop (the MFE CDF)."""
|
||||
rs: list[float] = []
|
||||
net_rs: list[float] = []
|
||||
wins = 0
|
||||
for c in cands:
|
||||
risk = c.get("risk_pct") or 0.0
|
||||
if risk <= 0:
|
||||
continue
|
||||
if c.get("mfe_pct", 0.0) >= tp:
|
||||
rs.append(tp / risk)
|
||||
r = tp / risk
|
||||
wins += 1
|
||||
elif c.get("tp_stopped"):
|
||||
rs.append(-1.0)
|
||||
r = -1.0
|
||||
else:
|
||||
rs.append((c.get("tp_close_pct", 0.0)) / risk)
|
||||
r = (c.get("tp_close_pct", 0.0)) / risk
|
||||
rs.append(r)
|
||||
net_rs.append(r - _cost_r(c))
|
||||
total = len(rs)
|
||||
return {
|
||||
"tp_pct": round(tp * 100, 1),
|
||||
@@ -414,6 +482,8 @@ def _take_profit_bucket(cands: list[dict], tp: float) -> dict:
|
||||
"hit_rate": round(wins / total * 100, 1) if total else None,
|
||||
"avg_r": round(sum(rs) / total, 3) if total else None,
|
||||
"total_r": round(sum(rs), 2) if total else None,
|
||||
"net_avg_r": round(sum(net_rs) / total, 3) if total else None,
|
||||
"net_total_r": round(sum(net_rs), 2) if total else None,
|
||||
}
|
||||
|
||||
|
||||
@@ -421,12 +491,14 @@ def _trailing_bucket(cands: list[dict], trail_pct: int) -> dict:
|
||||
"""Stats for a trailing-stop exit of width ``trail_pct`` (integer percent).
|
||||
Each candidate carries its realized R for this width in ``trail_r``; a "win"
|
||||
is simply an exit in profit (R > 0)."""
|
||||
rs = [
|
||||
c["trail_r"][trail_pct]
|
||||
pairs = [
|
||||
(c["trail_r"][trail_pct], _cost_r(c))
|
||||
for c in cands
|
||||
if c.get("trail_r", {}).get(trail_pct) is not None
|
||||
]
|
||||
total = len(rs)
|
||||
total = len(pairs)
|
||||
rs = [r for r, _ in pairs]
|
||||
net_rs = [r - cost for r, cost in pairs]
|
||||
wins = sum(1 for r in rs if r > 0)
|
||||
return {
|
||||
"trail_pct": trail_pct,
|
||||
@@ -435,6 +507,33 @@ def _trailing_bucket(cands: list[dict], trail_pct: int) -> dict:
|
||||
"win_rate": round(wins / total * 100, 1) if total else None,
|
||||
"avg_r": round(sum(rs) / total, 3) if total else None,
|
||||
"total_r": round(sum(rs), 2) if total else None,
|
||||
"net_avg_r": round(sum(net_rs) / total, 3) if total else None,
|
||||
"net_total_r": round(sum(net_rs), 2) if total else None,
|
||||
}
|
||||
|
||||
|
||||
def _time_exit_bucket(cands: list[dict], hold_days: int) -> dict:
|
||||
"""Stats for the hold-``hold_days`` exit: initial stop active, otherwise out
|
||||
at the day-N close. Each candidate carries its realized R per hold length in
|
||||
``time_r``; a "win" is an exit in profit (R > 0)."""
|
||||
pairs = [
|
||||
(c["time_r"][hold_days], _cost_r(c))
|
||||
for c in cands
|
||||
if c.get("time_r", {}).get(hold_days) is not None
|
||||
]
|
||||
total = len(pairs)
|
||||
rs = [r for r, _ in pairs]
|
||||
net_rs = [r - cost for r, cost in pairs]
|
||||
wins = sum(1 for r in rs if r > 0)
|
||||
return {
|
||||
"hold_days": hold_days,
|
||||
"total": total,
|
||||
"wins": wins,
|
||||
"win_rate": round(wins / total * 100, 1) if total else None,
|
||||
"avg_r": round(sum(rs) / total, 3) if total else None,
|
||||
"total_r": round(sum(rs), 2) if total else None,
|
||||
"net_avg_r": round(sum(net_rs) / total, 3) if total else None,
|
||||
"net_total_r": round(sum(net_rs), 2) if total else None,
|
||||
}
|
||||
|
||||
|
||||
@@ -754,6 +853,72 @@ def _momentum_qualifies(cand: dict, threshold: float) -> bool:
|
||||
return mp is not None and mp >= threshold
|
||||
|
||||
|
||||
def _gate_ablation(candidates: list[dict], activation: dict, threshold: float) -> list[dict]:
|
||||
"""Which floors earn their keep: re-qualify the same candidates at the
|
||||
current momentum cutoff with one floor removed per row (long-only
|
||||
throughout, matching the live gate).
|
||||
|
||||
``all_floors`` uses the stored ``meets_core`` so it reproduces the qualified
|
||||
set exactly; the ablation rows recompute the remaining floors from stored
|
||||
candidate fields with the same comparisons as
|
||||
``qualification.setup_qualifies``. Optional tighteners (high-conviction /
|
||||
conflict exclusion), when enabled, stay applied in every ablation row so
|
||||
only the named floor varies.
|
||||
"""
|
||||
min_rr = float(activation.get("min_rr", 0.0))
|
||||
min_conf = float(activation.get("min_confidence", 0.0))
|
||||
exclude_neutral = bool(activation.get("exclude_neutral", False))
|
||||
require_high = bool(activation.get("require_high_conviction", False))
|
||||
exclude_conflicts = bool(activation.get("exclude_conflicts", False))
|
||||
|
||||
def momentum_ok(c: dict) -> bool:
|
||||
# Mirrors the momentum part of _momentum_qualifies: long-only while the
|
||||
# gate is active; threshold 0 disables it (shorts pass too).
|
||||
if threshold <= 0:
|
||||
return True
|
||||
if c["direction"] == "short":
|
||||
return False
|
||||
mp = c.get("momentum_percentile")
|
||||
return mp is not None and mp >= threshold
|
||||
|
||||
def rr_ok(c: dict) -> bool:
|
||||
return c["rr"] >= min_rr
|
||||
|
||||
def conf_ok(c: dict) -> bool:
|
||||
return (c["confidence"] or 0.0) >= min_conf
|
||||
|
||||
def neutral_ok(c: dict) -> bool:
|
||||
return not exclude_neutral or (c.get("action") or "NEUTRAL") != "NEUTRAL"
|
||||
|
||||
def tighteners_ok(c: dict) -> bool:
|
||||
if require_high and (c.get("action") or "") not in HIGH_CONVICTION_ACTIONS:
|
||||
return False
|
||||
if exclude_conflicts and (c.get("risk_level") or "") != "Low":
|
||||
return False
|
||||
return True
|
||||
|
||||
def core_ok(c: dict) -> bool:
|
||||
return bool(c["meets_core"])
|
||||
|
||||
variants: list[tuple[str, list]] = [
|
||||
("all_floors", [core_ok]),
|
||||
("no_confidence_floor", [rr_ok, neutral_ok, tighteners_ok]),
|
||||
("no_rr_floor", [conf_ok, neutral_ok, tighteners_ok]),
|
||||
("no_neutral_exclusion", [rr_ok, conf_ok, tighteners_ok]),
|
||||
("momentum_only", []),
|
||||
]
|
||||
return [
|
||||
{
|
||||
"variant": name,
|
||||
**_bucket_stats([
|
||||
c for c in candidates
|
||||
if momentum_ok(c) and all(check(c) for check in checks)
|
||||
]),
|
||||
}
|
||||
for name, checks in variants
|
||||
]
|
||||
|
||||
|
||||
async def run_backtest(
|
||||
db: AsyncSession,
|
||||
progress_cb: Callable[[int, int, str], None] | None = None,
|
||||
@@ -862,7 +1027,12 @@ async def run_backtest(
|
||||
"tickers": total,
|
||||
"candidates": len(candidates),
|
||||
"qualified": len(qualified),
|
||||
"params": {"step_days": STEP_DAYS, "horizon_days": HORIZON, "min_lookback": MIN_LOOKBACK},
|
||||
"params": {
|
||||
"step_days": STEP_DAYS,
|
||||
"horizon_days": HORIZON,
|
||||
"min_lookback": MIN_LOOKBACK,
|
||||
"cost_per_side_pct": round(COST_PER_SIDE * 100, 3),
|
||||
},
|
||||
"activation": activation,
|
||||
"overall_qualified": _bucket_stats(qualified),
|
||||
"overall_all": _bucket_stats(candidates),
|
||||
@@ -872,8 +1042,16 @@ async def run_backtest(
|
||||
},
|
||||
"min_momentum_percentile": current_min_pct,
|
||||
"sweep": sweep,
|
||||
"gate_ablation": _gate_ablation(candidates, activation, current_min_pct),
|
||||
"gate_ablation_note": (
|
||||
"Each row re-qualifies the same candidates at the current momentum "
|
||||
f"cutoff ({current_min_pct:.0f}) with one floor removed (long-only "
|
||||
"while the momentum gate is active). If dropping a floor doesn't "
|
||||
"hurt net expectancy, that floor isn't pulling its weight."
|
||||
),
|
||||
"take_profit_sweep": [_take_profit_bucket(qualified, tp) for tp in TP_LEVELS],
|
||||
"trailing_sweep": [_trailing_bucket(qualified, round(f * 100)) for f in TRAIL_LEVELS],
|
||||
"time_exit_sweep": [_time_exit_bucket(qualified, n) for n in TIME_EXIT_DAYS],
|
||||
"calibration": _calibration(candidates),
|
||||
"signal_eval": _signal_evaluation(collected),
|
||||
"signal_eval_note": (
|
||||
|
||||
@@ -32,6 +32,20 @@ const SIGNAL_LABELS: Record<string, string> = {
|
||||
vol_6m: '6-month realized volatility',
|
||||
};
|
||||
|
||||
const ABLATION_LABELS: Record<string, string> = {
|
||||
all_floors: 'All floors (current gate)',
|
||||
no_confidence_floor: 'Without confidence floor',
|
||||
no_rr_floor: 'Without R:R floor',
|
||||
no_neutral_exclusion: 'Without NEUTRAL exclusion',
|
||||
momentum_only: 'Momentum only (no floors)',
|
||||
};
|
||||
|
||||
// Prefer the net-of-costs number when the report carries it; older cached
|
||||
// reports (pre-cost model) fall back to gross.
|
||||
function netOrGross(r: { avg_r: number | null; net_avg_r?: number | null }): number | null {
|
||||
return r.net_avg_r ?? r.avg_r;
|
||||
}
|
||||
|
||||
// An |IC| this large, with a consistent sign, is a real (if small) edge worth
|
||||
// building on; below it, ranking on the signal sorts essentially nothing.
|
||||
const IC_EDGE_THRESHOLD = 0.03;
|
||||
@@ -76,6 +90,7 @@ function BucketRow({ label, b }: { label: string; b: BacktestBucket }) {
|
||||
<td className="num px-4 py-2.5 text-right text-gray-400">{b.expired}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(b.hit_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(b.avg_r)}`}>{fmtR(b.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(b.net_avg_r ?? null)}`}>{fmtR(b.net_avg_r ?? null)}</td>
|
||||
</tr>
|
||||
);
|
||||
}
|
||||
@@ -87,11 +102,15 @@ export function BacktestPanel() {
|
||||
|
||||
const bestTpAvgR =
|
||||
report?.take_profit_sweep && report.take_profit_sweep.length > 0
|
||||
? Math.max(...report.take_profit_sweep.map((r) => r.avg_r ?? -Infinity))
|
||||
? Math.max(...report.take_profit_sweep.map((r) => netOrGross(r) ?? -Infinity))
|
||||
: null;
|
||||
const bestTrailAvgR =
|
||||
report?.trailing_sweep && report.trailing_sweep.length > 0
|
||||
? Math.max(...report.trailing_sweep.map((r) => r.avg_r ?? -Infinity))
|
||||
? Math.max(...report.trailing_sweep.map((r) => netOrGross(r) ?? -Infinity))
|
||||
: null;
|
||||
const bestTimeAvgR =
|
||||
report?.time_exit_sweep && report.time_exit_sweep.length > 0
|
||||
? Math.max(...report.time_exit_sweep.map((r) => netOrGross(r) ?? -Infinity))
|
||||
: null;
|
||||
|
||||
const run = useMutation({
|
||||
@@ -140,6 +159,9 @@ export function BacktestPanel() {
|
||||
<p className="text-[11px] text-gray-500">
|
||||
Ran {timeAgo(report.generated_at)} · {report.tickers} tickers · {report.candidates} setups
|
||||
({report.qualified} qualified) · weekly cadence, {report.params.horizon_days}-day horizon
|
||||
{report.params.cost_per_side_pct != null && (
|
||||
<> · net assumes {report.params.cost_per_side_pct}%/side costs</>
|
||||
)}
|
||||
</p>
|
||||
|
||||
<div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-4">
|
||||
@@ -179,6 +201,7 @@ export function BacktestPanel() {
|
||||
<th className="px-4 py-2.5 text-right">Expired</th>
|
||||
<th className="px-4 py-2.5 text-right">Hit Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
@@ -214,6 +237,7 @@ export function BacktestPanel() {
|
||||
<th className="px-4 py-2.5 text-right">Losses</th>
|
||||
<th className="px-4 py-2.5 text-right">Hit Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Total R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
@@ -231,6 +255,7 @@ export function BacktestPanel() {
|
||||
<td className="num px-4 py-2.5 text-right text-red-400">{row.losses}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(row.hit_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.net_avg_r ?? null)}`}>{fmtR(row.net_avg_r ?? null)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.total_r)}`}>{fmtR(row.total_r)}</td>
|
||||
</tr>
|
||||
);
|
||||
@@ -241,6 +266,51 @@ export function BacktestPanel() {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{report.gate_ablation && report.gate_ablation.length > 0 && (
|
||||
<div>
|
||||
<p className="mb-2 text-xs font-medium uppercase tracking-widest text-gray-500">
|
||||
Gate ablation — which floors earn their keep
|
||||
</p>
|
||||
<p className="mb-2 text-[11px] text-gray-500">
|
||||
{report.gate_ablation_note ??
|
||||
'Each row re-qualifies the same candidates at the current momentum cutoff with one floor removed (long-only throughout).'}
|
||||
</p>
|
||||
<div className="glass overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
<thead>
|
||||
<tr className="border-b border-white/[0.06] text-left text-xs uppercase tracking-wider text-gray-500">
|
||||
<th className="px-4 py-2.5">Variant</th>
|
||||
<th className="px-4 py-2.5 text-right">Setups</th>
|
||||
<th className="px-4 py-2.5 text-right">Hit Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Total R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{report.gate_ablation.map((row) => (
|
||||
<tr
|
||||
key={row.variant}
|
||||
className={`border-b border-white/[0.04] ${row.variant === 'all_floors' ? 'bg-blue-400/10' : ''}`}
|
||||
>
|
||||
<td className="px-4 py-2.5 font-medium text-gray-200">
|
||||
{ABLATION_LABELS[row.variant] ?? row.variant}
|
||||
</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{row.total}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(row.hit_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.net_avg_r ?? null)}`}>
|
||||
{fmtR(row.net_avg_r ?? null)}
|
||||
</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.total_r)}`}>{fmtR(row.total_r)}</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{report.take_profit_sweep && report.take_profit_sweep.length > 0 && (
|
||||
<div>
|
||||
<p className="mb-2 text-xs font-medium uppercase tracking-widest text-gray-500">
|
||||
@@ -253,7 +323,7 @@ export function BacktestPanel() {
|
||||
target model above. <span className="text-gray-300">Hit Rate = how often you'd have banked
|
||||
+X%</span> (how far winners actually run) — no top-ticking, it's the level you'd really set.
|
||||
The setup's own S/R target is <em>not</em> used here (exiting at that target is the model
|
||||
above); this is a pure fixed-% exit. ★ = best avg R.
|
||||
above); this is a pure fixed-% exit. ★ = best net avg R.
|
||||
</p>
|
||||
<div className="glass overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
@@ -264,12 +334,13 @@ export function BacktestPanel() {
|
||||
<th className="px-4 py-2.5 text-right">Hit (banked)</th>
|
||||
<th className="px-4 py-2.5 text-right">Hit Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Total R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{report.take_profit_sweep.map((row) => {
|
||||
const best = row.avg_r != null && row.avg_r === bestTpAvgR;
|
||||
const best = netOrGross(row) != null && netOrGross(row) === bestTpAvgR;
|
||||
return (
|
||||
<tr key={row.tp_pct} className={`border-b border-white/[0.04] ${best ? 'bg-emerald-400/[0.06]' : ''}`}>
|
||||
<td className="num px-4 py-2.5 text-gray-200">
|
||||
@@ -279,7 +350,8 @@ export function BacktestPanel() {
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{row.total}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-emerald-400">{row.wins}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(row.hit_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.net_avg_r ?? null)}`}>{fmtR(row.net_avg_r ?? null)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.total_r)}`}>{fmtR(row.total_r)}</td>
|
||||
</tr>
|
||||
);
|
||||
@@ -299,7 +371,7 @@ export function BacktestPanel() {
|
||||
Let it run, but exit when price gives back <span className="text-gray-300">X% from its
|
||||
peak</span> (the stop only ratchets up, never below the initial stop). Captures the tail
|
||||
without the fixed take-profit's all-or-nothing miss, and protects gains. In R vs the initial
|
||||
risk. <span className="text-gray-300">Win Rate = share closed in profit.</span> ★ = best avg R.
|
||||
risk. <span className="text-gray-300">Win Rate = share closed in profit.</span> ★ = best net avg R.
|
||||
</p>
|
||||
<div className="glass overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
@@ -310,12 +382,13 @@ export function BacktestPanel() {
|
||||
<th className="px-4 py-2.5 text-right">Profitable</th>
|
||||
<th className="px-4 py-2.5 text-right">Win Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Total R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{report.trailing_sweep.map((row) => {
|
||||
const best = row.avg_r != null && row.avg_r === bestTrailAvgR;
|
||||
const best = netOrGross(row) != null && netOrGross(row) === bestTrailAvgR;
|
||||
return (
|
||||
<tr key={row.trail_pct} className={`border-b border-white/[0.04] ${best ? 'bg-emerald-400/[0.06]' : ''}`}>
|
||||
<td className="num px-4 py-2.5 text-gray-200">
|
||||
@@ -325,7 +398,56 @@ export function BacktestPanel() {
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{row.total}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-emerald-400">{row.wins}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(row.win_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.net_avg_r ?? null)}`}>{fmtR(row.net_avg_r ?? null)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.total_r)}`}>{fmtR(row.total_r)}</td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{report.time_exit_sweep && report.time_exit_sweep.length > 0 && (
|
||||
<div>
|
||||
<p className="mb-2 text-xs font-medium uppercase tracking-widest text-gray-500">
|
||||
Time-based exit
|
||||
</p>
|
||||
<p className="mb-2 text-[11px] text-gray-500">
|
||||
Buy at detection, keep the initial ATR stop, and exit at the{' '}
|
||||
<span className="text-gray-300">day-N close</span> — no target, no trailing. This is the
|
||||
classic cross-sectional momentum implementation (hold ~a month, re-rank).{' '}
|
||||
<span className="text-gray-300">Win Rate = share closed in profit.</span> ★ = best net avg R.
|
||||
</p>
|
||||
<div className="glass overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
<thead>
|
||||
<tr className="border-b border-white/[0.06] text-left text-xs uppercase tracking-wider text-gray-500">
|
||||
<th className="px-4 py-2.5">Hold</th>
|
||||
<th className="px-4 py-2.5 text-right">Setups</th>
|
||||
<th className="px-4 py-2.5 text-right">Profitable</th>
|
||||
<th className="px-4 py-2.5 text-right">Win Rate</th>
|
||||
<th className="px-4 py-2.5 text-right">Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Net Avg R</th>
|
||||
<th className="px-4 py-2.5 text-right">Total R</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{report.time_exit_sweep.map((row) => {
|
||||
const best = netOrGross(row) != null && netOrGross(row) === bestTimeAvgR;
|
||||
return (
|
||||
<tr key={row.hold_days} className={`border-b border-white/[0.04] ${best ? 'bg-emerald-400/[0.06]' : ''}`}>
|
||||
<td className="num px-4 py-2.5 text-gray-200">
|
||||
{best && <span className="mr-1 text-emerald-300">★</span>}
|
||||
{row.hold_days}d
|
||||
</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{row.total}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-emerald-400">{row.wins}</td>
|
||||
<td className="num px-4 py-2.5 text-right text-gray-200">{fmtPct(row.win_rate)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.avg_r)}`}>{fmtR(row.avg_r)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right font-semibold ${rColor(row.net_avg_r ?? null)}`}>{fmtR(row.net_avg_r ?? null)}</td>
|
||||
<td className={`num px-4 py-2.5 text-right ${rColor(row.total_r)}`}>{fmtR(row.total_r)}</td>
|
||||
</tr>
|
||||
);
|
||||
|
||||
@@ -229,6 +229,9 @@ export interface BacktestBucket {
|
||||
hit_rate: number | null;
|
||||
avg_r: number | null;
|
||||
total_r: number | null;
|
||||
// Net of transaction costs — optional so a stale cached report still renders.
|
||||
net_avg_r?: number | null;
|
||||
net_total_r?: number | null;
|
||||
}
|
||||
|
||||
export interface BacktestCalibrationRow {
|
||||
@@ -249,6 +252,8 @@ export interface BacktestTakeProfitRow {
|
||||
hit_rate: number | null;
|
||||
avg_r: number | null;
|
||||
total_r: number | null;
|
||||
net_avg_r?: number | null;
|
||||
net_total_r?: number | null;
|
||||
}
|
||||
|
||||
export interface BacktestTrailingRow {
|
||||
@@ -258,6 +263,23 @@ export interface BacktestTrailingRow {
|
||||
win_rate: number | null;
|
||||
avg_r: number | null;
|
||||
total_r: number | null;
|
||||
net_avg_r?: number | null;
|
||||
net_total_r?: number | null;
|
||||
}
|
||||
|
||||
export interface BacktestTimeExitRow {
|
||||
hold_days: number;
|
||||
total: number;
|
||||
wins: number;
|
||||
win_rate: number | null;
|
||||
avg_r: number | null;
|
||||
total_r: number | null;
|
||||
net_avg_r?: number | null;
|
||||
net_total_r?: number | null;
|
||||
}
|
||||
|
||||
export interface BacktestGateAblationRow extends BacktestBucket {
|
||||
variant: string;
|
||||
}
|
||||
|
||||
export interface BacktestSignalEvalRow {
|
||||
@@ -276,14 +298,22 @@ export interface BacktestReport {
|
||||
tickers: number;
|
||||
candidates: number;
|
||||
qualified: number;
|
||||
params: { step_days: number; horizon_days: number; min_lookback: number };
|
||||
params: {
|
||||
step_days: number;
|
||||
horizon_days: number;
|
||||
min_lookback: number;
|
||||
cost_per_side_pct?: number;
|
||||
};
|
||||
overall_qualified: BacktestBucket;
|
||||
overall_all: BacktestBucket;
|
||||
by_direction: Record<string, BacktestBucket>;
|
||||
min_momentum_percentile: number;
|
||||
sweep: BacktestSweepRow[];
|
||||
gate_ablation?: BacktestGateAblationRow[];
|
||||
gate_ablation_note?: string;
|
||||
take_profit_sweep?: BacktestTakeProfitRow[];
|
||||
trailing_sweep?: BacktestTrailingRow[];
|
||||
time_exit_sweep?: BacktestTimeExitRow[];
|
||||
calibration: BacktestCalibrationRow[];
|
||||
signal_eval?: BacktestSignalEvalRow[];
|
||||
signal_eval_note?: string;
|
||||
|
||||
@@ -1 +1 @@
|
||||
{"root":["./src/app.tsx","./src/main.tsx","./src/vite-env.d.ts","./src/api/activation.ts","./src/api/admin.ts","./src/api/auth.ts","./src/api/client.ts","./src/api/fundamentals.ts","./src/api/health.ts","./src/api/indicators.ts","./src/api/ingestion.ts","./src/api/jobs.ts","./src/api/market.ts","./src/api/ohlcv.ts","./src/api/papertrades.ts","./src/api/performance.ts","./src/api/scores.ts","./src/api/sentiment.ts","./src/api/sr-levels.ts","./src/api/tickers.ts","./src/api/trades.ts","./src/api/watchlist.ts","./src/components/admin/activationsettings.tsx","./src/components/admin/alertsettings.tsx","./src/components/admin/datacleanup.tsx","./src/components/admin/jobcontrols.tsx","./src/components/admin/pipelinereadinesspanel.tsx","./src/components/admin/recommendationsettings.tsx","./src/components/admin/schedulesettings.tsx","./src/components/admin/sentimentprovidersettings.tsx","./src/components/admin/settingsform.tsx","./src/components/admin/tickermanagement.tsx","./src/components/admin/tickeruniversebootstrap.tsx","./src/components/admin/usertable.tsx","./src/components/auth/protectedroute.tsx","./src/components/charts/candlestickchart.tsx","./src/components/dashboard/opentradespanel.tsx","./src/components/layout/appshell.tsx","./src/components/layout/mobilenav.tsx","./src/components/layout/sidebar.tsx","./src/components/rankings/rankingstable.tsx","./src/components/rankings/weightsform.tsx","./src/components/scanner/tradetable.tsx","./src/components/signals/backtestpanel.tsx","./src/components/signals/mytradespanel.tsx","./src/components/signals/setupspanel.tsx","./src/components/signals/trackrecordpanel.tsx","./src/components/ticker/dimensionbreakdownpanel.tsx","./src/components/ticker/fundamentalspanel.tsx","./src/components/ticker/indicatorselector.tsx","./src/components/ticker/recommendationpanel.tsx","./src/components/ticker/sroverlay.tsx","./src/components/ticker/sentimentpanel.tsx","./src/components/ui/badge.tsx","./src/components/ui/button.tsx","./src/components/ui/callout.tsx","./src/components/ui/confirmdialog.tsx","./src/components/ui/disclosure.tsx","./src/components/ui/dropdown.tsx","./src/components/ui/field.tsx","./src/components/ui/pageheader.tsx","./src/components/ui/scorecard.tsx","./src/components/ui/section.tsx","./src/components/ui/skeleton.tsx","./src/components/ui/tabs.tsx","./src/components/ui/toast.tsx","./src/components/watchlist/addtickerform.tsx","./src/components/watchlist/watchlisttable.tsx","./src/hooks/useactivation.ts","./src/hooks/useadmin.ts","./src/hooks/useauth.ts","./src/hooks/usefetchsymboldata.ts","./src/hooks/usemarketregime.ts","./src/hooks/usepapertrades.ts","./src/hooks/useperformance.ts","./src/hooks/userisksettings.ts","./src/hooks/usescores.ts","./src/hooks/usetickerdetail.ts","./src/hooks/usetickers.ts","./src/hooks/usetrades.ts","./src/hooks/usewatchlist.ts","./src/lib/format.ts","./src/lib/ingestionstatus.ts","./src/lib/papertrade.ts","./src/lib/position.ts","./src/lib/qualification.ts","./src/lib/recommendation.ts","./src/lib/regime.ts","./src/lib/types.ts","./src/pages/adminpage.tsx","./src/pages/dashboardpage.tsx","./src/pages/loginpage.tsx","./src/pages/marketpage.tsx","./src/pages/registerpage.tsx","./src/pages/signalspage.tsx","./src/pages/tickerdetailpage.tsx","./src/stores/authstore.ts"],"version":"5.6.3"}
|
||||
{"root":["./src/app.tsx","./src/main.tsx","./src/vite-env.d.ts","./src/api/activation.ts","./src/api/admin.ts","./src/api/auth.ts","./src/api/client.ts","./src/api/fundamentals.ts","./src/api/health.ts","./src/api/indicators.ts","./src/api/ingestion.ts","./src/api/jobs.ts","./src/api/market.ts","./src/api/ohlcv.ts","./src/api/papertrades.ts","./src/api/performance.ts","./src/api/regime.ts","./src/api/scores.ts","./src/api/sentiment.ts","./src/api/sr-levels.ts","./src/api/tickers.ts","./src/api/trades.ts","./src/api/watchlist.ts","./src/components/admin/activationsettings.tsx","./src/components/admin/alertsettings.tsx","./src/components/admin/datacleanup.tsx","./src/components/admin/exitpolicysettings.tsx","./src/components/admin/jobcontrols.tsx","./src/components/admin/pipelinereadinesspanel.tsx","./src/components/admin/recommendationsettings.tsx","./src/components/admin/schedulesettings.tsx","./src/components/admin/sentimentprovidersettings.tsx","./src/components/admin/settingsform.tsx","./src/components/admin/tickermanagement.tsx","./src/components/admin/tickeruniversebootstrap.tsx","./src/components/admin/usertable.tsx","./src/components/auth/protectedroute.tsx","./src/components/charts/candlestickchart.tsx","./src/components/dashboard/opentradespanel.tsx","./src/components/layout/appshell.tsx","./src/components/layout/mobilenav.tsx","./src/components/layout/sidebar.tsx","./src/components/layout/tickersearch.tsx","./src/components/rankings/rankingstable.tsx","./src/components/rankings/weightsform.tsx","./src/components/regime/regimequadrant.tsx","./src/components/regime/scorehistorychart.tsx","./src/components/scanner/tradetable.tsx","./src/components/signals/backtestpanel.tsx","./src/components/signals/mytradespanel.tsx","./src/components/signals/setupspanel.tsx","./src/components/signals/trackrecordpanel.tsx","./src/components/ticker/dimensionbreakdownpanel.tsx","./src/components/ticker/fundamentalspanel.tsx","./src/components/ticker/indicatorselector.tsx","./src/components/ticker/recommendationpanel.tsx","./src/components/ticker/sroverlay.tsx","./src/components/ticker/sentimentpanel.tsx","./src/components/ticker/standingmatrix.tsx","./src/components/ui/badge.tsx","./src/components/ui/button.tsx","./src/components/ui/callout.tsx","./src/components/ui/confirmdialog.tsx","./src/components/ui/disclosure.tsx","./src/components/ui/dropdown.tsx","./src/components/ui/field.tsx","./src/components/ui/pageheader.tsx","./src/components/ui/scorecard.tsx","./src/components/ui/section.tsx","./src/components/ui/skeleton.tsx","./src/components/ui/tabs.tsx","./src/components/ui/toast.tsx","./src/components/watchlist/addtickerform.tsx","./src/components/watchlist/watchlisttable.tsx","./src/hooks/useactivation.ts","./src/hooks/useadmin.ts","./src/hooks/useauth.ts","./src/hooks/usefetchsymboldata.ts","./src/hooks/usemarketregime.ts","./src/hooks/usepapertrades.ts","./src/hooks/useperformance.ts","./src/hooks/userisksettings.ts","./src/hooks/usescores.ts","./src/hooks/usetickerdetail.ts","./src/hooks/usetickers.ts","./src/hooks/usetrades.ts","./src/hooks/usewatchlist.ts","./src/lib/format.ts","./src/lib/ingestionstatus.ts","./src/lib/papertrade.ts","./src/lib/position.ts","./src/lib/qualification.ts","./src/lib/recommendation.ts","./src/lib/regime.ts","./src/lib/types.ts","./src/pages/adminpage.tsx","./src/pages/dashboardpage.tsx","./src/pages/loginpage.tsx","./src/pages/marketpage.tsx","./src/pages/regimepage.tsx","./src/pages/registerpage.tsx","./src/pages/signalspage.tsx","./src/pages/tickerdetailpage.tsx","./src/stores/authstore.ts"],"version":"5.6.3"}
|
||||
@@ -25,7 +25,14 @@ async def session():
|
||||
yield s
|
||||
|
||||
|
||||
def _cand(prob: float, outcome: str, rr: float, qualified: bool = True, direction: str = "long") -> dict:
|
||||
def _cand(
|
||||
prob: float,
|
||||
outcome: str,
|
||||
rr: float,
|
||||
qualified: bool = True,
|
||||
direction: str = "long",
|
||||
risk_pct: float = 0.05,
|
||||
) -> dict:
|
||||
target_hit = outcome == OUTCOME_TARGET_HIT
|
||||
realized = rr if target_hit else (0.0 if outcome == OUTCOME_EXPIRED else -1.0)
|
||||
return {
|
||||
@@ -36,9 +43,14 @@ def _cand(prob: float, outcome: str, rr: float, qualified: bool = True, directio
|
||||
"realized_r": realized,
|
||||
"qualified": qualified,
|
||||
"direction": direction,
|
||||
"risk_pct": risk_pct,
|
||||
}
|
||||
|
||||
|
||||
# Round-trip cost in R for the default _cand risk_pct: 2 * 0.001 / 0.05 = 0.04R.
|
||||
_COST_R_005 = 2 * bt.COST_PER_SIDE / 0.05
|
||||
|
||||
|
||||
def _bar(high: float, low: float, close: float) -> SimpleNamespace:
|
||||
return SimpleNamespace(high=high, low=low, close=close)
|
||||
|
||||
@@ -87,6 +99,9 @@ class TestTakeProfitBucket:
|
||||
assert b["hit_rate"] == pytest.approx(33.3, abs=0.1)
|
||||
assert b["total_r"] == pytest.approx(0.8, abs=0.01)
|
||||
assert b["avg_r"] == pytest.approx(0.267, abs=0.01)
|
||||
# net: minus a 0.04R round trip per candidate (risk_pct 0.05)
|
||||
assert b["net_total_r"] == pytest.approx(0.8 - 3 * _COST_R_005, abs=0.01)
|
||||
assert b["net_avg_r"] == pytest.approx((0.8 - 3 * _COST_R_005) / 3, abs=0.01)
|
||||
|
||||
def test_zero_risk_skipped(self):
|
||||
cands = [{"risk_pct": 0.0, "mfe_pct": 0.2, "tp_stopped": False, "tp_close_pct": 0.1}]
|
||||
@@ -120,9 +135,9 @@ class TestTrailingExits:
|
||||
class TestTrailingBucket:
|
||||
def test_bucket(self):
|
||||
cands = [
|
||||
{"trail_r": {5: 1.4, 10: 0.8}},
|
||||
{"trail_r": {5: -1.0, 10: -1.0}},
|
||||
{"trail_r": {5: 0.5, 10: 0.5}},
|
||||
{"trail_r": {5: 1.4, 10: 0.8}, "risk_pct": 0.10},
|
||||
{"trail_r": {5: -1.0, 10: -1.0}, "risk_pct": 0.10},
|
||||
{"trail_r": {5: 0.5, 10: 0.5}, "risk_pct": 0.10},
|
||||
]
|
||||
b = bt._trailing_bucket(cands, 5)
|
||||
assert b["total"] == 3
|
||||
@@ -130,6 +145,116 @@ class TestTrailingBucket:
|
||||
assert b["win_rate"] == pytest.approx(66.7, abs=0.1)
|
||||
assert b["total_r"] == pytest.approx(0.9, abs=0.01)
|
||||
assert b["avg_r"] == pytest.approx(0.3, abs=0.01)
|
||||
# net: 0.02R round trip per candidate (risk_pct 0.10)
|
||||
assert b["net_total_r"] == pytest.approx(0.9 - 3 * 0.02, abs=0.01)
|
||||
assert b["net_avg_r"] == pytest.approx(0.28, abs=0.01)
|
||||
|
||||
|
||||
class TestTimeExits:
|
||||
def test_long_exits_at_horizon_close(self):
|
||||
bars = [_bar(103, 99, 102), _bar(105, 101, 104), _bar(107, 103, 106)]
|
||||
res = bt._time_exits("long", 100.0, 95.0, bars, (2, 5))
|
||||
assert res[2] == pytest.approx(0.8) # close 104 → +4% / 5% risk
|
||||
assert res[5] == pytest.approx(1.2) # only 3 bars → last close 106
|
||||
|
||||
def test_stop_on_first_bar_loses_everywhere(self):
|
||||
res = bt._time_exits("long", 100.0, 95.0, [_bar(101, 94, 96), _bar(105, 101, 104)], (1, 5))
|
||||
assert res[1] == pytest.approx(-1.0)
|
||||
assert res[5] == pytest.approx(-1.0)
|
||||
|
||||
def test_stop_after_short_horizon_only_hits_long_hold(self):
|
||||
# Day-2 close banked by the 2-day hold; the stop on day 3 only hits n=5.
|
||||
bars = [_bar(103, 99, 102), _bar(104, 100, 103), _bar(101, 94, 95)]
|
||||
res = bt._time_exits("long", 100.0, 95.0, bars, (2, 5))
|
||||
assert res[2] == pytest.approx(0.6) # close 103 → +3% / 5% risk
|
||||
assert res[5] == pytest.approx(-1.0)
|
||||
|
||||
def test_short_direction(self):
|
||||
res = bt._time_exits("short", 100.0, 105.0, [_bar(101, 95, 96)], (1,))
|
||||
assert res[1] == pytest.approx(0.8) # close 96 → +4% / 5% risk
|
||||
|
||||
def test_zero_risk_returns_zero(self):
|
||||
res = bt._time_exits("long", 100.0, 100.0, [_bar(103, 99, 102)], (5,))
|
||||
assert res[5] == 0.0
|
||||
|
||||
|
||||
class TestTimeExitBucket:
|
||||
def test_bucket(self):
|
||||
cands = [
|
||||
{"time_r": {5: 1.4, 21: 0.8}, "risk_pct": 0.10},
|
||||
{"time_r": {5: -1.0, 21: -1.0}, "risk_pct": 0.10},
|
||||
{"time_r": {5: 0.5, 21: 0.5}, "risk_pct": 0.10},
|
||||
]
|
||||
b = bt._time_exit_bucket(cands, 5)
|
||||
assert b["hold_days"] == 5
|
||||
assert b["total"] == 3
|
||||
assert b["wins"] == 2
|
||||
assert b["win_rate"] == pytest.approx(66.7, abs=0.1)
|
||||
assert b["avg_r"] == pytest.approx(0.3, abs=0.01)
|
||||
assert b["net_avg_r"] == pytest.approx(0.28, abs=0.01)
|
||||
|
||||
def test_missing_hold_skipped(self):
|
||||
b = bt._time_exit_bucket([{"time_r": {5: 1.0}}], 21)
|
||||
assert b["total"] == 0
|
||||
assert b["avg_r"] is None
|
||||
|
||||
|
||||
def _acand(
|
||||
rr: float = 2.0,
|
||||
conf: float = 60.0,
|
||||
action: str = "LONG_MODERATE",
|
||||
mp: float | None = 90.0,
|
||||
direction: str = "long",
|
||||
) -> dict:
|
||||
"""Ablation candidate: meets_core mirrors the default floors (min_rr 1.2,
|
||||
min_confidence 55, exclude_neutral on)."""
|
||||
meets = rr >= 1.2 and conf >= 55.0 and action != "NEUTRAL"
|
||||
return {
|
||||
"rr": rr,
|
||||
"confidence": conf,
|
||||
"action": action,
|
||||
"momentum_percentile": mp,
|
||||
"direction": direction,
|
||||
"meets_core": meets,
|
||||
"risk_level": "Low",
|
||||
"target_hit": True,
|
||||
"outcome": OUTCOME_TARGET_HIT,
|
||||
"realized_r": rr,
|
||||
"risk_pct": 0.05,
|
||||
}
|
||||
|
||||
|
||||
class TestGateAblation:
|
||||
ACTIVATION = {
|
||||
"min_rr": 1.2,
|
||||
"min_confidence": 55.0,
|
||||
"exclude_neutral": True,
|
||||
"require_high_conviction": False,
|
||||
"exclude_conflicts": False,
|
||||
}
|
||||
|
||||
def test_variant_counts(self):
|
||||
cands = [
|
||||
_acand(), # clears everything
|
||||
_acand(conf=40.0), # fails confidence floor
|
||||
_acand(rr=1.0), # fails R:R floor
|
||||
_acand(action="NEUTRAL"), # fails NEUTRAL exclusion
|
||||
_acand(mp=50.0), # fails the momentum cutoff
|
||||
_acand(direction="short", mp=95.0), # short — gated out
|
||||
]
|
||||
rows = {r["variant"]: r for r in bt._gate_ablation(cands, self.ACTIVATION, 80.0)}
|
||||
assert rows["all_floors"]["total"] == 1
|
||||
assert rows["no_confidence_floor"]["total"] == 2
|
||||
assert rows["no_rr_floor"]["total"] == 2
|
||||
assert rows["no_neutral_exclusion"]["total"] == 2
|
||||
assert rows["momentum_only"]["total"] == 4
|
||||
assert rows["all_floors"]["net_avg_r"] is not None
|
||||
|
||||
def test_threshold_zero_disables_momentum_gate(self):
|
||||
# Floors only: the short and the low-momentum long both pass all_floors.
|
||||
cands = [_acand(mp=50.0), _acand(direction="short", mp=None)]
|
||||
rows = {r["variant"]: r for r in bt._gate_ablation(cands, self.ACTIVATION, 0.0)}
|
||||
assert rows["all_floors"]["total"] == 2
|
||||
|
||||
|
||||
def test_bucket_stats_counts_and_expectancy():
|
||||
@@ -149,6 +274,9 @@ def test_bucket_stats_counts_and_expectancy():
|
||||
# avg R = (3 + 2 - 1 + 0) / 4 = 1.0
|
||||
assert s["avg_r"] == 1.0
|
||||
assert s["total_r"] == 4.0
|
||||
# net = gross minus a 0.04R round trip per candidate (risk_pct 0.05)
|
||||
assert s["net_avg_r"] == pytest.approx(1.0 - _COST_R_005, abs=0.001)
|
||||
assert s["net_total_r"] == pytest.approx(4.0 - 4 * _COST_R_005, abs=0.01)
|
||||
|
||||
|
||||
def test_bucket_stats_empty():
|
||||
@@ -156,6 +284,15 @@ def test_bucket_stats_empty():
|
||||
assert s["total"] == 0
|
||||
assert s["hit_rate"] is None
|
||||
assert s["avg_r"] is None
|
||||
assert s["net_avg_r"] is None
|
||||
|
||||
|
||||
def test_bucket_stats_no_risk_pct_means_no_cost():
|
||||
c = _cand(50, OUTCOME_TARGET_HIT, 2.0)
|
||||
del c["risk_pct"]
|
||||
s = bt._bucket_stats([c])
|
||||
assert s["net_avg_r"] == s["avg_r"]
|
||||
assert s["net_total_r"] == s["total_r"]
|
||||
|
||||
|
||||
def test_calibration_buckets():
|
||||
@@ -202,11 +339,25 @@ async def test_run_backtest_smoke(session):
|
||||
# well-formed report
|
||||
assert report["tickers"] == 1
|
||||
assert isinstance(report["candidates"], int)
|
||||
for key in ("overall_qualified", "overall_all", "by_direction", "calibration", "sweep"):
|
||||
for key in (
|
||||
"overall_qualified", "overall_all", "by_direction", "calibration", "sweep",
|
||||
"gate_ablation", "time_exit_sweep",
|
||||
):
|
||||
assert key in report
|
||||
# the oscillating series should yield at least some resolved setups
|
||||
assert report["candidates"] >= 1
|
||||
|
||||
# cost assumption is reported, and every bucket carries net numbers
|
||||
assert report["params"]["cost_per_side_pct"] == pytest.approx(bt.COST_PER_SIDE * 100)
|
||||
assert "net_avg_r" in report["overall_all"]
|
||||
|
||||
# ablation baseline reproduces the qualified set exactly
|
||||
ablation = {r["variant"]: r for r in report["gate_ablation"]}
|
||||
assert ablation["all_floors"]["total"] == report["overall_qualified"]["total"]
|
||||
|
||||
# time-exit sweep covers the configured hold lengths
|
||||
assert [r["hold_days"] for r in report["time_exit_sweep"]] == list(bt.TIME_EXIT_DAYS)
|
||||
|
||||
# sweep: lowering the momentum-percentile cutoff can only add qualifiers
|
||||
sweep = sorted(report["sweep"], key=lambda r: r["min_momentum_percentile"], reverse=True)
|
||||
counts = [r["total"] for r in sweep]
|
||||
|
||||
Reference in New Issue
Block a user