fix: chain morning alerts for regime Telegram delivery

Regime is computed at 02:00 ET; without a morning alert pass, quadrant
changes waited until 15:30. Dispatcher is change-driven so quiet days stay
quiet. Drop unused alerts_frequency config.

README.md

@@ -60,11 +60,11 @@ flowchart TD
 
 **What happens after an initial stop.** The stop always closes the trade and realizes its costs. The ticker is then locked until a successful daily full-universe scan first observes it outside the production gate and a later scan observes a fresh qualification. A continuously qualified ticker therefore cannot generate an immediate duplicate entry. Other exit reasons do not start this reset. See the [daily post-stop re-entry study](docs/research/post-stop-reentry.md).
 
-**Live timing matters.** The full daily pipeline runs the R:R scan before Outcome Eval. A stop closed by that Outcome Eval—or by an intraday evaluation after the day's full scan—therefore cannot use its stop-day gate state. The earliest failure observation is the next successful full scan, and requalification needs a subsequent full scan. The research `gate_reset` arm evaluated the stop before its same-session gate check; the live boundary is consequently analogous to the study's stricter `strict_gate_reset` arm. This known event-ordering difference is quantified below.
+**Live timing matters.** The **only** full-universe R:R scan runs near the US close (~15:30 ET), then Telegram alerts fire immediately so manual fills can still hit MOC. Outcome eval runs later (~16:45 ET) after a fresh OHLCV fetch of the final bar. Morning jobs refresh data/sentiment/regime without scanning. Stops closed by earlier same-day intraday outcome evals can get a **same-day** fail observation at the near-close scan — closer to the promoted research `gate_reset` arm than the old morning-scan `strict_gate_reset` analogue. Stops after the bell still need a later day. Same-day fail+qualify cannot unlock: `trade_policy` requires the failure to fall on an earlier America/New_York trading date.
 
 ## How It Works
 
-Scheduled pipelines turn raw prices into a ranked, gated list of tradeable setups. Everything downstream of OHLCV is recomputed from stored data, so each refresh is cheap and idempotent. Job timing is cron-based and configurable in **Admin → Jobs** (default timezone Europe/Berlin).
+Scheduled pipelines turn raw prices into a ranked, gated list of tradeable setups. Everything downstream of OHLCV is recomputed from stored data, so each refresh is cheap and idempotent. Job timing is cron-based and configurable in **Admin → Jobs** (default timezone **America/New_York** so the near-close scan tracks the cash close through DST).
 
 ### Price-level architecture: two different jobs
 
@@ -127,26 +127,35 @@ percentile rails show each input. Only momentum carries the live activation-
 gate marker. These are cross-sectional scan percentiles, not historical chart
 indicators.
 
-### Daily Load — the full refresh
+### Pipelines (America/New_York)
 
-Once a day (default 07:00). Steps run **in dependency order**, each consuming the previous step's fresh output:
+**Morning** (~02:00 ET) — data and display only, **no** qualifying R:R scan:
 
-1. **OHLCV** — fetch the latest daily bars for every tracked ticker (Alpaca); new tickers backfill ~5 years.
-2. **Sentiment** — fetch sentiment for the names that matter and are stale (> 5 days): top-pick feeders (residual-momentum leaders with a tradeable long setup), the watchlist, and open paper trades, plus a top-N-by-composite discovery net. Runs *before* the scan so the scan sees fresh sentiment.
-3. **R:R Scan** — persist clean Structural S/R for charts/alerts, recompute the 5-dimension scores, and build long/short setups from a transient Gate Target Ladder (ATR stops and nominal gate targets) for every ticker. Attach each ticker's residual 12‑1 momentum activation percentile plus the promoted 80/20 production rank. The completed full-universe scan also advances post-stop locks from gate failure to later requalification; failed scans never count as a transition.
-4. **Outcome Eval** — resolve setups that hit target/stop or expired (default 30 trading days) and auto-close paper trades per the exit policy (default: 3x ATR trail with a 30-trading-day max hold).
-5. **Market Regime** — recompute the regime index (breadth/trend).
-6. **Regime Monitor** — separate v2 State/Warning risk thermometer with fixed-basket breadth, VIX, credit, and point-in-time fundamentals; feeds no trades.
+1. **OHLCV** — latest daily bars (Alpaca); new tickers backfill ~5 years.
+2. **Sentiment** — stale names that matter (top-pick feeders, watchlist, open paper, discovery net). Display context only; the activation gate is price-only.
+3. **Market Regime** + **Regime Monitor** — breadth/trend and the v2 risk thermometer; feed no trades.
+4. **Telegram alerts** — change-driven (regime-quadrant etc.); quiet days stay quiet. Setup alerts still fire on the near-close pipeline after the scan.
 
-A failing step is logged; the pipeline continues with the next.
+**Near-close** (~15:30 ET Mon–Fri) — the only full-universe qualifying observation:
+
+1. **OHLCV fetch** — refresh the in-progress day-t bar (same path as intraday).
+2. **R:R Scan** — Structural S/R, scores, Gate Target Ladder setups, residual 12‑1 + 80/20 rank. Advances post-stop gate-reset transitions; failed scans never count.
+3. **Telegram alerts** — chained immediately so manual MOC fills can still hit ~15:50/15:55.
+
+**After close** (~16:45 ET Mon–Fri):
+
+1. **OHLCV fetch** — final bar (not the partial near-close bar).
+2. **Outcome Eval** — resolve setups and auto-close paper trades (default 3× ATR trail, 30-day max hold).
+
+A failing step is logged; the pipeline continues with the next. Near-close duration is logged; warn if > 10 minutes.
 
 ### Intraday — light refresh
 
-Hourly across the US session (Mon–Fri): only **OHLCV → Outcome Eval**, to keep prices current and close paper trades intraday. No scan/sentiment — the dashboard recomputes live R:R from the latest price, so fresh prices are enough.
+Hourly mid-session (Mon–Fri ~10:00–15:00 ET): only **OHLCV → Outcome Eval**, to keep prices current and close paper trades intraday. No scan/sentiment — the dashboard recomputes live R:R from the latest price.
 
 ### Other jobs
 
-Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (weekly) · Ticker-universe sync (daily). Deep history backfill and event study are manual-only (Admin → Jobs).
+Fundamentals (weekly, early Monday ET) · Backtest (weekly) · Ticker-universe sync (daily). Alerts auto-fire only via the near-close pipeline (still manually triggerable). Deep history backfill and event study are manual-only (Admin → Jobs).
 
 ### From score to "top pick"
 
@@ -193,7 +202,7 @@ The production policy is **normal gate reset**, evaluated with daily setup oppor
 
 In the disjoint 2025+ book, gate reset also beat immediate re-entry (Sharpe 1.66 vs 1.55; CAGR 41.8% vs 39.3%) and the fixed five-session rule (Sharpe 1.43; CAGR 32.7%). Its lead over both survived costs of 0.2% and 0.3% per side. The result is capacity-specific: cooldown 5 won at capacity 5, while immediate had slightly higher return and Sharpe at capacity 15. Production uses capacity 10, so that is the portfolio for which this decision is valid.
 
-Those promotion numbers belong to the selected normal-reset study arm. Under the live scheduler's stricter first-observation timing, the full-period analogue was Sharpe 1.68 / CAGR 44.8% / DD 23.4%; in the disjoint 2025+ book it was Sharpe 1.38 / CAGR 32.9% / DD 21.0%. The matrix therefore validates the state-machine choice but is not exact scheduler-order parity. Closing this timing gap would require a separately reviewed pipeline-order change, not a documentation reinterpretation.
+Those promotion numbers belong to the selected normal-reset study arm. Under the **pre-cutover** morning-scan scheduler (scan always before any outcome eval), live first-observation timing matched the stricter `strict_gate_reset` analogue (full-period Sharpe 1.68 / CAGR 44.8% / DD 23.4%). After the **near-close cutover** (2026-07), stops closed by earlier same-day intraday evals can receive a same-day fail observation at ~15:30 ET — moving live behavior **toward** the promoted `gate_reset` arm. Requalification still requires a later America/New_York trading date than the failure (`trade_policy` distinct-day guard). Full definitions and all nine policy arms: [docs/research/post-stop-reentry.md](docs/research/post-stop-reentry.md); execution evidence: [docs/research/execution-recovery.md](docs/research/execution-recovery.md).
 
 `gate_reset` and a simple `next_session` block happened to produce the same executed live-universe portfolio in this sample. Their rules are still different: this establishes that same-day re-entry was harmful here, but does not isolate a separate historical return premium from the reset condition. Gate reset was promoted because it represents a genuinely new signal episode and did not sacrifice results in the production book. Full definitions, all nine policy arms, cost/capacity sensitivity, and legacy-rank results are in [docs/research/post-stop-reentry.md](docs/research/post-stop-reentry.md); source report: [`reports/daily_reentry_matrix.json`](reports/daily_reentry_matrix.json).
 

alembic/versions/023_near_close_schedule_and_fill_mode.py

@@ -0,0 +1,73 @@
+"""near-close schedule cutover + paper trade fill_mode era tag
+
+Revision ID: 023
+Revises: 022
+Create Date: 2026-07-18 00:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+revision: str = "023"
+down_revision: Union[str, None] = "022"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+# Deliberate schedule rewrite (not a soft defaults refresh). Old stored values
+# are logged then replaced so prod does not keep scanning at 07:00 Berlin.
+_SCHEDULE_REWRITE: dict[str, str] = {
+    "schedule_timezone": "America/New_York",
+    "schedule_daily_pipeline_cron": "0 2 * * *",
+    "schedule_near_close_pipeline_cron": "30 15 * * 1-5",
+    "schedule_after_close_pipeline_cron": "45 16 * * 1-5",
+    "schedule_intraday_pipeline_cron": "0 10-15 * * 1-5",
+    "schedule_fundamentals_cron": "0 1 * * 1",
+}
+
+
+def upgrade() -> None:
+    op.add_column(
+        "paper_trades",
+        sa.Column("fill_mode", sa.String(length=20), nullable=True),
+    )
+
+    conn = op.get_bind()
+    settings = sa.table(
+        "system_settings",
+        sa.column("id", sa.Integer),
+        sa.column("key", sa.String),
+        sa.column("value", sa.Text),
+        sa.column("updated_at", sa.DateTime(timezone=True)),
+    )
+    now = sa.func.now()
+
+    for key, new_value in _SCHEDULE_REWRITE.items():
+        row = conn.execute(
+            sa.select(settings.c.value).where(settings.c.key == key)
+        ).fetchone()
+        old_value = row[0] if row is not None else None
+        # Always log so ops can recover the pre-cutover schedule from migration output.
+        print(
+            f"schedule_cutover {key}: {old_value!r} -> {new_value!r}",
+            flush=True,
+        )
+        if row is None:
+            conn.execute(
+                sa.insert(settings).values(
+                    key=key, value=new_value, updated_at=now
+                )
+            )
+        else:
+            conn.execute(
+                sa.update(settings)
+                .where(settings.c.key == key)
+                .values(value=new_value, updated_at=now)
+            )
+
+
+def downgrade() -> None:
+    op.drop_column("paper_trades", "fill_mode")
+    # Do not restore old crons — unknown prior values; leave stored schedule as-is.

app/config.py

@@ -59,8 +59,8 @@ class Settings(BaseSettings):
     sentiment_fresh_hours: int = 120
     sentiment_top_composite: int = 30
     fundamental_fetch_frequency: str = "weekly"  # quarterly-ish data; weekly conserves API quota
-    rr_scan_frequency: str = "daily"
-    alerts_frequency: str = "hourly"
+    rr_scan_frequency: str = "daily"  # legacy label; qualifying scan is cron near-close
+    # alerts_frequency removed: alerts fire only via morning + near-close pipelines
     fundamental_rate_limit_retries: int = 3
     fundamental_rate_limit_backoff_seconds: int = 15
     # Pause between tickers in the bulk fundamentals job. Free tiers throttle

app/models/paper_trade.py

@@ -46,3 +46,6 @@ class PaperTrade(Base):
     reentry_gate_requalified_at: Mapped[datetime | None] = mapped_column(
         DateTime(timezone=True), nullable=True
     )
+    # Execution era for forward vs backtest comparison:
+    # null/legacy = pre-cutover morning-scan, "near_close" = post near-close cutover.
+    fill_mode: Mapped[str | None] = mapped_column(String(20), nullable=True)

app/scheduler.py

@@ -93,7 +93,9 @@ _JOB_NAMES = [
     "regime_monitor",
     "event_study",
     "backtest",
-    "daily_pipeline",
+    "daily_pipeline",  # morning: OHLCV/sentiment/regime — no qualifying scan
+    "near_close_pipeline",  # OHLCV fetch → R:R scan → Telegram alerts
+    "after_close_pipeline",  # OHLCV fetch → outcome eval (final bar)
     "intraday_pipeline",
 ]
 
@@ -1177,15 +1179,43 @@ async def sync_ticker_universe() -> None:
 # updates its own runtime status while the pipeline runs.
 #
 # Daily (full): the complete data→signal refresh, once a day.
+# Morning (America/New_York ~02:00): refresh data + display context. No R:R scan
+# — the qualifying full-universe scan runs once near the US close so post-stop
+# gate-reset sees one observation per trading day (plus the trade_policy
+# distinct-day guard for manual re-scans).
 _DAILY_PIPELINE_STEPS = [
     ("data_collector", "collect_ohlcv"),
     ("benchmark_collector", "collect_benchmark"),
     ("sentiment_collector", "collect_sentiment"),
-    ("rr_scanner", "scan_rr"),
-    ("outcome_evaluator", "evaluate_outcomes"),
     ("market_regime", "compute_market_regime"),
-    # Observational only — runs here for scheduling; its output feeds nothing else.
+    # Observational only — display/alerts; not trade selection.
     ("regime_monitor", "compute_regime_monitor"),
+    # Alerts after regime so quadrant changes reach Telegram in the morning.
+    # Dispatcher is change-driven; quiet days stay quiet. Setup alerts still
+    # fire on the near-close pipeline after the qualifying scan.
+    ("alerts", "dispatch_alerts_job"),
+]
+
+# Near-close (~15:30 ET Mon–Fri): refresh in-progress day-t bars (already how
+# the intraday pipeline keeps the dashboard live), then the only daily
+# qualifying R:R scan, then Telegram immediately so manual fills can still hit
+# MOC cutoffs (~15:50/15:55). Under a 15-minute delayed SIP feed a 15:30 scan
+# may see ~15:15 prices — immaterial for a 12-1 momentum signal.
+#
+# US early-close days (~3/year, 13:00 ET close): this job runs post-close and
+# entries behave like stale_close (still acceptable per execution-recovery matrix).
+# No exchange calendar dependency.
+_NEAR_CLOSE_PIPELINE_STEPS = [
+    ("data_collector", "collect_ohlcv"),
+    ("rr_scanner", "scan_rr"),
+    ("alerts", "dispatch_alerts_job"),
+]
+
+# After close (~16:45 ET Mon–Fri): fresh OHLCV fetch so outcomes resolve on the
+# final bar, not the near-close partial bar, then outcome/paper close.
+_AFTER_CLOSE_PIPELINE_STEPS = [
+    ("data_collector", "collect_ohlcv"),
+    ("outcome_evaluator", "evaluate_outcomes"),
 ]
 
 # Intraday (light): keep prices current and resolve outcomes through the day,
@@ -1197,6 +1227,10 @@ _INTRADAY_PIPELINE_STEPS = [
     ("outcome_evaluator", "evaluate_outcomes"),
 ]
 
+# Warn if near-close fetch+scan+alert drifts past this — entries leave the close
+# and the stale_close floor quietly becomes the ceiling.
+_NEAR_CLOSE_DURATION_WARN_SECONDS = 600
+
 
 async def _run_pipeline(job_name: str, steps: list[tuple[str, str]]) -> None:
     """Run an ordered list of (step_name, coroutine_name) steps.
@@ -1232,11 +1266,44 @@ async def _run_pipeline(job_name: str, steps: list[tuple[str, str]]) -> None:
 
 
 async def run_daily_pipeline() -> None:
-    """Full daily flow: OHLCV → benchmark → sentiment → R:R scan → outcome eval
-    (+paper close) → market regime."""
+    """Morning flow: OHLCV → benchmark → sentiment → market regime (no scan)."""
     await _run_pipeline("daily_pipeline", _DAILY_PIPELINE_STEPS)
 
 
+async def run_near_close_pipeline() -> None:
+    """Near-close flow: OHLCV fetch → R:R scan → Telegram alerts.
+
+    Logs wall duration; warn if past 10 minutes so operators notice close drift.
+    """
+    import time
+
+    started = time.monotonic()
+    await _run_pipeline("near_close_pipeline", _NEAR_CLOSE_PIPELINE_STEPS)
+    elapsed = time.monotonic() - started
+    payload = {
+        "job": "near_close_pipeline",
+        "duration_seconds": round(elapsed, 1),
+    }
+    if elapsed > _NEAR_CLOSE_DURATION_WARN_SECONDS:
+        _log_event(
+            logging.WARNING,
+            "near_close_pipeline_slow",
+            **payload,
+            threshold_seconds=_NEAR_CLOSE_DURATION_WARN_SECONDS,
+            message=(
+                "Near-close pipeline exceeded 10 minutes — entries may drift from "
+                "the close toward the stale_close research floor"
+            ),
+        )
+    else:
+        _log_event(logging.INFO, "near_close_pipeline_duration", **payload)
+
+
+async def run_after_close_pipeline() -> None:
+    """After-close flow: OHLCV fetch (final bar) → outcome eval (+paper close)."""
+    await _run_pipeline("after_close_pipeline", _AFTER_CLOSE_PIPELINE_STEPS)
+
+
 async def run_intraday_pipeline() -> None:
     """Light intraday flow: refresh OHLCV → evaluate outcomes (+paper close)."""
     await _run_pipeline("intraday_pipeline", _INTRADAY_PIPELINE_STEPS)
@@ -1268,16 +1335,28 @@ def _parse_frequency(freq: str) -> dict[str, int]:
 # every process restart, so on a box that's redeployed often it can keep being
 # deferred and never fire. Cron fires at a fixed local time regardless.
 
+# All wall times are America/New_York after the near-close execution cutover.
+# Stored SystemSetting values shadow these defaults — deploy migration 023
+# rewrites schedule_* keys so prod does not keep scanning at 07:00 Berlin.
 SCHEDULE_DEFAULTS: dict[str, str] = {
-    "schedule_timezone": "Europe/Berlin",
-    "schedule_daily_pipeline_cron": "0 7 * * *",        # full refresh, ready by ~8am
-    "schedule_intraday_pipeline_cron": "0 14-22 * * 1-5",  # hourly across the US session
-    "schedule_fundamentals_cron": "0 4 * * 1",          # weekly, early Monday (slow job)
+    "schedule_timezone": "America/New_York",
+    # Morning data/display refresh (no qualifying R:R scan).
+    "schedule_daily_pipeline_cron": "0 2 * * *",
+    # Fetch in-progress bars → scan → Telegram (manual MOC window).
+    "schedule_near_close_pipeline_cron": "30 15 * * 1-5",
+    # Fetch final bars → outcome eval (must not run on the partial near-close bar).
+    "schedule_after_close_pipeline_cron": "45 16 * * 1-5",
+    # Hourly mid-session price + outcome (10:00–15:00 ET Mon–Fri).
+    "schedule_intraday_pipeline_cron": "0 10-15 * * 1-5",
+    # Weekly fundamentals early Monday NY.
+    "schedule_fundamentals_cron": "0 1 * * 1",
 }
 
 # job id -> schedule setting key
 _CRON_JOBS: dict[str, str] = {
     "daily_pipeline": "schedule_daily_pipeline_cron",
+    "near_close_pipeline": "schedule_near_close_pipeline_cron",
+    "after_close_pipeline": "schedule_after_close_pipeline_cron",
     "intraday_pipeline": "schedule_intraday_pipeline_cron",
     "fundamental_collector": "schedule_fundamentals_cron",
 }
@@ -1357,7 +1436,29 @@ def configure_scheduler(schedule_config: dict[str, str] | None = None) -> None:
     scheduler.add_job(
         run_daily_pipeline,
         _cron_trigger(cfg["schedule_daily_pipeline_cron"], tz, "schedule_daily_pipeline_cron"),
-        id="daily_pipeline", name="Daily Pipeline", replace_existing=True,
+        id="daily_pipeline", name="Morning Pipeline", replace_existing=True,
+    )
+    scheduler.add_job(
+        run_near_close_pipeline,
+        _cron_trigger(
+            cfg["schedule_near_close_pipeline_cron"],
+            tz,
+            "schedule_near_close_pipeline_cron",
+        ),
+        id="near_close_pipeline",
+        name="Near-Close Pipeline (scan+alert)",
+        replace_existing=True,
+    )
+    scheduler.add_job(
+        run_after_close_pipeline,
+        _cron_trigger(
+            cfg["schedule_after_close_pipeline_cron"],
+            tz,
+            "schedule_after_close_pipeline_cron",
+        ),
+        id="after_close_pipeline",
+        name="After-Close Pipeline (outcome)",
+        replace_existing=True,
     )
     scheduler.add_job(
         run_intraday_pipeline,
@@ -1377,10 +1478,12 @@ def configure_scheduler(schedule_config: dict[str, str] | None = None) -> None:
         sync_ticker_universe, "interval", hours=24,
         id="ticker_universe_sync", name="Ticker Universe Sync", replace_existing=True,
     )
-    alerts_interval = _parse_frequency(settings.alerts_frequency)
+    # Alerts auto-fire only via near_close_pipeline (scan → alert before MOC).
+    # Keep the job registered for Admin manual trigger; no independent interval.
     scheduler.add_job(
-        dispatch_alerts_job, "interval", **alerts_interval,
-        id="alerts", name="Alerts Dispatcher", replace_existing=True,
+        dispatch_alerts_job, "interval", weeks=520,
+        id="alerts", name="Alerts Dispatcher",
+        replace_existing=True, next_run_time=None,
     )
     scheduler.add_job(
         run_backtest_job, "interval", hours=168,
@@ -1400,10 +1503,27 @@ def configure_scheduler(schedule_config: dict[str, str] | None = None) -> None:
         replace_existing=True, next_run_time=None,
     )
 
-    _log_event(logging.INFO, "scheduler_configured", timezone=tz, daily_pipeline={
-                "cron": cfg["schedule_daily_pipeline_cron"],
-                "steps": [name for name, _ in _DAILY_PIPELINE_STEPS],
-            }, intraday_pipeline={
-                "cron": cfg["schedule_intraday_pipeline_cron"],
-                "steps": [name for name, _ in _INTRADAY_PIPELINE_STEPS],
-            }, fundamental_collector={"cron": cfg["schedule_fundamentals_cron"]}, independent=["ticker_universe_sync", "alerts", "backtest"])
+    _log_event(
+        logging.INFO,
+        "scheduler_configured",
+        timezone=tz,
+        daily_pipeline={
+            "cron": cfg["schedule_daily_pipeline_cron"],
+            "steps": [name for name, _ in _DAILY_PIPELINE_STEPS],
+        },
+        near_close_pipeline={
+            "cron": cfg["schedule_near_close_pipeline_cron"],
+            "steps": [name for name, _ in _NEAR_CLOSE_PIPELINE_STEPS],
+        },
+        after_close_pipeline={
+            "cron": cfg["schedule_after_close_pipeline_cron"],
+            "steps": [name for name, _ in _AFTER_CLOSE_PIPELINE_STEPS],
+        },
+        intraday_pipeline={
+            "cron": cfg["schedule_intraday_pipeline_cron"],
+            "steps": [name for name, _ in _INTRADAY_PIPELINE_STEPS],
+        },
+        fundamental_collector={"cron": cfg["schedule_fundamentals_cron"]},
+        independent=["ticker_universe_sync", "backtest"],
+        manual_only=["alerts", "data_backfill", "event_study"],
+    )

app/schemas/admin.py

@@ -75,9 +75,11 @@ class ActivationConfigUpdate(BaseModel):
 
 class ScheduleConfigUpdate(BaseModel):
     """Cron schedule for the pipelines + fundamentals. Crons are 5-field
-    (min hour dom month dow); timezone is an IANA name (e.g. Europe/Berlin)."""
+    (min hour dom month dow); timezone is an IANA name (e.g. America/New_York)."""
     schedule_timezone: str | None = Field(default=None, max_length=64)
     schedule_daily_pipeline_cron: str | None = Field(default=None, max_length=120)
+    schedule_near_close_pipeline_cron: str | None = Field(default=None, max_length=120)
+    schedule_after_close_pipeline_cron: str | None = Field(default=None, max_length=120)
     schedule_intraday_pipeline_cron: str | None = Field(default=None, max_length=120)
     schedule_fundamentals_cron: str | None = Field(default=None, max_length=120)
 

app/schemas/paper_trade.py

@@ -47,6 +47,8 @@ class PaperTradeResponse(BaseModel):
     alpha_pct: float | None = None
     alpha_usd: float | None = None
     close_reason: str | None = None
+    # Execution era: null = pre-cutover / unknown; "near_close" = post schedule cutover.
+    fill_mode: str | None = None
     # Live trailing-stop level + how far price sits above it (% ), for open trades
     # when the trailing exit policy is active.
     trailing_stop: float | None = None

app/services/admin_service.py

@@ -566,6 +566,8 @@ VALID_JOB_NAMES = {
     "event_study",
     "backtest",
     "daily_pipeline",
+    "near_close_pipeline",
+    "after_close_pipeline",
     "intraday_pipeline",
 }
 
@@ -583,17 +585,20 @@ JOB_LABELS = {
     "regime_monitor": "Regime Monitor",
     "event_study": "Event Study",
     "backtest": "Backtest",
-    "daily_pipeline": "Daily Pipeline",
+    "daily_pipeline": "Morning Pipeline",
+    "near_close_pipeline": "Near-Close Pipeline (scan+alert)",
+    "after_close_pipeline": "After-Close Pipeline (outcome)",
     "intraday_pipeline": "Intraday Pipeline",
 }
 
-# Jobs driven by the daily_pipeline (in order) rather than their own timer.
+# Jobs driven by a pipeline (in order) rather than their own auto timer.
 PIPELINE_MEMBERS = {
     "data_collector",
     "benchmark_collector",
     "sentiment_collector",
     "rr_scanner",
     "outcome_evaluator",
+    "alerts",
     "market_regime",
     "regime_monitor",
 }

app/services/backtest_service.py

@@ -688,6 +688,24 @@ TIME_EXIT_DAYS = (5, 10, 21, 30)
 # round trip, converted into R via the setup's stop distance (the 1R unit).
 COST_PER_SIDE = 0.001
 
+# Portfolio vol-targeting defaults (Barroso & Santa-Clara style, equity-curve vol).
+VOL_TARGET_LOOKBACK_HEADLINE = 60
+VOL_TARGET_LOOKBACK_SENSITIVITY = (20, 126)
+VOL_TARGET_GRID = (0.15, 0.20, 0.25)
+VOL_TARGET_CLAMP_HEADLINE = (0.5, 1.5)
+VOL_TARGET_CLAMP_WIDE = (0.25, 2.0)
+
+# Entry fill modes for the capital-constrained book simulator.
+# close: signal and fill at the same bar's close (historical optimistic control).
+# next_open: signal at t close, fill at t+1 open (honest for an overnight scanner).
+# stale_close: signal at t−1 close, fill at t close (near-close / MOC-style execution
+# with a one-session-stale signal — the recovery hypothesis for the next_open gap).
+FILL_MODE_CLOSE = "close"
+FILL_MODE_NEXT_OPEN = "next_open"
+FILL_MODE_STALE_CLOSE = "stale_close"
+FILL_MODES = (FILL_MODE_CLOSE, FILL_MODE_NEXT_OPEN, FILL_MODE_STALE_CLOSE)
+DELAYED_FILL_MODES = (FILL_MODE_NEXT_OPEN, FILL_MODE_STALE_CLOSE)
+
 
 def _cost_r(cand: dict) -> float:
     """Round-trip transaction cost in R units: two sides over the 1R stop
@@ -800,6 +818,42 @@ def _realized_vol_6m(closes: list[float], i: int) -> float | None:
     return compute_realized_vol_6m(closes[: i + 1])
 
 
+def _fip_id(closes: list[float], i: int) -> float | None:
+    """Da/Gurun/Warachka information discreteness over the 12-1 formation window.
+
+    Formation matches ``mom_12_1``: cumulative return from close[i-252] to
+    close[i-21] (231 daily returns ending one month before as-of).
+
+        ID = sign(PRET) × (%neg − %pos)
+
+    where %pos / %neg are fractions of up / down days over the formation window
+    (zero-return days count in neither numerator, but remain in the denominator).
+    Lower ID = smoother / more continuous path → expect negative cross-sectional
+    IC (continuous-information winners outperform).
+    """
+    if i - 252 < 0 or closes[i - 252] <= 0 or closes[i - 21] <= 0:
+        return None
+    pret = closes[i - 21] / closes[i - 252] - 1.0
+    rets: list[float] = []
+    for k in range(i - 251, i - 20):
+        prev = closes[k - 1]
+        if prev <= 0:
+            return None
+        rets.append(closes[k] / prev - 1.0)
+    if len(rets) < 200:
+        return None
+    n = len(rets)
+    pct_pos = sum(1 for r in rets if r > 0) / n
+    pct_neg = sum(1 for r in rets if r < 0) / n
+    if pret > 0:
+        sign = 1.0
+    elif pret < 0:
+        sign = -1.0
+    else:
+        sign = 0.0
+    return sign * (pct_neg - pct_pos)
+
+
 def _signal_values(
     dates: list[date],
     closes: list[float],
@@ -816,6 +870,7 @@ def _signal_values(
     is closeness to the trailing 52-week high (George/Hwang anchoring effect:
     higher = nearer the high, expect positive IC). ``vol_6m`` is 126-day realized
     volatility (expect negative IC if the low-volatility anomaly holds).
+    ``fip_id`` is Da/Gurun/Warachka information discreteness (expect negative IC).
     """
     out: dict[str, float] = {}
     if i - 252 >= 0 and closes[i - 252] > 0:
@@ -823,6 +878,9 @@ def _signal_values(
         residual = _residual_momentum_12_1(dates, closes, i, benchmark_closes)
         if residual is not None:
             out["mom_12_1_resid"] = residual
+        fip = _fip_id(closes, i)
+        if fip is not None:
+            out["fip_id"] = fip
     if i - 126 >= 0 and closes[i - 126] > 0:
         out["mom_6_1"] = closes[i - 21] / closes[i - 126] - 1.0
     if i - 63 >= 0 and closes[i - 63] > 0:
@@ -1427,6 +1485,221 @@ SIM_STARTING_CAPITAL = 10_000.0
 SIM_MAX_POSITIONS = 10
 SIM_RISK_PER_TRADE = 0.01  # fraction of equity risked per position (entry→stop)
 SIM_NOTIONAL_CAP = 0.20    # max fraction of equity per position (no margin)
+_EULER_MASCHERONI = 0.5772156649015329
+
+
+def _sample_moments(rets: list[float]) -> dict[str, float | int | None]:
+    """Mean / std / skew / kurtosis of a return series. Kurtosis is raw (not excess)."""
+    n = len(rets)
+    if n < 3:
+        return {
+            "n": n,
+            "mean": None,
+            "std": None,
+            "skew": None,
+            "kurtosis": None,
+        }
+    mean = sum(rets) / n
+    # Sample variance with n-1 (matches the historical Sharpe path).
+    var = sum((x - mean) ** 2 for x in rets) / (n - 1)
+    if var <= 0:
+        return {
+            "n": n,
+            "mean": mean,
+            "std": 0.0,
+            "skew": None,
+            "kurtosis": None,
+        }
+    std = math.sqrt(var)
+    m3 = sum((x - mean) ** 3 for x in rets) / n
+    m4 = sum((x - mean) ** 4 for x in rets) / n
+    skew = m3 / (std ** 3) if std > 0 else None
+    kurtosis = m4 / (std ** 4) if std > 0 else None
+    return {
+        "n": n,
+        "mean": mean,
+        "std": std,
+        "skew": skew,
+        "kurtosis": kurtosis,
+    }
+
+
+def _mertens_sharpe_se(
+    sharpe_periodic: float,
+    n: int,
+    skew: float | None,
+    kurtosis: float | None,
+) -> float | None:
+    """Mertens/Lo standard error of the non-annualized Sharpe ratio.
+
+    Accounts for non-normality via skew and kurtosis — material for this
+    right-skewed momentum book. Returns SE of the *periodic* Sharpe (mean/std);
+    annualize by multiplying by sqrt(252) alongside the point estimate.
+    """
+    if n < 3:
+        return None
+    g3 = 0.0 if skew is None else float(skew)
+    # Fall back to Gaussian kurtosis (=3) when undefined.
+    g4 = 3.0 if kurtosis is None else float(kurtosis)
+    sr = float(sharpe_periodic)
+    inside = 1.0 + 0.5 * sr * sr - g3 * sr + ((g4 - 3.0) / 4.0) * sr * sr
+    if inside <= 0:
+        return None
+    return math.sqrt(inside / (n - 1))
+
+
+def sharpe_diagnostics(rets: list[float], *, periods_per_year: float = 252.0) -> dict:
+    """Annualized Sharpe plus Mertens SE and PSR vs zero for a daily return series.
+
+    Additive report fields only — safe for the weekly production backtest and every
+    research matrix row. Deflated Sharpe is *not* included here: DSR needs a
+    pre-registered trial count N and is computed by ``deflated_sharpe_ratio``.
+    """
+    moments = _sample_moments(rets)
+    n = int(moments["n"] or 0)
+    mean = moments["mean"]
+    std = moments["std"]
+    skew = moments["skew"]
+    kurtosis = moments["kurtosis"]
+    empty = {
+        "sharpe": None,
+        "sharpe_se": None,
+        "psr": None,
+        "n_returns": n,
+        "return_skew": None if skew is None else round(float(skew), 4),
+        "return_kurtosis": None if kurtosis is None else round(float(kurtosis), 4),
+    }
+    if mean is None or std is None or std <= 0 or n < 3:
+        return empty
+    sr_p = float(mean) / float(std)
+    scale = math.sqrt(periods_per_year)
+    sharpe = sr_p * scale
+    se_p = _mertens_sharpe_se(sr_p, n, None if skew is None else float(skew),
+                              None if kurtosis is None else float(kurtosis))
+    se = se_p * scale if se_p is not None else None
+    psr = None
+    if se is not None and se > 0:
+        # PSR(SR*=0): Φ(sharpe / se) using the annualized numbers (scale cancels).
+        psr = statistics.NormalDist().cdf(sharpe / se)
+    return {
+        "sharpe": round(sharpe, 2),
+        "sharpe_se": round(se, 3) if se is not None else None,
+        "psr": round(psr, 4) if psr is not None else None,
+        "n_returns": n,
+        "return_skew": None if skew is None else round(float(skew), 4),
+        "return_kurtosis": None if kurtosis is None else round(float(kurtosis), 4),
+    }
+
+
+def deflated_sharpe_ratio(
+    sharpe: float | None,
+    sharpe_se: float | None,
+    n_trials: int,
+    *,
+    n_returns: int | None = None,
+    return_skew: float | None = None,
+    return_kurtosis: float | None = None,
+) -> float | None:
+    """Bailey & López de Prado Deflated Sharpe Ratio for a multi-arm matrix.
+
+    ``n_trials`` must be the *pre-registered* arm count for the matrix (not
+    invented after the fact). Returns None when inputs are insufficient — never
+    fabricates a DSR for a standalone single-arm run.
+    """
+    if (
+        sharpe is None
+        or sharpe_se is None
+        or sharpe_se <= 0
+        or n_trials < 2
+        or n_returns is None
+        or n_returns < 3
+    ):
+        return None
+    # Expected maximum Sharpe under the null across N independent trials
+    # (Bailey & López de Prado 2014), using Euler-Mascheroni blending of
+    # extreme-value quantiles. Variance under the null uses the observed
+    # higher moments evaluated at SR=0 → SE_null = 1/sqrt(n-1) * ann_scale,
+    # recovered from the reported annualized SE via the Mertens factor at the
+    # observed SR (we back out the periodic SE from sharpe_se).
+    nd = statistics.NormalDist()
+    # Annualized expected max of N zero-mean unit-variance Sharpes, then
+    # scaled by the null SE. With SR*=0 null variance of the *annualized*
+    # Sharpe is approximately (periods_per_year)/(n-1) when returns are IID
+    # normal; recover ann_scale^2/(n-1) from se under Gaussian assumption as
+    # a fallback, but prefer moment-adjusted null at SR=0:
+    # SE_null_periodic = sqrt(1/(n-1)); SE_null_ann = SE_null_p * (se/se_p).
+    # We don't store se_p, so invert: se_ann / sqrt(1+0.5 SR_p^2 - ...) * sqrt(1/(n-1))
+    # Simpler standard form used in practice:
+    # SR* = se_null * ((1-γ) Z^{-1}(1-1/N) + γ Z^{-1}(1-1/(N e)))
+    # with se_null = sharpe_se evaluated under null ≈ sqrt(periods/(n-1)).
+    # Approximate se_null from n_returns assuming daily data:
+    se_null = math.sqrt(252.0 / (n_returns - 1))
+    z1 = nd.inv_cdf(1.0 - 1.0 / n_trials)
+    z2 = nd.inv_cdf(1.0 - 1.0 / (n_trials * math.e))
+    sr_star = se_null * ((1.0 - _EULER_MASCHERONI) * z1 + _EULER_MASCHERONI * z2)
+    # PSR-style DSR using the observed (skew/kurt-adjusted) SE.
+    dsr = nd.cdf((float(sharpe) - sr_star) / float(sharpe_se))
+    return round(dsr, 4)
+
+
+def _equity_curve_realized_vol(
+    curve: list[tuple[int, float]], lookback: int
+) -> float | None:
+    """Annualized realized vol of the last ``lookback`` equity-curve daily returns."""
+    if lookback < 2 or len(curve) < lookback + 1:
+        return None
+    rets: list[float] = []
+    for i in range(len(curve) - lookback, len(curve)):
+        prev = curve[i - 1][1]
+        cur = curve[i][1]
+        if prev <= 0:
+            return None
+        rets.append(cur / prev - 1.0)
+    if len(rets) < lookback:
+        return None
+    mean = sum(rets) / len(rets)
+    var = sum((x - mean) ** 2 for x in rets) / (len(rets) - 1)
+    if var <= 0:
+        return None
+    return math.sqrt(var) * math.sqrt(252.0)
+
+
+def _clamp(value: float, lo: float, hi: float) -> float:
+    return max(lo, min(hi, value))
+
+
+def _daily_returns_ending_at(
+    closes: list[float], end_idx: int, lookback: int
+) -> list[float] | None:
+    """``lookback`` daily returns ending at ``end_idx`` (inclusive close)."""
+    if end_idx < lookback or end_idx >= len(closes):
+        return None
+    rets: list[float] = []
+    start = end_idx - lookback + 1
+    for k in range(start, end_idx + 1):
+        prev = closes[k - 1]
+        if prev <= 0 or closes[k] <= 0:
+            return None
+        rets.append(closes[k] / prev - 1.0)
+    return rets
+
+
+def _max_corr_vs_open(
+    candidate_rets: list[float],
+    open_rets: list[list[float]],
+) -> float | None:
+    """Max pairwise Pearson correlation of candidate vs each open-position series."""
+    if not open_rets:
+        return None
+    best: float | None = None
+    for other in open_rets:
+        if len(other) != len(candidate_rets):
+            continue
+        rho = _pearson(candidate_rets, other)
+        if rho is None:
+            continue
+        best = rho if best is None else max(best, rho)
+    return best
 # The "atr_trail3" research policy's trail width. Must equal the live default
 # (paper_trade_service.DEFAULT_ATR_MULTIPLIER) — enforced by the parity test.
 # The production portfolio-monitor row additionally follows the *runtime* Admin
@@ -1533,6 +1806,15 @@ def _simulate_portfolio(
     end_date: date | None = None,
     include_curve: bool = False,
     include_trades: bool = False,
+    fill_mode: str = FILL_MODE_CLOSE,
+    max_entry_gap_pct: float | None = None,
+    vol_target: float | None = None,
+    vol_lookback: int = VOL_TARGET_LOOKBACK_HEADLINE,
+    vol_clamp: tuple[float, float] = VOL_TARGET_CLAMP_HEADLINE,
+    corr_max: float | None = None,
+    corr_lookback: int = 120,
+    corr_action: str = "skip",
+    corr_min_overlap: int = 60,
 ) -> dict | None:
     """Replay the qualified setups as ONE capital-constrained book and report
     portfolio economics from the daily equity curve (return, CAGR, drawdown,
@@ -1552,13 +1834,39 @@ def _simulate_portfolio(
     stop when the active initial stop is touched; the replacement is still
     checked against the same bar. ``post_stop_reentry_fn`` turns an initial
     stop-out into a stateful episode and is the only path by which that ticker
-    can re-enter until the callback emits a new candidate. Returns None when
-    there is nothing to trade. ``cost_per_side`` is charged on entry and exit
-    and therefore changes both cash availability and subsequent position sizing.
+    can re-enter until the callback emits a new candidate.
+
+    ``fill_mode``: ``close`` enters at the signal-bar close with the candidate's
+    stop (historical control). ``next_open`` fills at the next session's open
+    with stop = fill − 1.5×ATR(signal bar); missing next bar skips the entry.
+    ``stale_close`` fills at the next session's *close* (one-session-stale
+    signal, MOC-style) with the same stop re-anchor. ``max_entry_gap_pct``
+    (next_open only) skips entries whose open gaps up more than that fraction
+    vs the signal close (e.g. 0.02 = +2%). Vol targeting scales
+    ``risk_per_trade`` at entry only from equity-curve realized vol. Correlation
+    caps skip or half-size candidates whose max pairwise 120d return correlation
+    with open holdings exceeds ``corr_max``.
+
+    Returns None when there is nothing to trade. ``cost_per_side`` is charged on
+    entry and exit and therefore changes both cash availability and subsequent
+    position sizing.
     """
     cost_rate = float(cost_per_side)
     if not 0.0 <= cost_rate < 1.0:
         raise ValueError("cost_per_side must be between 0 (inclusive) and 1")
+    if fill_mode not in FILL_MODES:
+        raise ValueError(f"fill_mode must be one of {FILL_MODES}")
+    if max_entry_gap_pct is not None and max_entry_gap_pct < 0:
+        raise ValueError("max_entry_gap_pct must be non-negative when set")
+    if max_entry_gap_pct is not None and fill_mode != FILL_MODE_NEXT_OPEN:
+        raise ValueError("max_entry_gap_pct only applies to fill_mode=next_open")
+    if corr_action not in ("skip", "half_size"):
+        raise ValueError("corr_action must be 'skip' or 'half_size'")
+    if vol_target is not None and vol_target <= 0:
+        raise ValueError("vol_target must be positive when set")
+    clamp_lo, clamp_hi = float(vol_clamp[0]), float(vol_clamp[1])
+    if clamp_lo <= 0 or clamp_hi < clamp_lo:
+        raise ValueError("vol_clamp must satisfy 0 < lo <= hi")
     if qualified_fn is None:
         def _default_qualified(c: dict) -> bool:
             return bool(c.get("qualified"))
@@ -1576,7 +1884,7 @@ def _simulate_portfolio(
         if start_ord is not None and entry_ord < start_ord:
             continue
         if end_ord is not None and entry_ord >= end_ord:
-            continue  # holdout: entries strictly before the split
+            continue  # holdout/validation: entries strictly before the split
         if not c.get("entry") or not c.get("stop"):
             continue
         entries_by_ord[entry_ord].append(c)
@@ -1593,17 +1901,16 @@ def _simulate_portfolio(
     if not calendar:
         return None
 
-    if end_ord is not None:
-        # Holdout train book: entries stop at the split, but the calendar would
-        # otherwise still run to the last bar in the data — leaving the book in
-        # flat cash for the whole test period and deflating CAGR/Sharpe into
-        # something that looks like a result and isn't. Every open position
-        # resolves within `hold_days` bars of the last entry, so cut there.
-        last_entry_ord = max(entries_by_ord)
-        cut = bisect.bisect_left(calendar, last_entry_ord) + hold_days + 1
-        calendar = calendar[:cut]
-        if not calendar:
-            return None
+    # Always truncate the calendar to last_signal + hold_days (+1 for delayed
+    # fill lag). Prevents trailing flat-cash after the last resolvable entry —
+    # the clear-air train-window bug — for train, validation, and full-period
+    # books alike (including max-hold sweeps out to 90 days).
+    last_signal_ord = max(entries_by_ord)
+    resolve_pad = hold_days + (1 if fill_mode in DELAYED_FILL_MODES else 0)
+    cut = bisect.bisect_left(calendar, last_signal_ord) + resolve_pad + 1
+    calendar = calendar[:cut]
+    if not calendar:
+        return None
 
     cash = SIM_STARTING_CAPITAL
     positions: dict[str, dict] = {}
@@ -1611,6 +1918,9 @@ def _simulate_portfolio(
     trades: list[dict] = []
     skipped_full = 0
     skipped_cooldown = 0
+    skipped_corr = 0
+    skipped_missing_fill = 0
+    skipped_gap_cap = 0
     cooldown_until_index: dict[str, int] = {}
     stop_refresh_attempts = 0
     stop_refreshes = 0
@@ -1620,6 +1930,9 @@ def _simulate_portfolio(
     reentry_events: list[dict] = []
     technical_cache: dict[tuple[str, int], float | None] = {}
     atr_cache: dict[tuple[str, int], float | None] = {}
+    vol_scalars: list[float] = []
+    overnight_slippage_pct: list[float] = []
+    pending_delayed: list[dict] = []
 
     def _bar(sym: str, o: int):
         idx = index_of.get(sym, {}).get(o)
@@ -1795,7 +2108,7 @@ def _simulate_portfolio(
                     if next_stop < bar.close:
                         pos["stop"] = max(pos["stop"], next_stop)
 
-        # 2) entries at today's close, best momentum first
+        # 2) entries — close-fill at signal close, or next-open fills of prior signals
         equity = _marked_equity()
         fixed_todays = list(entries_by_ord.get(o, ()))
         reentry_todays: list[dict] = []
@@ -1820,32 +2133,92 @@ def _simulate_portfolio(
                 tagged = dict(candidate)
                 tagged["_post_stop_reentry"] = True
                 reentry_todays.append(tagged)
-        todays = sorted(
+        signal_todays = sorted(
             fixed_todays + reentry_todays,
             key=lambda c: c.get(ranking_key) or 0.0,
             reverse=True,
         )
-        for c in todays:
+
+        if fill_mode in DELAYED_FILL_MODES:
+            fill_candidates = sorted(
+                pending_delayed,
+                key=lambda c: c.get(ranking_key) or 0.0,
+                reverse=True,
+            )
+            pending_delayed = []
+        else:
+            fill_candidates = signal_todays
+
+        def _corr_scale_for(sym: str, asof_idx: int) -> float | None:
+            """1.0 ok, 0.5 half-size, None = skip. Missing history → uncorrelated."""
+            if corr_max is None or not positions:
+                return 1.0
+            closes = prices[sym][4]
+            cand_rets = _daily_returns_ending_at(closes, asof_idx, corr_lookback)
+            if cand_rets is None or len(cand_rets) < corr_min_overlap:
+                return 1.0
+            open_series: list[list[float]] = []
+            for open_sym in positions:
+                open_idx = index_of.get(open_sym, {}).get(o)
+                if open_idx is None:
+                    continue
+                other = _daily_returns_ending_at(
+                    prices[open_sym][4], open_idx, corr_lookback
+                )
+                if other is None or len(other) < corr_min_overlap:
+                    continue
+                open_series.append(other)
+            if not open_series:
+                return 1.0
+            n = min(len(cand_rets), min(len(s) for s in open_series))
+            if n < corr_min_overlap:
+                return 1.0
+            rho = _max_corr_vs_open(
+                cand_rets[-n:], [s[-n:] for s in open_series]
+            )
+            if rho is None or rho <= corr_max:
+                return 1.0
+            if corr_action == "half_size":
+                return 0.5
+            return None
+
+        def _open_position(
+            c: dict,
+            *,
+            entry: float,
+            stop: float,
+            entry_ord: int,
+            signal_close: float | None,
+            corr_scale: float,
+            fill_bar: Any | None,
+        ) -> None:
+            nonlocal cash, equity, skipped_full, skipped_cooldown, post_stop_events
             sym = c["symbol"]
             if sym in positions:
-                continue
+                return
             if calendar_index < cooldown_until_index.get(sym, -1):
                 skipped_cooldown += 1
-                continue
+                return
             if len(positions) >= max_positions:
                 skipped_full += 1
-                continue
-            entry, stop = float(c["entry"]), float(c["stop"])
+                return
             risk_ps = entry - stop
             if risk_ps <= 0 or entry <= 0:
-                continue
+                return
+            scalar = 1.0
+            if vol_target is not None:
+                realized = _equity_curve_realized_vol(curve, int(vol_lookback))
+                if realized is not None and realized > 0:
+                    scalar = _clamp(float(vol_target) / realized, clamp_lo, clamp_hi)
+                vol_scalars.append(scalar)
+            effective_risk = float(risk_per_trade) * scalar * corr_scale
             shares = min(
-                (equity * risk_per_trade) / risk_ps,
+                (equity * effective_risk) / risk_ps,
                 (equity * SIM_NOTIONAL_CAP) / entry,
                 max(cash, 0.0) / (entry * (1.0 + cost_rate)),
             )
-            if shares * entry < 1.0:  # can't fund a meaningful position
-                continue
+            if shares * entry < 1.0:
+                return
             entry_cost = shares * entry * cost_rate
             cash -= shares * entry + entry_cost
             is_reentry = bool(c.get("_post_stop_reentry"))
@@ -1857,14 +2230,16 @@ def _simulate_portfolio(
                     reentry_events.append({
                         "symbol": sym,
                         "stop_ord": state["stop_ord"],
-                        "reentry_ord": o,
+                        "reentry_ord": entry_ord,
                         "wait_sessions": reentry_wait_sessions,
                         "reason": c.get("_reentry_reason"),
                     })
+            if signal_close is not None and signal_close > 0:
+                overnight_slippage_pct.append((entry / signal_close - 1.0) * 100.0)
             positions[sym] = {
                 "shares": shares,
                 "entry": entry,
-                "entry_ord": o,
+                "entry_ord": entry_ord,
                 "initial_stop": stop,
                 "stop": stop,
                 "target": float(c["target"]) if c.get("target") else None,
@@ -1878,9 +2253,110 @@ def _simulate_portfolio(
                 "stop_refreshes": 0,
                 "is_reentry": is_reentry,
                 "reentry_wait_sessions": reentry_wait_sessions,
+                "vol_scalar": scalar,
+                "corr_scale": corr_scale,
             }
+            # next_open only: fill is at the open, so the rest of the bar can stop out.
+            # stale_close fills at the close — same-day stop after entry does not apply.
+            # bars_held stays 0 on the fill day (matches close-fill cadence).
+            if fill_mode == FILL_MODE_NEXT_OPEN and fill_bar is not None:
+                positions[sym]["last_close"] = fill_bar.close
+                positions[sym]["highest_close"] = max(entry, fill_bar.close)
+                if fill_bar.low <= stop:
+                    fill = min(stop, fill_bar.open)
+                    closed = _close_trade(sym, fill, "stop")
+                    if cooldown_sessions:
+                        cooldown_until_index[sym] = calendar_index + cooldown_sessions
+                    if post_stop_reentry_fn is not None:
+                        post_stop_events += 1
+                        post_stop_states[sym] = {
+                            "stop_ord": o,
+                            "stop_calendar_index": calendar_index,
+                            "stop_day_high": float(fill_bar.high),
+                            "stop_day_low": float(fill_bar.low),
+                            "stop_day_close": float(fill_bar.close),
+                            "exit_fill": float(fill),
+                            "previous_entry": float(closed["entry"]),
+                            "previous_stop": float(closed["initial_stop"]),
+                            "previous_rank": closed["entry_rank"],
+                            "gate_went_unqualified": False,
+                        }
+                elif exit_policy in ("atr_trail3", "atr_trail3_target"):
+                    atr = _atr(sym, fill_bar.idx)
+                    if atr is not None:
+                        next_stop = (
+                            positions[sym]["highest_close"]
+                            - atr_trail_multiplier * atr
+                        )
+                        if next_stop < fill_bar.close:
+                            positions[sym]["stop"] = max(
+                                positions[sym]["stop"], next_stop
+                            )
             equity = _marked_equity()
 
+        for c in fill_candidates:
+            sym = c["symbol"]
+            if fill_mode == FILL_MODE_CLOSE:
+                entry, stop = float(c["entry"]), float(c["stop"])
+                signal_idx = index_of.get(sym, {}).get(o)
+                if signal_idx is None:
+                    corr_scale: float | None = 1.0
+                else:
+                    corr_scale = _corr_scale_for(sym, signal_idx)
+                if corr_scale is None:
+                    skipped_corr += 1
+                    continue
+                _open_position(
+                    c,
+                    entry=entry,
+                    stop=stop,
+                    entry_ord=o,
+                    signal_close=None,
+                    corr_scale=corr_scale,
+                    fill_bar=None,
+                )
+            else:
+                # Delayed fill: prior-day signal → today's open (next_open) or close
+                # (stale_close). Stop always re-anchored to fill − 1.5×ATR(signal).
+                signal_ord = date.fromisoformat(str(c["date"])).toordinal()
+                signal_idx = index_of.get(sym, {}).get(signal_ord)
+                fill_bar = _bar(sym, o)
+                if fill_bar is None or signal_idx is None:
+                    skipped_missing_fill += 1
+                    continue
+                atr = _atr(sym, signal_idx)
+                if atr is None or atr <= 0:
+                    skipped_missing_fill += 1
+                    continue
+                signal_close = float(prices[sym][4][signal_idx])
+                if fill_mode == FILL_MODE_NEXT_OPEN:
+                    entry = float(fill_bar.open)
+                    if max_entry_gap_pct is not None and signal_close > 0:
+                        gap = entry / signal_close - 1.0
+                        if gap > float(max_entry_gap_pct):
+                            skipped_gap_cap += 1
+                            continue
+                else:
+                    entry = float(fill_bar.close)
+                stop = entry - ATR_MULTIPLIER * atr
+                corr_scale = _corr_scale_for(sym, signal_idx)
+                if corr_scale is None:
+                    skipped_corr += 1
+                    continue
+                _open_position(
+                    c,
+                    entry=entry,
+                    stop=stop,
+                    entry_ord=o,
+                    signal_close=signal_close,
+                    corr_scale=corr_scale,
+                    fill_bar=fill_bar if fill_mode == FILL_MODE_NEXT_OPEN else None,
+                )
+
+        if fill_mode in DELAYED_FILL_MODES:
+            # Queue today's signals for the next session's fill.
+            pending_delayed.extend(signal_todays)
+
         curve.append((o, _marked_equity()))
 
     # Close whatever is still open at its last mark so final equity is realized.
@@ -1905,12 +2381,8 @@ def _simulate_portfolio(
             max_dd = max(max_dd, (peak - eq) / peak)
 
     rets = [b / a - 1.0 for (_, a), (_, b) in zip(curve, curve[1:]) if a > 0]
-    sharpe = None
-    if len(rets) > 2:
-        mean = sum(rets) / len(rets)
-        var = sum((x - mean) ** 2 for x in rets) / (len(rets) - 1)
-        if var > 0:
-            sharpe = mean / math.sqrt(var) * math.sqrt(252)
+    diag = sharpe_diagnostics(rets)
+    sharpe = diag["sharpe"]
 
     # Per-calendar-year returns off the equity curve — shows whether every year
     # contributed or one exceptional stretch carried the result.
@@ -1981,14 +2453,25 @@ def _simulate_portfolio(
                     "return_pct": round((close / base_spy - 1.0) * 100.0, 2),
                 })
 
+    max_dd_pct = max_dd * 100.0
+    calmar = None
+    if cagr_pct is not None and max_dd_pct > 0:
+        calmar = float(cagr_pct) / max_dd_pct
     result = {
         "starting_capital": SIM_STARTING_CAPITAL,
         "cost_per_side_pct": round(cost_rate * 100.0, 3),
+        "fill_mode": fill_mode,
         "final_equity": round(final_equity, 2),
         "total_return_pct": round(total_return_pct, 1),
         "cagr_pct": round(cagr_pct, 1) if cagr_pct is not None else None,
-        "max_drawdown_pct": round(max_dd * 100.0, 1),
-        "sharpe": round(sharpe, 2) if sharpe is not None else None,
+        "max_drawdown_pct": round(max_dd_pct, 1),
+        "calmar": round(calmar, 2) if calmar is not None else None,
+        "sharpe": sharpe,
+        "sharpe_se": diag["sharpe_se"],
+        "psr": diag["psr"],
+        "n_returns": diag["n_returns"],
+        "return_skew": diag["return_skew"],
+        "return_kurtosis": diag["return_kurtosis"],
         "trades": len(trades),
         "win_rate": round(wins / len(trades) * 100.0, 1) if trades else None,
         "avg_trade_pnl": round(sum(pnls) / len(pnls), 2) if pnls else None,
@@ -2006,6 +2489,42 @@ def _simulate_portfolio(
         "start_date": date.fromordinal(calendar[0]).isoformat(),
         "end_date": date.fromordinal(calendar[-1]).isoformat(),
     }
+    if vol_target is not None:
+        result["vol_target"] = vol_target
+        result["vol_lookback"] = int(vol_lookback)
+        result["vol_clamp"] = [clamp_lo, clamp_hi]
+        result["avg_vol_scalar"] = (
+            round(sum(vol_scalars) / len(vol_scalars), 4) if vol_scalars else None
+        )
+        result["vol_scalar_entries"] = len(vol_scalars)
+    if corr_max is not None:
+        result["corr_max"] = corr_max
+        result["corr_action"] = corr_action
+        result["corr_lookback"] = corr_lookback
+        result["skipped_corr"] = skipped_corr
+    if fill_mode in DELAYED_FILL_MODES:
+        result["skipped_missing_fill"] = skipped_missing_fill
+        if overnight_slippage_pct:
+            slip = sorted(overnight_slippage_pct)
+            mid = len(slip) // 2
+            slip_payload = {
+                "n": len(slip),
+                "mean_pct": round(sum(slip) / len(slip), 4),
+                "median_pct": round(
+                    slip[mid] if len(slip) % 2 == 1 else (slip[mid - 1] + slip[mid]) / 2.0,
+                    4,
+                ),
+                "p05_pct": round(slip[max(0, int(0.05 * (len(slip) - 1)))], 4),
+                "p95_pct": round(slip[min(len(slip) - 1, int(0.95 * (len(slip) - 1)))], 4),
+            }
+            # next_open: true overnight gap; stale_close: one full session of drift.
+            if fill_mode == FILL_MODE_NEXT_OPEN:
+                result["overnight_slippage"] = slip_payload
+            else:
+                result["signal_to_fill_drift"] = slip_payload
+    if max_entry_gap_pct is not None:
+        result["max_entry_gap_pct"] = max_entry_gap_pct
+        result["skipped_gap_cap"] = skipped_gap_cap
     if curve_payload is not None:
         result["equity_curve"] = curve_payload
     if benchmark_payload is not None:

app/services/paper_trade_service.py

@@ -333,6 +333,9 @@ async def create_trade(
         target=target,
         status="open",
         opened_at=datetime.now(timezone.utc),
+        # Near-close cutover era — Track Record must not mix with morning-scan
+        # fills or future broker-routed fills when comparing to backtests.
+        fill_mode="near_close",
     )
     db.add(trade)
     await db.commit()
@@ -386,6 +389,7 @@ def _to_dict(
         "alpha_pct": alpha_pct,
         "alpha_usd": alpha_usd,
         "close_reason": trade.close_reason,
+        "fill_mode": trade.fill_mode,
         "trailing_stop": trailing[0] if trailing else None,
         "trailing_distance_pct": trailing[1] if trailing else None,
     }

app/services/trade_policy.py

@@ -3,13 +3,24 @@
 from __future__ import annotations
 
 from collections.abc import Iterable
-from datetime import datetime, timezone
+from datetime import date, datetime, timezone
+from zoneinfo import ZoneInfo
 
 from sqlalchemy import func, select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.paper_trade import PaperTrade
 
+# Gate-reset "day" boundary matches US cash equities session calendar, not UTC.
+_REENTRY_DAY_TZ = ZoneInfo("America/New_York")
+
+
+def _ny_trading_date(moment: datetime) -> date:
+    """Calendar date in America/New_York for a gate-reset observation."""
+    if moment.tzinfo is None:
+        moment = moment.replace(tzinfo=timezone.utc)
+    return moment.astimezone(_REENTRY_DAY_TZ).date()
+
 
 async def _latest_initial_stop_trades(
     db: AsyncSession,
@@ -93,8 +104,14 @@ async def observe_reentry_gate_transitions(
                 trade.reentry_gate_failed_at = timestamp
                 updated.add(ticker_id)
         elif ticker_id in qualified:
-            trade.reentry_gate_requalified_at = timestamp
-            updated.add(ticker_id)
+            # Study semantics: requalify only on a *subsequent* daily observation.
+            # Same America/New_York calendar day as the failure does not unlock,
+            # even if multiple full-universe scans run (manual + near-close).
+            if _ny_trading_date(trade.reentry_gate_failed_at) < _ny_trading_date(
+                timestamp
+            ):
+                trade.reentry_gate_requalified_at = timestamp
+                updated.add(ticker_id)
 
     if updated:
         await db.flush()

docs/research/README.md

@@ -105,22 +105,55 @@ and it would also sever the last dependency the *gate* has on the weak S/R detec
 
 ---
 
-## 4. Open leads
+## 4. Phase A matrix (2026-07-18) — closed
+
+Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** ·
+`reports/research-matrix-phase-a.json`.
+
+| Arm | Decision |
+|---|---|
+| Max-hold {45,60,90} | **Note and move on** — validation glitter, train collapse (regime interaction) |
+| Equity-curve vol targeting | **Reject as edge** on this sample; park vt25 as optional DD insurance only |
+| Correlation caps | **Reject**; sector caps stay Phase B with reduced expectations |
+| Next-open fill | **Discovery, not reject** — honest deployable ~Sharpe 1.2 / CAGR 30% under overnight scanner. Decision baseline until near-close ships = `next_open` |
+| `fip_id` re-derive | **Validated** (IC −0.045, t = −2.92) |
+
+### Execution recovery (same day) — closed as evidence
+
+Full write-up: **[execution-recovery.md](execution-recovery.md)** ·
+`reports/execution_recovery_matrix.json`.
+
+| Finding | Decision |
+|---|---|
+| Monotone fill timing (next_open → stale → close) + DD recovery | **When you fill**, not decaying alpha |
+| Live bracket **[1.57, 1.77]** full Sharpe | Near-close expected near top of bracket; no more fill-timing sim |
+| Gap-cap | **Dead** — third tail-trim instance |
+| Auto-`recover: false` | Not a null — bar hit the lower-bound arm by 0.03 train SE |
+
+**Highest-leverage open work:** **ops** — move the single daily R:R scan to
+`America/New_York` near-close (checklist in execution-recovery.md). Not more research
+knobs.
+
+---
+
+## 5. Open leads
 
 | Lead | Why it's interesting | Blocker |
 |---|---|---|
-| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign | Doesn't improve *this* book (the momentum gate already captures it in-sample). Revisit when the universe broadens |
-| **Broader universe** (`nasdaq_all`) | Strengthens every week's cross-section and the IC t-stat | Also where `fip_id` could become tradeable |
-| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time |
+| **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Implement schedule + partial-bar scan path; one qualifying scan/day only |
+| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign; re-derived fingerprint matched Phase A | Doesn't improve *this* book. Revisit when the universe broadens — **after** execution path is decided |
+| **Broader universe** (`nasdaq_all`) | Strengthens every week's cross-section and the IC t-stat | Grade under the fill mode you will trade |
+| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
 | **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
 
 ---
 
-## 5. Method rules learned the hard way
+## 6. Method rules learned the hard way
 
 1. **Nested lookback windows are NOT out-of-sample.** The clear-air result (#2) was
    clean, large, and consistent across five nested windows — and still died on a
-   proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT`.
+   proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT` / a named
+   validation window — and do not pretend a repeatedly opened window is pristine.
 2. **Check what population an ablation actually admits.** The blanket fallback (#3)
    looked like it tested the "resistance famine" hypothesis. It didn't — 65% of the
    setups it let in were a different population entirely, and they drove the result.
@@ -130,10 +163,16 @@ and it would also sever the last dependency the *gate* has on the weak S/R detec
 4. **The iron rule:** a signal earns its way into selection *only* through the
    factor harness — |mean IC| ≳ 0.03, consistent sign, `reliable: true` (≥ 12
    non-overlapping windows). Never let an unvalidated score gate setups.
+5. **Momentum filters are guilty of tail-trimming until proven otherwise.**
+   Independent failures: take-profit exits, FIP as an in-book filter, gap-up entry
+   caps. Cosmetic quality up, P&L down — the right tail *is* the edge.
+6. **Fill timing is part of the strategy.** Close-fill reports are not deployable
+   numbers for an overnight scanner. Grade promotion under the fill mode you will
+   actually trade.
 
 ---
 
-## 6. Why we stay with the current strategy
+## 7. Why we stay with the current strategy
 
 Everything we've tried to add has either failed the backtest, failed
 out-of-sample, or turned out to be measuring something other than what it claimed.

docs/research/execution-recovery.md

@@ -0,0 +1,192 @@
+# Execution recovery (2026-07-18) — when you fill is the edge you leave on the table
+
+Report: `reports/execution_recovery_matrix.json` / `.md`  
+Follows Phase A A4 ([phase-a-matrix.md](phase-a-matrix.md)).  
+Arms: `close_control` · `next_open` · `stale_close` · `next_open_gap2`.
+
+Mechanics verified before sign-off: `stale_close` re-anchors stop to
+fill − 1.5×ATR(signal day); no same-day stop after a close fill; each t−1
+candidate keeps its own gate/rank cross-section (no lookahead). Numbers trusted.
+
+---
+
+## Results
+
+| Arm | Full Sharpe | Full CAGR | Full DD | Train Sharpe | Val Sharpe |
+|---|---:|---:|---:|---:|---:|
+| **close_control** | **1.77** | **48.3%** | **21.6%** | 1.75 | 1.68 |
+| **stale_close** | **1.57** | **40.6%** | **21.8%** | 1.36 | **1.74** |
+| next_open | 1.20 | 30.0% | 28.2% | 0.94 | 1.44 |
+| next_open_gap2 | 1.06 | 25.2% | 27.8% | 0.72 | 1.41 |
+
+### Monotonicity (the strongest evidence)
+
+In every window, Sharpe recovers as the fill moves **toward** the signal:
+
+| Window | next_open → stale_close → close |
+|---|---|
+| Train | 0.94 → 1.36 → 1.75 |
+| Validation | 1.44 → 1.74 → 1.68* |
+| Full | 1.20 → 1.57 → 1.77 |
+
+\*Val close 1.68 is within SE of stale 1.74 — not a break of the story.
+
+A dead edge does **not** produce a monotone gradient in fill timing. A live edge
+that is progressively surrendered to execution delay does. Combined with full-period
+**DD recovery** (28.2% → 21.8% ≈ close’s 21.6%), this confirms the diagnosis:
+**when you fill**, not decaying alpha.
+
+### Auto-flag `recover: False` is not a null
+
+Pre-registered recovery applied the “≥ close − 0.5 SE” bar to **`stale_close`**,
+the **lower-bound** arm (one full session of lag). Flags:
+
+| Flag | Result |
+|---|---|
+| near_close_control (val) | **True** (Δ +0.06) |
+| beats_next_open | **True** (val Δ +0.30) |
+| train_ok | **False** (1.36 vs need ≥ ~1.39 — miss by ~0.03) |
+
+The floor missed “≥ close − 0.5 SE” by 0.03 while the live design is expected to
+sit **above** the floor. The flag worked correctly on the wrong object.
+
+**Log sentence:** *Partial recovery proven; full recovery needs same-day fill.*
+
+### Live outcome bracket
+
+| Bound | Arm | Meaning |
+|---|---|---|
+| Floor | stale_close ~1.57 full | One-session-stale close fill (conservative) |
+| Ceiling | close_control ~1.77 full | Same-day close; optimistic only by final ~15 min of signal info |
+
+Real near-close execution (scan ~15:30–15:40 ET on a ~99% complete bar, MOC by
+15:50/15:55) is signal-at-partial-bar filled at the same close the control uses.
+**Live truth is bracketed [1.57, 1.77]** with residual uncertainty of ~15 minutes
+of staleness, not 24 hours — expect near the **top** of the bracket.
+
+`stale_close` alone already justifies the schedule change. **No further fill-timing
+simulation on this snapshot** — the bracket is the result. You cannot simulate
+15:45 partial bars from daily data, and you do not need to.
+
+### Gap-cap — dead (third tail-trim instance)
+
+`next_open_gap2` worse than plain next_open on every window (full Sharpe 1.06 vs
+1.20). **262** full-period gap-ups skipped — they were continuations.
+
+This is the **third independent instance** of the same lesson:
+
+1. Take-profit exits (gate target as TP)  
+2. FIP as an in-book filter  
+3. **Gap-up entry caps**
+
+Any rule that trims the right tail improves cosmetic quality metrics and destroys
+P&L. **Standing method rule:** momentum filters must be presumed guilty of
+tail-trimming until shown otherwise.
+
+---
+
+## Decisions locked
+
+1. **Biggest lever is execution scheduling**, not a strategy rewrite.  
+2. **Until near-close execution is live:** grade strategy promotion under
+   `fill_mode=next_open`; keep close-fill as historical control.  
+3. **After near-close ships:** grade under a close-like / near-close fill mode
+   (actual live path).  
+4. **No more sim arms** on fill timing for this snapshot.  
+5. **Gap-cap:** do not ship.  
+6. **Strategy work** (nasdaq_all, fip_id, sector) waits until the execution path
+   is decided — those experiments must be graded under the fill mode you will trade.
+
+---
+
+## Ops design — implementation plan (code-checked)
+
+Assumptions verified against current code before ship:
+
+- Intraday pipeline already fetches/upserts the **in-progress day-t bar** all
+  session (`fetch_ohlcv` end_date defaults to today). Near-close job =
+  **OHLCV fetch → R:R scan** (no new snapshot synthesizer).
+- One global `schedule_timezone` (default `Europe/Berlin`); stored
+  `SystemSetting` values shadow code defaults — **defaults alone do not
+  migrate prod**.
+- `observe_reentry_gate_transitions` stamps timestamps with **no same-day
+  guard** today — dual scans would accelerate fail→requalify unless fixed in
+  `trade_policy`.
+
+### Semantic guard (ship step 1 — precondition)
+
+In `trade_policy` (not the scheduler):
+
+> `reentry_gate_requalified_at` may only be set when `reentry_gate_failed_at`
+> falls on an **earlier America/New_York trading date** than the current
+> observation.
+
+Manual mid-day scans stay allowed; same-day fail+qualify cannot unlock.
+Unit test: fail 10:00 / qualify 15:35 same day → still locked; qualify next day → unlocked.
+
+### Schedule split
+
+| Slot (America/New_York) | Jobs |
+|---|---|
+| Morning (~02:00) | OHLCV backfill, benchmark, sentiment, fundamentals — **no** qualifying R:R scan |
+| Near-close (~15:30 Mon–Fri) | OHLCV fetch (refresh day-t bar) → **R:R scan** (only daily qualifying observation) |
+| After close (~16:30–17:00) | **Outcome eval** on its own slot (not chained to the partial-bar scan) |
+| Intraday hourly | Unchanged in NY terms; last ~16:00 still mid-session under 15m feed |
+
+- Near-close scan **1–5 only**; US-holiday no-ops are fine (stale identical data
+  can’t flip gates) — comment only, no exchange calendar.
+- **Do not** run morning + near-close qualifying scans; move the scan, don’t add a second.
+
+### Behavior change to document (not an accident)
+
+With scan at ~15:35 ET, stops closed by **earlier same-day** intraday outcome
+evals can get a **same-day fail observation** — closer to the **promoted**
+`gate_reset` arm (stop-day close may establish failure) than today’s
+`strict_gate_reset` analogue (scan always before any eval). Stops after the
+bell still wait a day. Rewrite README “Live timing matters” / post-stop sections
+and a line here when shipping.
+
+### Feed / paper honesty
+
+- Document 15-minute delayed SIP: 15:35 scan may see ~15:20 prices; OK for 12-1.
+- Paper entry price ≈ scan entry (near close) is nearly automatic; add
+  **`fill_mode=near_close` era tag** so Track Record can separate morning-scan /
+  near-close / future broker-routed eras.
+- Morning sentiment staleness is display-only; gate is price-only (GTL parity /
+  neutral-sentiment backtest). One doc line closes that.
+
+### Stored settings migration (ship step 4)
+
+Flip global default TZ to `America/New_York` and re-express crons in NY time.
+**Also** migration (or documented Admin rewrite) of stored `schedule_*` keys so
+prod does not keep 07:00 Berlin silently.
+
+### Ship order
+
+1. `trade_policy` distinct-day requalify guard + unit test  
+2. Near-close job = existing fetch → scan; outcome eval own after-close slot  
+3. Paper `fill_mode=near_close` era tag; verify entry marking  
+4. Defaults + **stored settings migration** + README/research timing rewrite  
+
+### Out of scope
+
+Broker MOC routing, more fill-timing sim, nasdaq_all / fip / sector (grade later
+under the fill mode you trade).
+
+---
+
+## Implementation status
+
+| Item | Status |
+|---|---|
+| Research evidence | **Closed** — this doc + matrix report |
+| Distinct-day gate-reset guard | **Shipped** — `trade_policy` + unit test |
+| Schedule split + near-close scan→alert | **Shipped** — morning / near-close / after-close (fetch→outcome) |
+| Paper era tag | **Shipped** — `fill_mode=near_close` on new paper trades |
+| Settings migration + docs | **Shipped** — alembic 023 rewrites schedule_*; README updated |
+
+### Shipped behavior change (not accidental)
+
+Near-close scan at ~15:30 ET lets same-day fail observations after earlier
+intraday stop closes — closer to promoted `gate_reset` than the old
+morning-scan `strict_gate_reset` analogue. Documented in README.

docs/research/phase-a-matrix.md

@@ -0,0 +1,120 @@
+# Phase A research matrix (2026-07-18) — results and decisions
+
+Report: `reports/research-matrix-phase-a.json` / `.md`  
+Branch: `research/portfolio-vol-and-followups`  
+Validation split: entries ≥ **2024-07-01** (called *validation*, not holdout — this window has been opened before).  
+Cadence: daily, production gate/rank/trail + gate-reset re-entry.  
+Pre-registered N for DSR: **20**.
+
+## Pre-registered promotion rule (unchanged after run start)
+
+Promote only if **all** of:
+
+1. Validation Sharpe ≥ control  
+2. Validation max DD not worse by more than **2pp**  
+3. Train Sharpe not worse (both-windows consistency)  
+
+Always report whether validation ΔSharpe exceeds **1 × SE** (expect most will not).
+
+Mechanics guards confirmed before reading results: calendar truncation asserted on every arm; next-open re-anchors stop to fill − 1.5×ATR(signal); vol scalars apply at entry only.
+
+---
+
+## Control baseline
+
+| Window | Sharpe | SE | CAGR | MaxDD | Calmar | Trades |
+|---|---:|---:|---:|---:|---:|---:|
+| Train | 1.75 | 0.68 | 49.8% | 17.9% | 2.78 | 240 |
+| **Validation** | **1.68** | **0.72** | **41.6%** | **20.9%** | **1.99** | **239** |
+| Full (close-fill) | 1.77 | 0.50 | 48.3% | 21.6% | 2.23 | 472 |
+
+Validation SE ≈ 0.72 — almost no arm clears a 1-SE delta.
+
+---
+
+## Per-arm decisions
+
+### A2 — Max hold {30, 45, 60, 90} — **note and move on**
+
+| Hold | Val Sharpe | Val DD | Train Sharpe | Trades train |
+|---:|---:|---:|---:|---:|
+| 30 | 1.68 | 20.9 | 1.75 | 240 |
+| 45 | **2.07** | 19.3 | **1.43** | 218 |
+| 60 | **2.12** | 19.3 | **1.11** | 186 |
+| 90 | 2.04 | 23.1 | 1.30 | 179 |
+
+Validation-only would have “found” +0.4 Sharpe. Train collapses: longer holds leave stale names blocking slots (240 → 186 trades at hold-60). This is a **regime interaction** (trend validation vs chop train), not a free knob. A regime-conditional hold is a large research program; prior regime-overlay work already argues against that path.
+
+**Decision: keep max hold 30. Do not ship longer static holds.**
+
+### A3 — Equity-curve vol targeting — **reject as edge; park as optional insurance**
+
+Scalars averaged 0.77–1.07 as designed (grid straddled historical book vol ~22–25%). Lower targets de-levered; **vt25** was nearly neutral (val Sharpe 1.63 vs 1.68). Wide clamp ≈ headline clamp. Lookback sensitivity did not unlock a win.
+
+This sample has **no major vol-regime shift**, so the run **rejects vol targeting as an edge on this data** — it does **not** reject crash-insurance value in a future high-vol regime. The ~0.02 Sharpe cost at vt25 is a nearly free insurance policy if drawdown tolerance ever tightens.
+
+**Decision: do not ship. Settles “Phase 2 = vol-scaled momentum” as an edge plan on this snapshot. Park vt25 as optional risk preference only.**
+
+### A5 — Correlation caps — **reject; sector caps stay Phase B with reduced expectations**
+
+Best near-miss: **0.6 skip** — val Sharpe 1.70, val DD **17.0%** (tempting), but train Sharpe 1.61 < 1.75 and full-period Sharpe **1.59 vs 1.77** (the cap deletes real momentum concentration profit). Half-size variants were worse.
+
+**Decision: no pure corr cap. Sector caps remain Phase B with reduced expectations.**
+
+### A4 — Next-open fill — **not a reject; the discovery**
+
+| | Close control | Next-open |
+|---|---:|---:|
+| Full Sharpe | 1.77 | **1.20** |
+| Full CAGR | 48.3% | **30.0%** |
+| Val Sharpe | 1.68 | 1.44 |
+| Val DD | 20.9% | **28.2%** |
+
+Overnight gap on validation entries: mean **−0.52%**, median −0.18%, p05 −4.5%, p95 +2.1% (n=243).
+
+This is not “slippage noise.” It is largely the **overnight momentum drift** that close-fill earns and a 07:00-Berlin scanner (signal yesterday’s close → fill tomorrow’s open) **structurally cannot**. Honest deployable number under that schedule is ~Sharpe 1.2 / CAGR 30%, not 1.77 / 48%.
+
+**Decision baseline going forward (until near-close ships):** grade **promotion** under
+`fill_mode=next_open`; keep close-fill as the historical control for comparability
+with prior reports.
+
+**Follow-up (done):** execution recovery matrix — see
+**[execution-recovery.md](execution-recovery.md)**. Short version: monotone fill-timing
+gradient + DD recovery prove this is *when you fill*; live bracket **[1.57, 1.77]**;
+gap-cap dead; no more fill-timing sim on this snapshot; ops move R:R scan to NY
+near-close (one scan/day).
+
+### `fip_id` re-derivation — **validated**
+
+Weekly IC fingerprint on this snapshot: **mean IC −0.045, t = −2.92**, reliable (35 weeks). Matches the July record. Safe to reuse when the universe broadens.
+
+---
+
+## Promotion table (rule as written)
+
+| Outcome | Arms |
+|---|---|
+| Promote | only `a2_hold_30` (identity with control) |
+| Reject | every other arm |
+
+No arm cleared ΔSharpe > 1 SE.
+
+---
+
+## What not to do next
+
+- Re-litigate rejected-table items, min_rr, GTL  
+- Regime-conditional max-hold as a “small” experiment  
+- Treat validation-only max-hold glitter as a free CAGR lift  
+- Ship vol targeting as edge without a vol-regime sample  
+- More fill-timing simulation on this snapshot (settled — see execution-recovery.md)  
+- Dual daily qualifying scans (would break gate-reset validation)  
+- Gap-up entry filters (third tail-trim failure)  
+
+## What to do next
+
+1. **Ship near-close execution** — ops checklist in [execution-recovery.md](execution-recovery.md)  
+   (one R:R scan/day in `America/New_York`, MOC window, partial-bar honesty).  
+2. Until that ships: **decision baseline = next_open**.  
+3. Strategy work (nasdaq_all, fip_id, sector) only **after** execution path is decided,
+   graded under the fill mode you will trade.

frontend/src/components/admin/ScheduleSettings.tsx

@@ -4,34 +4,48 @@ import { useScheduleSettings, useUpdateScheduleSettings } from '../../hooks/useA
 import { SkeletonTable } from '../ui/Skeleton';
 
 const DEFAULTS: ScheduleConfig = {
-  schedule_timezone: 'Europe/Berlin',
-  schedule_daily_pipeline_cron: '0 7 * * *',
-  schedule_intraday_pipeline_cron: '0 14-22 * * 1-5',
-  schedule_fundamentals_cron: '0 4 * * 1',
+  schedule_timezone: 'America/New_York',
+  schedule_daily_pipeline_cron: '0 2 * * *',
+  schedule_near_close_pipeline_cron: '30 15 * * 1-5',
+  schedule_after_close_pipeline_cron: '45 16 * * 1-5',
+  schedule_intraday_pipeline_cron: '0 10-15 * * 1-5',
+  schedule_fundamentals_cron: '0 1 * * 1',
 };
 
 const FIELDS: { key: keyof ScheduleConfig; label: string; hint: string; mono?: boolean }[] = [
   {
     key: 'schedule_timezone',
     label: 'Timezone',
-    hint: 'IANA name, e.g. Europe/Berlin. All times below are in this zone.',
+    hint: 'IANA name. Prefer America/New_York so the near-close scan tracks the US cash close through DST.',
   },
   {
     key: 'schedule_daily_pipeline_cron',
-    label: 'Daily pipeline (full)',
-    hint: 'OHLCV → sentiment → R:R scan → outcomes → regime. Default 07:00 so data is ready by 8.',
+    label: 'Morning pipeline',
+    hint: 'OHLCV → benchmark → sentiment → regime → alerts (no R:R scan). Default 02:00 ET so regime-quadrant changes hit Telegram in the morning.',
+    mono: true,
+  },
+  {
+    key: 'schedule_near_close_pipeline_cron',
+    label: 'Near-close pipeline (scan + alert)',
+    hint: 'OHLCV fetch → R:R scan → Telegram. Default 15:30 ET Mon–Fri so manual MOC fills can still hit ~15:50/15:55.',
+    mono: true,
+  },
+  {
+    key: 'schedule_after_close_pipeline_cron',
+    label: 'After-close pipeline (outcome)',
+    hint: 'OHLCV fetch (final bar) → outcome eval. Default 16:45 ET Mon–Fri — not chained to the partial near-close bar.',
     mono: true,
   },
   {
     key: 'schedule_intraday_pipeline_cron',
     label: 'Intraday pipeline (light)',
-    hint: 'Refresh prices + resolve outcomes. Default hourly across the US session, weekdays.',
+    hint: 'Refresh prices + resolve outcomes mid-session. Default hourly 10:00–15:00 ET weekdays.',
     mono: true,
   },
   {
     key: 'schedule_fundamentals_cron',
     label: 'Fundamentals (weekly)',
-    hint: 'Slow, rate-limited. Default early Monday so it finishes well before the day starts.',
+    hint: 'Slow, rate-limited. Default early Monday ET.',
     mono: true,
   },
 ];
@@ -43,7 +57,7 @@ export function ScheduleSettings() {
   const [form, setForm] = useState<ScheduleConfig>(DEFAULTS);
 
   useEffect(() => {
-    if (data) setForm(data);
+    if (data) setForm({ ...DEFAULTS, ...data });
   }, [data]);
 
   if (isLoading) return <SkeletonTable rows={2} cols={2} />;
@@ -55,8 +69,8 @@ export function ScheduleSettings() {
         <h3 className="text-sm font-semibold text-gray-200">Pipeline Schedule</h3>
         <p className="mt-1 text-xs text-gray-500">
           When the jobs run, as 5-field cron (<span className="num">min hour day month weekday</span>).
-          Saved changes apply to the running scheduler immediately — no redeploy. The big nightly run
-          does the full refresh; the light intraday run just keeps prices current.
+          Saved changes apply to the running scheduler immediately — no redeploy.
+          One qualifying R:R scan per day is the near-close job; alerts fire immediately after that scan.
         </p>
       </div>
 
@@ -66,7 +80,7 @@ export function ScheduleSettings() {
             <span className="text-xs text-gray-400">{f.label}</span>
             <input
               type="text"
-              value={form[f.key]}
+              value={form[f.key] ?? ''}
               spellCheck={false}
               onChange={(e) => setForm((prev) => ({ ...prev, [f.key]: e.target.value }))}
               className={`w-full input-glass px-3 py-2 text-sm ${f.mono ? 'num' : ''}`}

frontend/src/lib/types.ts

@@ -187,10 +187,12 @@ export interface ActivationConfig {
   exclude_neutral: boolean;
 }
 
-// Cron schedule for the daily/intraday pipelines + fundamentals
+// Cron schedule for morning / near-close / after-close / intraday + fundamentals
 export interface ScheduleConfig {
   schedule_timezone: string;
   schedule_daily_pipeline_cron: string;
+  schedule_near_close_pipeline_cron: string;
+  schedule_after_close_pipeline_cron: string;
   schedule_intraday_pipeline_cron: string;
   schedule_fundamentals_cron: string;
 }

reports/execution_recovery_matrix.json

@@ -0,0 +1,796 @@
+{
+  "generated_at": "2026-07-18T16:56:10.007302",
+  "snapshot": "/Users/taathde3/git/lab/signal_platform/backtest_snapshots/prod.sqlite",
+  "validation_split": "2024-07-01",
+  "n_trials": 4,
+  "hypothesis": "stale_close (signal t-1, fill t close) recovers close-fill economics; the next_open haircut is scheduling, not lost edge",
+  "decision_baseline": "next_open",
+  "qualified_longs": 5189,
+  "arms": [
+    {
+      "id": "close_control",
+      "label": "Close fill (historical control)",
+      "config": {
+        "fill_mode": "close"
+      },
+      "windows": [
+        {
+          "window": "train",
+          "dsr": 0.936,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "close",
+          "final_equity": 23706.04,
+          "total_return_pct": 137.1,
+          "cagr_pct": 49.8,
+          "max_drawdown_pct": 17.9,
+          "calmar": 2.78,
+          "sharpe": 1.75,
+          "sharpe_se": 0.675,
+          "psr": 0.9953,
+          "n_returns": 535,
+          "return_skew": 0.4266,
+          "return_kurtosis": 4.9374,
+          "trades": 240,
+          "win_rate": 36.7,
+          "avg_trade_pnl": 57.11,
+          "best_trade_r": 12.02,
+          "worst_trade_r": -3.1,
+          "best_trade_pnl": 2008.56,
+          "worst_trade_pnl": -561.69,
+          "avg_hold_days": 15.3,
+          "exit_reasons": {
+            "stop": 112,
+            "time": 53,
+            "trailing_stop": 75
+          },
+          "skipped_book_full": 327,
+          "spy_return_pct": 36.7,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": 11.9
+            },
+            {
+              "year": 2023,
+              "return_pct": 78.1
+            },
+            {
+              "year": 2024,
+              "return_pct": 19.1
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2024-08-12",
+          "post_stop_events": 112,
+          "post_stop_reentries": 61,
+          "post_stop_states_open_at_end": 51
+        },
+        {
+          "window": "validation",
+          "dsr": 0.9039,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "close",
+          "final_equity": 20060.98,
+          "total_return_pct": 100.6,
+          "cagr_pct": 41.6,
+          "max_drawdown_pct": 20.9,
+          "calmar": 1.99,
+          "sharpe": 1.68,
+          "sharpe_se": 0.716,
+          "psr": 0.9905,
+          "n_returns": 502,
+          "return_skew": -0.0866,
+          "return_kurtosis": 4.395,
+          "trades": 239,
+          "win_rate": 35.1,
+          "avg_trade_pnl": 42.1,
+          "best_trade_r": 12.03,
+          "worst_trade_r": -2.82,
+          "best_trade_pnl": 2012.72,
+          "worst_trade_pnl": -330.01,
+          "avg_hold_days": 14.0,
+          "exit_reasons": {
+            "stop": 121,
+            "time": 44,
+            "trailing_stop": 74
+          },
+          "skipped_book_full": 207,
+          "spy_return_pct": 36.6,
+          "yearly_returns": [
+            {
+              "year": 2024,
+              "return_pct": 16.7
+            },
+            {
+              "year": 2025,
+              "return_pct": 50.3
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.5
+            }
+          ],
+          "start_date": "2024-07-01",
+          "end_date": "2026-07-02",
+          "post_stop_events": 121,
+          "post_stop_reentries": 63,
+          "post_stop_states_open_at_end": 58
+        },
+        {
+          "window": "full",
+          "dsr": 0.9939,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "close",
+          "final_equity": 48810.86,
+          "total_return_pct": 388.1,
+          "cagr_pct": 48.3,
+          "max_drawdown_pct": 21.6,
+          "calmar": 2.23,
+          "sharpe": 1.77,
+          "sharpe_se": 0.496,
+          "psr": 0.9998,
+          "n_returns": 1008,
+          "return_skew": 0.2714,
+          "return_kurtosis": 4.9751,
+          "trades": 472,
+          "win_rate": 36.2,
+          "avg_trade_pnl": 82.23,
+          "best_trade_r": 12.03,
+          "worst_trade_r": -3.1,
+          "best_trade_pnl": 4897.19,
+          "worst_trade_pnl": -802.94,
+          "avg_hold_days": 14.6,
+          "exit_reasons": {
+            "stop": 229,
+            "time": 96,
+            "trailing_stop": 147
+          },
+          "skipped_book_full": 519,
+          "spy_return_pct": 90.9,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": 11.9
+            },
+            {
+              "year": 2023,
+              "return_pct": 78.1
+            },
+            {
+              "year": 2024,
+              "return_pct": 42.5
+            },
+            {
+              "year": 2025,
+              "return_pct": 50.3
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.5
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2026-07-02",
+          "post_stop_events": 229,
+          "post_stop_reentries": 142,
+          "post_stop_states_open_at_end": 87
+        }
+      ]
+    },
+    {
+      "id": "next_open",
+      "label": "Next-open fill (decision baseline)",
+      "config": {
+        "fill_mode": "next_open"
+      },
+      "windows": [
+        {
+          "window": "train",
+          "dsr": 0.6243,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 15364.14,
+          "total_return_pct": 53.6,
+          "cagr_pct": 22.2,
+          "max_drawdown_pct": 18.1,
+          "calmar": 1.23,
+          "sharpe": 0.94,
+          "sharpe_se": 0.688,
+          "psr": 0.9148,
+          "n_returns": 536,
+          "return_skew": -0.039,
+          "return_kurtosis": 4.2924,
+          "trades": 260,
+          "win_rate": 31.2,
+          "avg_trade_pnl": 20.63,
+          "best_trade_r": 11.53,
+          "worst_trade_r": -2.99,
+          "best_trade_pnl": 1265.39,
+          "worst_trade_pnl": -303.24,
+          "avg_hold_days": 13.7,
+          "exit_reasons": {
+            "stop": 133,
+            "time": 48,
+            "trailing_stop": 79
+          },
+          "skipped_book_full": 140,
+          "spy_return_pct": 39.0,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": 4.5
+            },
+            {
+              "year": 2023,
+              "return_pct": 44.6
+            },
+            {
+              "year": 2024,
+              "return_pct": 1.7
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2024-08-13",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 260,
+            "mean_pct": -0.063,
+            "median_pct": -0.1627,
+            "p05_pct": -2.5148,
+            "p95_pct": 1.8498
+          },
+          "post_stop_events": 133,
+          "post_stop_reentries": 79,
+          "post_stop_states_open_at_end": 50
+        },
+        {
+          "window": "validation",
+          "dsr": 0.8327,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 18586.06,
+          "total_return_pct": 85.9,
+          "cagr_pct": 36.3,
+          "max_drawdown_pct": 28.2,
+          "calmar": 1.29,
+          "sharpe": 1.44,
+          "sharpe_se": 0.719,
+          "psr": 0.9774,
+          "n_returns": 502,
+          "return_skew": -0.2169,
+          "return_kurtosis": 4.6486,
+          "trades": 243,
+          "win_rate": 35.0,
+          "avg_trade_pnl": 35.33,
+          "best_trade_r": 12.2,
+          "worst_trade_r": -4.0,
+          "best_trade_pnl": 1731.6,
+          "worst_trade_pnl": -196.48,
+          "avg_hold_days": 13.8,
+          "exit_reasons": {
+            "stop": 114,
+            "time": 48,
+            "trailing_stop": 81
+          },
+          "skipped_book_full": 197,
+          "spy_return_pct": 36.6,
+          "yearly_returns": [
+            {
+              "year": 2024,
+              "return_pct": 14.2
+            },
+            {
+              "year": 2025,
+              "return_pct": 42.6
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.1
+            }
+          ],
+          "start_date": "2024-07-01",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 243,
+            "mean_pct": -0.519,
+            "median_pct": -0.1786,
+            "p05_pct": -4.5194,
+            "p95_pct": 2.0911
+          },
+          "post_stop_events": 114,
+          "post_stop_reentries": 51,
+          "post_stop_states_open_at_end": 52
+        },
+        {
+          "window": "full",
+          "dsr": 0.9093,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 28701.57,
+          "total_return_pct": 187.0,
+          "cagr_pct": 30.0,
+          "max_drawdown_pct": 28.2,
+          "calmar": 1.06,
+          "sharpe": 1.2,
+          "sharpe_se": 0.504,
+          "psr": 0.9914,
+          "n_returns": 1008,
+          "return_skew": -0.1589,
+          "return_kurtosis": 4.4517,
+          "trades": 502,
+          "win_rate": 32.5,
+          "avg_trade_pnl": 37.25,
+          "best_trade_r": 12.2,
+          "worst_trade_r": -4.0,
+          "best_trade_pnl": 2674.03,
+          "worst_trade_pnl": -303.42,
+          "avg_hold_days": 13.7,
+          "exit_reasons": {
+            "stop": 251,
+            "time": 97,
+            "trailing_stop": 154
+          },
+          "skipped_book_full": 356,
+          "spy_return_pct": 90.9,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": 4.5
+            },
+            {
+              "year": 2023,
+              "return_pct": 44.6
+            },
+            {
+              "year": 2024,
+              "return_pct": 15.7
+            },
+            {
+              "year": 2025,
+              "return_pct": 44.0
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.1
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 502,
+            "mean_pct": -0.2566,
+            "median_pct": -0.1661,
+            "p05_pct": -2.7763,
+            "p95_pct": 1.8997
+          },
+          "post_stop_events": 251,
+          "post_stop_reentries": 157,
+          "post_stop_states_open_at_end": 81
+        }
+      ]
+    },
+    {
+      "id": "stale_close",
+      "label": "Stale-signal close fill (MOC proxy: signal t-1, fill t close)",
+      "config": {
+        "fill_mode": "stale_close"
+      },
+      "windows": [
+        {
+          "window": "train",
+          "dsr": 0.8241,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "stale_close",
+          "final_equity": 18942.53,
+          "total_return_pct": 89.4,
+          "cagr_pct": 34.8,
+          "max_drawdown_pct": 17.0,
+          "calmar": 2.05,
+          "sharpe": 1.36,
+          "sharpe_se": 0.685,
+          "psr": 0.9767,
+          "n_returns": 536,
+          "return_skew": 0.0918,
+          "return_kurtosis": 3.1111,
+          "trades": 246,
+          "win_rate": 39.0,
+          "avg_trade_pnl": 36.35,
+          "best_trade_r": 10.3,
+          "worst_trade_r": -2.67,
+          "best_trade_pnl": 1318.25,
+          "worst_trade_pnl": -328.1,
+          "avg_hold_days": 16.1,
+          "exit_reasons": {
+            "stop": 103,
+            "time": 54,
+            "trailing_stop": 89
+          },
+          "skipped_book_full": 668,
+          "spy_return_pct": 39.0,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": -0.9
+            },
+            {
+              "year": 2023,
+              "return_pct": 72.8
+            },
+            {
+              "year": 2024,
+              "return_pct": 10.6
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2024-08-13",
+          "skipped_missing_fill": 0,
+          "signal_to_fill_drift": {
+            "n": 246,
+            "mean_pct": -0.3836,
+            "median_pct": -0.4703,
+            "p05_pct": -4.841,
+            "p95_pct": 3.908
+          },
+          "post_stop_events": 103,
+          "post_stop_reentries": 54,
+          "post_stop_states_open_at_end": 47
+        },
+        {
+          "window": "validation",
+          "dsr": 0.9216,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "stale_close",
+          "final_equity": 20546.48,
+          "total_return_pct": 105.5,
+          "cagr_pct": 43.3,
+          "max_drawdown_pct": 21.6,
+          "calmar": 2.0,
+          "sharpe": 1.74,
+          "sharpe_se": 0.702,
+          "psr": 0.9933,
+          "n_returns": 502,
+          "return_skew": 0.3013,
+          "return_kurtosis": 5.6337,
+          "trades": 234,
+          "win_rate": 37.2,
+          "avg_trade_pnl": 45.07,
+          "best_trade_r": 11.94,
+          "worst_trade_r": -4.23,
+          "best_trade_pnl": 2186.87,
+          "worst_trade_pnl": -732.55,
+          "avg_hold_days": 15.2,
+          "exit_reasons": {
+            "open_at_end": 1,
+            "stop": 103,
+            "time": 50,
+            "trailing_stop": 80
+          },
+          "skipped_book_full": 223,
+          "spy_return_pct": 36.6,
+          "yearly_returns": [
+            {
+              "year": 2024,
+              "return_pct": 17.6
+            },
+            {
+              "year": 2025,
+              "return_pct": 74.8
+            },
+            {
+              "year": 2026,
+              "return_pct": -0.1
+            }
+          ],
+          "start_date": "2024-07-01",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "signal_to_fill_drift": {
+            "n": 234,
+            "mean_pct": -0.5217,
+            "median_pct": -0.0138,
+            "p05_pct": -6.5593,
+            "p95_pct": 4.5614
+          },
+          "post_stop_events": 103,
+          "post_stop_reentries": 45,
+          "post_stop_states_open_at_end": 53
+        },
+        {
+          "window": "full",
+          "dsr": 0.9816,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "stale_close",
+          "final_equity": 39381.26,
+          "total_return_pct": 293.8,
+          "cagr_pct": 40.6,
+          "max_drawdown_pct": 21.8,
+          "calmar": 1.87,
+          "sharpe": 1.57,
+          "sharpe_se": 0.5,
+          "psr": 0.9991,
+          "n_returns": 1008,
+          "return_skew": 0.0941,
+          "return_kurtosis": 4.1297,
+          "trades": 483,
+          "win_rate": 37.5,
+          "avg_trade_pnl": 60.83,
+          "best_trade_r": 11.94,
+          "worst_trade_r": -4.23,
+          "best_trade_pnl": 4191.56,
+          "worst_trade_pnl": -1404.07,
+          "avg_hold_days": 15.5,
+          "exit_reasons": {
+            "open_at_end": 1,
+            "stop": 208,
+            "time": 102,
+            "trailing_stop": 172
+          },
+          "skipped_book_full": 894,
+          "spy_return_pct": 90.9,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": -0.9
+            },
+            {
+              "year": 2023,
+              "return_pct": 72.8
+            },
+            {
+              "year": 2024,
+              "return_pct": 31.7
+            },
+            {
+              "year": 2025,
+              "return_pct": 74.7
+            },
+            {
+              "year": 2026,
+              "return_pct": -0.1
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "signal_to_fill_drift": {
+            "n": 483,
+            "mean_pct": -0.4799,
+            "median_pct": -0.381,
+            "p05_pct": -5.5621,
+            "p95_pct": 4.2818
+          },
+          "post_stop_events": 208,
+          "post_stop_reentries": 118,
+          "post_stop_states_open_at_end": 83
+        }
+      ]
+    },
+    {
+      "id": "next_open_gap2",
+      "label": "Next-open + skip gap-up > 2%",
+      "config": {
+        "fill_mode": "next_open",
+        "max_entry_gap_pct": 0.02
+      },
+      "windows": [
+        {
+          "window": "train",
+          "dsr": 0.4988,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 13619.82,
+          "total_return_pct": 36.2,
+          "cagr_pct": 15.5,
+          "max_drawdown_pct": 21.5,
+          "calmar": 0.72,
+          "sharpe": 0.72,
+          "sharpe_se": 0.686,
+          "psr": 0.8518,
+          "n_returns": 536,
+          "return_skew": 0.0343,
+          "return_kurtosis": 4.6315,
+          "trades": 274,
+          "win_rate": 31.4,
+          "avg_trade_pnl": 13.21,
+          "best_trade_r": 11.53,
+          "worst_trade_r": -2.99,
+          "best_trade_pnl": 1223.62,
+          "worst_trade_pnl": -268.81,
+          "avg_hold_days": 13.4,
+          "exit_reasons": {
+            "stop": 138,
+            "time": 47,
+            "trailing_stop": 89
+          },
+          "skipped_book_full": 325,
+          "spy_return_pct": 39.0,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": -4.1
+            },
+            {
+              "year": 2023,
+              "return_pct": 38.7
+            },
+            {
+              "year": 2024,
+              "return_pct": 2.4
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2024-08-13",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 274,
+            "mean_pct": -0.4245,
+            "median_pct": -0.2836,
+            "p05_pct": -2.6277,
+            "p95_pct": 1.3953
+          },
+          "max_entry_gap_pct": 0.02,
+          "skipped_gap_cap": 146,
+          "post_stop_events": 138,
+          "post_stop_reentries": 76,
+          "post_stop_states_open_at_end": 55
+        },
+        {
+          "window": "validation",
+          "dsr": 0.8234,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 18053.3,
+          "total_return_pct": 80.5,
+          "cagr_pct": 34.3,
+          "max_drawdown_pct": 28.2,
+          "calmar": 1.22,
+          "sharpe": 1.41,
+          "sharpe_se": 0.715,
+          "psr": 0.9758,
+          "n_returns": 502,
+          "return_skew": -0.0944,
+          "return_kurtosis": 4.7767,
+          "trades": 251,
+          "win_rate": 35.1,
+          "avg_trade_pnl": 32.08,
+          "best_trade_r": 12.2,
+          "worst_trade_r": -4.0,
+          "best_trade_pnl": 1693.37,
+          "worst_trade_pnl": -213.76,
+          "avg_hold_days": 13.7,
+          "exit_reasons": {
+            "open_at_end": 1,
+            "stop": 117,
+            "time": 49,
+            "trailing_stop": 84
+          },
+          "skipped_book_full": 200,
+          "spy_return_pct": 36.6,
+          "yearly_returns": [
+            {
+              "year": 2024,
+              "return_pct": 12.7
+            },
+            {
+              "year": 2025,
+              "return_pct": 40.4
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.1
+            }
+          ],
+          "start_date": "2024-07-01",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 251,
+            "mean_pct": -0.7233,
+            "median_pct": -0.2245,
+            "p05_pct": -4.6154,
+            "p95_pct": 1.0467
+          },
+          "max_entry_gap_pct": 0.02,
+          "skipped_gap_cap": 116,
+          "post_stop_events": 117,
+          "post_stop_reentries": 56,
+          "post_stop_states_open_at_end": 53
+        },
+        {
+          "window": "full",
+          "dsr": 0.8561,
+          "starting_capital": 10000.0,
+          "cost_per_side_pct": 0.1,
+          "fill_mode": "next_open",
+          "final_equity": 24698.25,
+          "total_return_pct": 147.0,
+          "cagr_pct": 25.2,
+          "max_drawdown_pct": 27.8,
+          "calmar": 0.91,
+          "sharpe": 1.06,
+          "sharpe_se": 0.502,
+          "psr": 0.983,
+          "n_returns": 1008,
+          "return_skew": -0.0453,
+          "return_kurtosis": 4.7254,
+          "trades": 523,
+          "win_rate": 32.7,
+          "avg_trade_pnl": 28.1,
+          "best_trade_r": 12.2,
+          "worst_trade_r": -4.0,
+          "best_trade_pnl": 2316.66,
+          "worst_trade_pnl": -292.44,
+          "avg_hold_days": 13.4,
+          "exit_reasons": {
+            "open_at_end": 1,
+            "stop": 258,
+            "time": 97,
+            "trailing_stop": 167
+          },
+          "skipped_book_full": 525,
+          "spy_return_pct": 90.9,
+          "yearly_returns": [
+            {
+              "year": 2022,
+              "return_pct": -4.1
+            },
+            {
+              "year": 2023,
+              "return_pct": 38.7
+            },
+            {
+              "year": 2024,
+              "return_pct": 14.8
+            },
+            {
+              "year": 2025,
+              "return_pct": 41.7
+            },
+            {
+              "year": 2026,
+              "return_pct": 14.1
+            }
+          ],
+          "start_date": "2022-06-24",
+          "end_date": "2026-07-02",
+          "skipped_missing_fill": 0,
+          "overnight_slippage": {
+            "n": 523,
+            "mean_pct": -0.5565,
+            "median_pct": -0.2628,
+            "p05_pct": -3.5771,
+            "p95_pct": 1.3699
+          },
+          "max_entry_gap_pct": 0.02,
+          "skipped_gap_cap": 262,
+          "post_stop_events": 258,
+          "post_stop_reentries": 158,
+          "post_stop_states_open_at_end": 87
+        }
+      ]
+    }
+  ],
+  "recovery": {
+    "recover": false,
+    "near_close_control": true,
+    "beats_next_open": true,
+    "train_ok": false,
+    "validation_delta_vs_close": 0.06,
+    "validation_delta_vs_next_open": 0.3,
+    "half_se": 0.358,
+    "reason": "stale_close does not meet recovery criteria \u2014 see flags"
+  }
+}

reports/execution_recovery_matrix.md

@@ -0,0 +1,23 @@
+# Execution recovery matrix — 2026-07-18T16:56:10.007302
+
+Validation split: **2024-07-01**. N for DSR: **4**.
+
+| arm | window | Sharpe | SE | CAGR | MaxDD | trades | gap/drift |
+|---|---|---:|---:|---:|---:|---:|---|
+| close_control | train | 1.75 | 0.675 | 49.8 | 17.9 | 240 | — |
+| close_control | validation | 1.68 | 0.716 | 41.6 | 20.9 | 239 | — |
+| close_control | full | 1.77 | 0.496 | 48.3 | 21.6 | 472 | — |
+| next_open | train | 0.94 | 0.688 | 22.2 | 18.1 | 260 | mean -0.063% n=260 |
+| next_open | validation | 1.44 | 0.719 | 36.3 | 28.2 | 243 | mean -0.519% n=243 |
+| next_open | full | 1.2 | 0.504 | 30.0 | 28.2 | 502 | mean -0.2566% n=502 |
+| stale_close | train | 1.36 | 0.685 | 34.8 | 17.0 | 246 | mean -0.3836% n=246 |
+| stale_close | validation | 1.74 | 0.702 | 43.3 | 21.6 | 234 | mean -0.5217% n=234 |
+| stale_close | full | 1.57 | 0.5 | 40.6 | 21.8 | 483 | mean -0.4799% n=483 |
+| next_open_gap2 | train | 0.72 | 0.686 | 15.5 | 21.5 | 274 | mean -0.4245% n=274; gap_skips=146 |
+| next_open_gap2 | validation | 1.41 | 0.715 | 34.3 | 28.2 | 251 | mean -0.7233% n=251; gap_skips=116 |
+| next_open_gap2 | full | 1.06 | 0.502 | 25.2 | 27.8 | 523 | mean -0.5565% n=523; gap_skips=262 |
+
+## Recovery decision (stale_close)
+
+- **recover: False** — stale_close does not meet recovery criteria — see flags
+- flags: {'near_close_control': True, 'beats_next_open': True, 'train_ok': False, 'validation_delta_vs_close': 0.06, 'validation_delta_vs_next_open': 0.3, 'half_se': 0.358}

reports/research-matrix-phase-a.md

@@ -0,0 +1,94 @@
+# Research matrix — 2026-07-18T15:17:10.310517
+
+Validation split: **2024-07-01**. Pre-registered N for DSR: **20**.
+
+## Promotion rule
+
+Promote only if validation Sharpe ≥ control, validation DD not worse by >2pp, and train Sharpe not worse. Always report whether validation Sharpe delta exceeds 1 SE (expect most will not).
+
+## Arms
+
+| arm | window | Sharpe | SE | PSR | DSR | CAGR | MaxDD | Calmar | trades | avg scalar |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| a0_control | train | 1.75 | 0.675 | 0.995 | 0.745 | 49.8 | 17.9 | 2.78 | 240 | — |
+| a0_control | validation | 1.68 | 0.716 | 0.991 | 0.679 | 41.6 | 20.9 | 1.99 | 239 | — |
+| a0_control | full | 1.77 | 0.496 | 1 | 0.951 | 48.3 | 21.6 | 2.23 | 472 | — |
+| a2_hold_30 | train | 1.75 | 0.675 | 0.995 | 0.745 | 49.8 | 17.9 | 2.78 | 240 | — |
+| a2_hold_30 | validation | 1.68 | 0.716 | 0.991 | 0.679 | 41.6 | 20.9 | 1.99 | 239 | — |
+| a2_hold_30 | full | 1.77 | 0.496 | 1 | 0.951 | 48.3 | 21.6 | 2.23 | 472 | — |
+| a2_hold_45 | train | 1.43 | 0.677 | 0.983 | 0.583 | 36.4 | 21.2 | 1.72 | 218 | — |
+| a2_hold_45 | validation | 2.07 | 0.698 | 0.999 | 0.85 | 59.1 | 19.3 | 3.07 | 202 | — |
+| a2_hold_45 | full | 1.78 | 0.497 | 1 | 0.952 | 49.2 | 21.2 | 2.32 | 415 | — |
+| a2_hold_60 | train | 1.11 | 0.668 | 0.952 | 0.405 | 26.3 | 21 | 1.25 | 186 | — |
+| a2_hold_60 | validation | 2.12 | 0.695 | 0.999 | 0.867 | 64.9 | 19.3 | 3.37 | 201 | — |
+| a2_hold_60 | full | 1.62 | 0.499 | 0.999 | 0.91 | 44.9 | 21 | 2.14 | 385 | — |
+| a2_hold_90 | train | 1.3 | 0.652 | 0.977 | 0.538 | 30.7 | 21 | 1.47 | 179 | — |
+| a2_hold_90 | validation | 2.04 | 0.711 | 0.998 | 0.835 | 61.8 | 23.1 | 2.67 | 191 | — |
+| a2_hold_90 | full | 1.67 | 0.503 | 1 | 0.924 | 46.8 | 23.5 | 1.99 | 367 | — |
+| a3_vt15_c05_15_lb60 | train | 1.54 | 0.674 | 0.989 | 0.636 | 37.7 | 17.8 | 2.12 | 283 | 0.769 |
+| a3_vt15_c05_15_lb60 | validation | 1.24 | 0.715 | 0.959 | 0.44 | 25.9 | 23.6 | 1.1 | 268 | 0.798 |
+| a3_vt15_c05_15_lb60 | full | 1.43 | 0.497 | 0.998 | 0.833 | 32.8 | 23.1 | 1.42 | 544 | 0.77 |
+| a3_vt20_c05_15_lb60 | train | 1.63 | 0.67 | 0.992 | 0.686 | 43.3 | 17.7 | 2.45 | 257 | 0.895 |
+| a3_vt20_c05_15_lb60 | validation | 1.48 | 0.715 | 0.981 | 0.573 | 34.7 | 23.7 | 1.46 | 248 | 0.918 |
+| a3_vt20_c05_15_lb60 | full | 1.59 | 0.495 | 0.999 | 0.902 | 40.5 | 23.5 | 1.72 | 498 | 0.903 |
+| a3_vt25_c05_15_lb60 | train | 1.68 | 0.671 | 0.994 | 0.712 | 47.4 | 17.8 | 2.66 | 239 | 1.05 |
+| a3_vt25_c05_15_lb60 | validation | 1.63 | 0.706 | 0.99 | 0.655 | 41.9 | 21.8 | 1.93 | 235 | 1.07 |
+| a3_vt25_c05_15_lb60 | full | 1.75 | 0.491 | 1 | 0.948 | 48.5 | 22.2 | 2.19 | 468 | 1.06 |
+| a3_vt15_c025_20_lb60 | train | 1.63 | 0.679 | 0.992 | 0.683 | 40.3 | 17.8 | 2.26 | 281 | 0.775 |
+| a3_vt15_c025_20_lb60 | validation | 1.24 | 0.715 | 0.959 | 0.44 | 25.9 | 23.6 | 1.1 | 268 | 0.798 |
+| a3_vt15_c025_20_lb60 | full | 1.49 | 0.499 | 0.999 | 0.86 | 34.2 | 23.1 | 1.48 | 542 | 0.774 |
+| a3_vt20_c025_20_lb60 | train | 1.63 | 0.67 | 0.992 | 0.686 | 43.3 | 17.7 | 2.45 | 257 | 0.895 |
+| a3_vt20_c025_20_lb60 | validation | 1.48 | 0.715 | 0.981 | 0.573 | 34.7 | 23.7 | 1.46 | 248 | 0.918 |
+| a3_vt20_c025_20_lb60 | full | 1.59 | 0.495 | 0.999 | 0.902 | 40.5 | 23.5 | 1.72 | 498 | 0.903 |
+| a3_vt25_c025_20_lb60 | train | 1.68 | 0.671 | 0.994 | 0.712 | 47.5 | 17.8 | 2.66 | 239 | 1.05 |
+| a3_vt25_c025_20_lb60 | validation | 1.63 | 0.706 | 0.99 | 0.655 | 41.9 | 21.8 | 1.93 | 235 | 1.07 |
+| a3_vt25_c025_20_lb60 | full | 1.75 | 0.49 | 1 | 0.949 | 48.5 | 22.2 | 2.19 | 468 | 1.06 |
+| a3_vt20_c05_15_lb20 | train | 1.76 | 0.667 | 0.996 | 0.752 | 48.7 | 17.1 | 2.84 | 253 | 0.976 |
+| a3_vt20_c05_15_lb20 | validation | 1.65 | 0.711 | 0.99 | 0.664 | 40.4 | 24.1 | 1.67 | 243 | 0.938 |
+| a3_vt20_c05_15_lb20 | full | 1.75 | 0.492 | 1 | 0.948 | 46.4 | 22.2 | 2.1 | 487 | 0.966 |
+| a3_vt20_c05_15_lb126 | train | 1.55 | 0.671 | 0.989 | 0.642 | 41.2 | 17.9 | 2.3 | 260 | 0.88 |
+| a3_vt20_c05_15_lb126 | validation | 1.37 | 0.714 | 0.972 | 0.512 | 30.6 | 20.5 | 1.49 | 247 | 0.933 |
+| a3_vt20_c05_15_lb126 | full | 1.54 | 0.495 | 0.999 | 0.883 | 38.6 | 19.9 | 1.94 | 497 | 0.895 |
+| a4_next_open | train | 0.94 | 0.688 | 0.915 | 0.298 | 22.2 | 18.1 | 1.23 | 260 | — |
+| a4_next_open | validation | 1.44 | 0.719 | 0.977 | 0.551 | 36.3 | 28.2 | 1.29 | 243 | — |
+| a4_next_open | full | 1.2 | 0.504 | 0.991 | 0.69 | 30 | 28.2 | 1.06 | 502 | — |
+| a5_corr06_skip | train | 1.61 | 0.677 | 0.991 | 0.673 | 39.9 | 20.7 | 1.93 | 235 | — |
+| a5_corr06_skip | validation | 1.7 | 0.714 | 0.991 | 0.689 | 39 | 17 | 2.3 | 222 | — |
+| a5_corr06_skip | full | 1.59 | 0.5 | 0.999 | 0.899 | 38.2 | 20.7 | 1.85 | 450 | — |
+| a5_corr07_skip | train | 1.4 | 0.676 | 0.981 | 0.555 | 34.3 | 19.9 | 1.73 | 244 | — |
+| a5_corr07_skip | validation | 1.63 | 0.709 | 0.989 | 0.655 | 39.4 | 25.5 | 1.55 | 234 | — |
+| a5_corr07_skip | full | 1.55 | 0.495 | 0.999 | 0.887 | 38.4 | 26 | 1.48 | 472 | — |
+| a5_corr08_skip | train | 1.53 | 0.672 | 0.989 | 0.631 | 40.4 | 19.1 | 2.11 | 246 | — |
+| a5_corr08_skip | validation | 1.37 | 0.709 | 0.973 | 0.512 | 32.4 | 25.8 | 1.26 | 233 | — |
+| a5_corr08_skip | full | 1.49 | 0.493 | 0.999 | 0.863 | 38 | 26.6 | 1.43 | 475 | — |
+| a5_corr06_half | train | 1.45 | 0.68 | 0.984 | 0.584 | 36.2 | 16.5 | 2.19 | 273 | — |
+| a5_corr06_half | validation | 1.19 | 0.713 | 0.952 | 0.412 | 25 | 25.7 | 0.97 | 255 | — |
+| a5_corr06_half | full | 1.38 | 0.497 | 0.997 | 0.806 | 32.5 | 25.4 | 1.28 | 521 | — |
+| a5_corr07_half | train | 1.62 | 0.674 | 0.992 | 0.679 | 42.5 | 16.2 | 2.62 | 253 | — |
+| a5_corr07_half | validation | 1.6 | 0.712 | 0.988 | 0.638 | 38.1 | 21.1 | 1.8 | 246 | — |
+| a5_corr07_half | full | 1.66 | 0.494 | 1 | 0.924 | 42.5 | 21.8 | 1.95 | 492 | — |
+| a5_corr08_half | train | 1.56 | 0.674 | 0.99 | 0.647 | 40.8 | 16.1 | 2.54 | 256 | — |
+| a5_corr08_half | validation | 1.42 | 0.709 | 0.978 | 0.54 | 33.2 | 21.1 | 1.57 | 243 | — |
+| a5_corr08_half | full | 1.55 | 0.494 | 0.999 | 0.887 | 39.2 | 21.8 | 1.8 | 492 | — |
+
+## Promotion decisions
+
+- **a2_hold_30**: PROMOTE — validation Sharpe 1.68 ≥ control 1.68; DD 20.9 within +2pp of 20.9; train Sharpe 1.75 ≥ 1.75 (delta ≤ 1 SE — distinguishable noise bar not cleared)
+- **a2_hold_45**: reject — failed: train_sharpe_not_worse
+- **a2_hold_60**: reject — failed: train_sharpe_not_worse
+- **a2_hold_90**: reject — failed: validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a3_vt15_c05_15_lb60**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a3_vt20_c05_15_lb60**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a3_vt25_c05_15_lb60**: reject — failed: validation_sharpe_ge_control, train_sharpe_not_worse
+- **a3_vt15_c025_20_lb60**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a3_vt20_c025_20_lb60**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a3_vt25_c025_20_lb60**: reject — failed: validation_sharpe_ge_control, train_sharpe_not_worse
+- **a3_vt20_c05_15_lb20**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp
+- **a3_vt20_c05_15_lb126**: reject — failed: validation_sharpe_ge_control, train_sharpe_not_worse
+- **a4_next_open**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a5_corr06_skip**: reject — failed: train_sharpe_not_worse
+- **a5_corr07_skip**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a5_corr08_skip**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a5_corr06_half**: reject — failed: validation_sharpe_ge_control, validation_dd_not_worse_by_2pp, train_sharpe_not_worse
+- **a5_corr07_half**: reject — failed: validation_sharpe_ge_control, train_sharpe_not_worse
+- **a5_corr08_half**: reject — failed: validation_sharpe_ge_control, train_sharpe_not_worse

scripts/run_execution_recovery_matrix.py

@@ -0,0 +1,494 @@
+"""Execution-recovery matrix: is the close→next_open gap recoverable by scheduling?
+
+Hypothesis (from Phase A A4)
+----------------------------
+The 1.77 → 1.20 full-period Sharpe gap under next_open fill is mostly overnight
+momentum drift that a 07:00-Berlin scanner cannot earn. Near-close / MOC-style
+execution (scan ~15:45 ET, fill at/near that close) should recover it.
+
+Pre-registered arms (N for DSR = 4)
+----------------------------------
+1. ``close_control`` — historical optimistic control (signal = fill at same close).
+2. ``next_open`` — honest overnight scanner (decision baseline for future promotion).
+3. ``stale_close`` — signal at t−1 close, fill at t close (one-session-stale MOC proxy).
+   Expectation: ≈ close_control; if so, the gap is scheduling, not physics.
+4. ``next_open_gap2`` — next_open but skip entries that open > +2% above signal close.
+   Measure whether large gap-ups are toxic or the best continuations.
+
+Promotion / read rule (pre-registered)
+--------------------------------------
+- Decision baseline for *future* strategy work: ``next_open``.
+- Recovery success for ``stale_close``: validation Sharpe within 0.5×SE of
+  ``close_control`` **and** validation Sharpe ≥ ``next_open``; train Sharpe not
+  worse than close_control by more than 0.5×SE. State 1-SE distinguishability.
+- ``next_open_gap2`` is measurement-only vs ``next_open`` (no auto-promote to live).
+
+Reuses the same daily candidate cache as the Phase A matrix when the cache key
+matches.
+
+Usage
+-----
+  python scripts/run_execution_recovery_matrix.py backtest_snapshots/prod.sqlite \\
+      --workers 7 --allow-spawn \\
+      --candidate-cache reports/.cache/research-cands.pkl \\
+      --out reports/execution-recovery-matrix.json
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import multiprocessing
+import os
+import pickle
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from datetime import date, datetime
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
+
+ROOT = Path(__file__).resolve().parents[1]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+# Must match Phase A cache when reusing research-cands.pkl
+CACHE_VERSION = "research-matrix-v1-daily-prod"
+
+PRE_REGISTERED_ARMS: tuple[dict[str, Any], ...] = (
+    {
+        "id": "close_control",
+        "label": "Close fill (historical control)",
+        "fill_mode": "close",
+    },
+    {
+        "id": "next_open",
+        "label": "Next-open fill (decision baseline)",
+        "fill_mode": "next_open",
+    },
+    {
+        "id": "stale_close",
+        "label": "Stale-signal close fill (MOC proxy: signal t-1, fill t close)",
+        "fill_mode": "stale_close",
+    },
+    {
+        "id": "next_open_gap2",
+        "label": "Next-open + skip gap-up > 2%",
+        "fill_mode": "next_open",
+        "max_entry_gap_pct": 0.02,
+    },
+)
+N_TRIALS = len(PRE_REGISTERED_ARMS)
+
+
+def _sqlite_url(path: Path) -> str:
+    return f"sqlite+aiosqlite:///{path.resolve().as_posix()}"
+
+
+def _parse_args() -> argparse.Namespace:
+    p = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    p.add_argument("snapshot")
+    p.add_argument("--workers", type=int, default=6)
+    p.add_argument("--allow-spawn", action="store_true")
+    p.add_argument("--out", default=None)
+    p.add_argument("--candidate-cache", default=None)
+    p.add_argument("--validation-split", default="2024-07-01")
+    p.add_argument("--cadence", choices=("daily", "weekly"), default="daily")
+    p.add_argument("--quiet", action="store_true")
+    return p.parse_args()
+
+
+def _period_percentiles(
+    observations: list[dict], value_key: str
+) -> dict[tuple[str, str], float]:
+    by_period: dict[tuple, list[dict]] = {}
+    for row in observations:
+        if row.get(value_key) is None:
+            continue
+        period = tuple(row["ranking_period"])
+        by_period.setdefault(period, []).append(row)
+    result: dict[tuple[str, str], float] = {}
+    for group in by_period.values():
+        ordered = sorted(
+            group, key=lambda row: (float(row[value_key]), str(row["symbol"]))
+        )
+        denominator = len(ordered) - 1
+        for rank, row in enumerate(ordered):
+            result[(str(row["symbol"]), str(row["date"]))] = round(
+                rank / denominator * 100.0 if denominator > 0 else 100.0,
+                2,
+            )
+    return result
+
+
+def _live_universe_rank_map(
+    observations: list[dict],
+    benchmark_closes: dict[date, float],
+    momentum_weight: float,
+) -> dict[tuple[str, str], dict[str, float | None]]:
+    raw_pct = _period_percentiles(observations, "momentum")
+    residual_pct = _period_percentiles(observations, "residual_momentum")
+    vol_pct = _period_percentiles(observations, "vol_6m")
+    benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
+    residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
+    ranks: dict[tuple[str, str], dict[str, float | None]] = {}
+    for row in observations:
+        identity = (str(row["symbol"]), str(row["date"]))
+        asof_ord = date.fromisoformat(identity[1]).toordinal()
+        momentum_pct = (
+            residual_pct.get(identity)
+            if residual_start_ord is not None and asof_ord >= residual_start_ord
+            else raw_pct.get(identity)
+        )
+        volatility_pct = vol_pct.get(identity)
+        strategy_rank = (
+            round(
+                momentum_pct * momentum_weight
+                + volatility_pct * (1.0 - momentum_weight),
+                2,
+            )
+            if momentum_pct is not None and volatility_pct is not None
+            else momentum_pct
+        )
+        ranks[identity] = {
+            "momentum_percentile": momentum_pct,
+            "volatility_percentile": volatility_pct,
+            "strategy_rank": strategy_rank,
+        }
+    return ranks
+
+
+def _window(arm: dict, name: str) -> dict | None:
+    for row in arm.get("windows") or []:
+        if row.get("window") == name:
+            return row
+    return None
+
+
+def _grade_stale_close(close_arm: dict, next_arm: dict, stale_arm: dict) -> dict:
+    c_val = _window(close_arm, "validation") or {}
+    n_val = _window(next_arm, "validation") or {}
+    s_val = _window(stale_arm, "validation") or {}
+    c_tr = _window(close_arm, "train") or {}
+    s_tr = _window(stale_arm, "train") or {}
+    keys = ("sharpe", "sharpe_se")
+    if any(c_val.get(k) is None for k in keys) or s_val.get("sharpe") is None:
+        return {"recover": False, "reason": "missing Sharpe rows"}
+    se = float(c_val.get("sharpe_se") or s_val.get("sharpe_se") or 0.0)
+    half_se = 0.5 * se if se > 0 else 0.0
+    cs, ss, ns = float(c_val["sharpe"]), float(s_val["sharpe"]), n_val.get("sharpe")
+    cts, sts = c_tr.get("sharpe"), s_tr.get("sharpe")
+    near_close = abs(ss - cs) <= half_se if half_se > 0 else abs(ss - cs) < 0.05
+    beats_next = ns is None or ss >= float(ns)
+    train_ok = (
+        cts is None
+        or sts is None
+        or float(sts) >= float(cts) - half_se
+    )
+    recover = near_close and beats_next and train_ok
+    return {
+        "recover": recover,
+        "near_close_control": near_close,
+        "beats_next_open": beats_next,
+        "train_ok": train_ok,
+        "validation_delta_vs_close": round(ss - cs, 4),
+        "validation_delta_vs_next_open": (
+            round(ss - float(ns), 4) if ns is not None else None
+        ),
+        "half_se": half_se,
+        "reason": (
+            "stale_close recovers close-fill economics (within 0.5 SE) and beats next_open"
+            if recover
+            else "stale_close does not meet recovery criteria — see flags"
+        ),
+    }
+
+
+def _markdown(report: dict) -> str:
+    lines = [
+        f"# Execution recovery matrix — {report.get('generated_at', '')}",
+        "",
+        f"Validation split: **{report.get('validation_split')}**. N for DSR: **{report.get('n_trials')}**.",
+        "",
+        "| arm | window | Sharpe | SE | CAGR | MaxDD | trades | gap/drift |",
+        "|---|---|---:|---:|---:|---:|---:|---|",
+    ]
+    for arm in report.get("arms") or []:
+        for w in arm.get("windows") or []:
+            slip = w.get("overnight_slippage") or w.get("signal_to_fill_drift") or {}
+            slip_s = (
+                f"mean {slip.get('mean_pct')}% n={slip.get('n')}"
+                if slip
+                else "—"
+            )
+            if w.get("skipped_gap_cap") is not None:
+                slip_s += f"; gap_skips={w.get('skipped_gap_cap')}"
+            lines.append(
+                f"| {arm.get('id')} | {w.get('window')} | {w.get('sharpe')} | "
+                f"{w.get('sharpe_se')} | {w.get('cagr_pct')} | {w.get('max_drawdown_pct')} | "
+                f"{w.get('trades')} | {slip_s} |"
+            )
+    rec = report.get("recovery") or {}
+    lines.extend(
+        [
+            "",
+            "## Recovery decision (stale_close)",
+            "",
+            f"- **recover: {rec.get('recover')}** — {rec.get('reason')}",
+            f"- flags: { {k: rec.get(k) for k in ('near_close_control', 'beats_next_open', 'train_ok', 'validation_delta_vs_close', 'validation_delta_vs_next_open', 'half_se')} }",
+            "",
+        ]
+    )
+    return "\n".join(lines)
+
+
+def _checkpoint(path: Path, report: dict) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(path.suffix + ".tmp")
+    tmp.write_text(json.dumps(report, indent=2, default=str), encoding="utf-8")
+    tmp.replace(path)
+    path.with_suffix(".md").write_text(_markdown(report), encoding="utf-8")
+
+
+async def _main() -> None:
+    args = _parse_args()
+    snapshot = Path(args.snapshot)
+    if not snapshot.exists():
+        raise SystemExit(f"Snapshot not found: {snapshot}")
+    validation_split = date.fromisoformat(args.validation_split)
+    out_path = (
+        Path(args.out)
+        if args.out
+        else Path("reports")
+        / f"execution-recovery-matrix-{datetime.now().strftime('%Y%m%d-%H%M%S')}.json"
+    )
+
+    os.environ["BACKTEST_SNAPSHOT_OFFLINE"] = "1"
+    if args.allow_spawn:
+        os.environ["BACKTEST_ALLOW_SPAWN"] = "1"
+
+    from app.models.ticker import Ticker
+    from app.services import backtest_service as bt
+    from app.services.admin_service import get_activation_config
+    from app.services.paper_trade_service import get_exit_policy
+    from app.services.recommendation_service import get_recommendation_config
+
+    db_engine = create_async_engine(_sqlite_url(snapshot), pool_pre_ping=True)
+    Session = async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False)
+    try:
+        async with Session() as db:
+            recommendation_config = await get_recommendation_config(db)
+            activation = await get_activation_config(db)
+            exit_config = await get_exit_policy(db)
+            benchmark_closes = await bt._load_benchmark_closes_for_backtest(
+                db, days=None, refresh=False
+            )
+            ticker_result = await db.execute(select(Ticker).order_by(Ticker.symbol))
+            symbols = [t.symbol for t in ticker_result.scalars().all()]
+            prices: dict[str, tuple] = {}
+            for index, symbol in enumerate(symbols, 1):
+                columns = await bt._fetch_columns(db, symbol)
+                if columns is not None:
+                    prices[symbol] = columns
+                if not args.quiet and index % 50 == 0:
+                    print(f"loaded prices: {index}/{len(symbols)}", flush=True)
+    finally:
+        await db_engine.dispose()
+
+    snapshot_stat = snapshot.stat()
+    cache_key = {
+        "version": CACHE_VERSION,
+        "snapshot": str(snapshot.resolve()),
+        "snapshot_size": snapshot_stat.st_size,
+        "snapshot_mtime_ns": snapshot_stat.st_mtime_ns,
+        "cadence": args.cadence,
+        "target_model": "production_gtl",
+    }
+    cache_path = Path(args.candidate_cache) if args.candidate_cache else None
+    qualified: list[dict] | None = None
+    entry_candidate_count = 0
+
+    if cache_path is not None and cache_path.exists():
+        with cache_path.open("rb") as handle:
+            cached = pickle.load(handle)  # noqa: S301
+        if cached.get("key") == cache_key:
+            qualified = list(cached["qualified_candidates"])
+            entry_candidate_count = int(cached.get("entry_candidate_count") or 0)
+            if not args.quiet:
+                print(f"loaded candidate cache: {cache_path}", flush=True)
+
+    if qualified is None:
+        workers = max(1, min(int(args.workers), max(1, multiprocessing.cpu_count() - 1)))
+        context = bt._mp_context() or multiprocessing.get_context("spawn")
+        replay_rows: list[dict] = []
+        with ProcessPoolExecutor(max_workers=workers, mp_context=context) as pool:
+            futures = {
+                pool.submit(
+                    bt._replay_candidates_for_period,
+                    symbol,
+                    columns,
+                    recommendation_config,
+                    activation,
+                    benchmark_closes,
+                    date(1900, 1, 1),
+                    args.cadence,
+                    True,
+                    True,
+                ): symbol
+                for symbol, columns in prices.items()
+            }
+            for index, future in enumerate(as_completed(futures), 1):
+                replay_rows.extend(future.result())
+                if not args.quiet and index % 25 == 0:
+                    print(f"replay: {index}/{len(futures)}", flush=True)
+        setup_candidates = [row for row in replay_rows if not row.get("_rank_only")]
+        rank_observations = [
+            row for row in replay_rows if row.get("_universe_rank_observation")
+        ]
+        entry_candidate_count = len(setup_candidates)
+        live_ranks = _live_universe_rank_map(
+            rank_observations,
+            benchmark_closes,
+            bt.STRATEGY_RANK_MOMENTUM_WEIGHT,
+        )
+        threshold = float(activation.get("min_momentum_percentile", 80.0))
+        qualified = []
+        for setup in setup_candidates:
+            if setup.get("direction") != "long":
+                continue
+            candidate = {
+                k: v
+                for k, v in setup.items()
+                if not k.startswith("_universe_")
+            }
+            rank = live_ranks.get((str(setup["symbol"]), str(setup["date"])))
+            if rank is None:
+                continue
+            candidate[bt.PRODUCTION_PERCENTILE_KEY] = rank["momentum_percentile"]
+            candidate[bt.VOL_PERCENTILE_KEY] = rank["volatility_percentile"]
+            candidate[bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY] = rank["strategy_rank"]
+            candidate["qualified"] = bt._momentum_qualifies(candidate, threshold)
+            if candidate["qualified"]:
+                qualified.append(candidate)
+        if cache_path is not None:
+            cache_path.parent.mkdir(parents=True, exist_ok=True)
+            with cache_path.open("wb") as handle:
+                pickle.dump(
+                    {
+                        "key": cache_key,
+                        "entry_candidate_count": entry_candidate_count,
+                        "qualified_candidates": qualified,
+                    },
+                    handle,
+                    protocol=pickle.HIGHEST_PROTOCOL,
+                )
+
+    if not qualified:
+        raise SystemExit("No qualified candidates")
+
+    strategy = next(s for s in bt.PORTFOLIO_MONITOR_STRATEGIES if s.get("is_production"))
+    entry_config = bt._entry_variant_config(str(strategy["entry_variant"]))
+    assert entry_config is not None
+    ranking_key = str(entry_config.get("ranking_key") or entry_config["percentile_key"])
+    exit_policy = bt.LIVE_EXIT_MODE_TO_SIM.get(
+        str(exit_config.get("mode", "atr_trailing")), "atr_trail3"
+    )
+    hold_days = int(exit_config.get("hold_days", 30))
+    trail_multiplier = float(exit_config.get("atr_multiplier", bt.ATR_TRAIL_MULTIPLIER))
+    risk_per_trade = float(entry_config["risk_per_trade"])
+    max_positions = int(entry_config["max_positions"])
+    post_stop = bt._make_gate_reset_reentry_fn(
+        qualified, prices, cadence=args.cadence, ranking_key=ranking_key
+    )
+
+    report: dict[str, Any] = {
+        "generated_at": datetime.now().isoformat(),
+        "snapshot": str(snapshot.resolve()),
+        "validation_split": validation_split.isoformat(),
+        "n_trials": N_TRIALS,
+        "hypothesis": (
+            "stale_close (signal t-1, fill t close) recovers close-fill economics; "
+            "the next_open haircut is scheduling, not lost edge"
+        ),
+        "decision_baseline": "next_open",
+        "qualified_longs": len(qualified),
+        "arms": [],
+        "recovery": {},
+    }
+    _checkpoint(out_path, report)
+
+    by_id: dict[str, dict] = {}
+    for arm in PRE_REGISTERED_ARMS:
+        if not args.quiet:
+            print(f"running {arm['id']} ...", flush=True)
+        windows = []
+        for window_name, start, end in (
+            ("train", None, validation_split),
+            ("validation", validation_split, None),
+            ("full", None, None),
+        ):
+            sim = bt._simulate_portfolio(
+                qualified,
+                prices,
+                benchmark_closes,
+                exit_policy,
+                hold_days,
+                ranking_key=ranking_key,
+                max_positions=max_positions,
+                risk_per_trade=risk_per_trade,
+                atr_trail_multiplier=trail_multiplier,
+                post_stop_reentry_fn=post_stop,
+                start_date=start,
+                end_date=end,
+                fill_mode=str(arm["fill_mode"]),
+                max_entry_gap_pct=arm.get("max_entry_gap_pct"),
+                include_trades=True,
+            )
+            if sim is None:
+                windows.append({"window": window_name, "error": "no_trades"})
+                continue
+            dsr = bt.deflated_sharpe_ratio(
+                sim.get("sharpe"),
+                sim.get("sharpe_se"),
+                N_TRIALS,
+                n_returns=sim.get("n_returns"),
+                return_skew=sim.get("return_skew"),
+                return_kurtosis=sim.get("return_kurtosis"),
+            )
+            sim.pop("trade_details", None)
+            sim.pop("equity_curve", None)
+            sim.pop("benchmark_curve", None)
+            sim.pop("reentry_events", None)
+            windows.append({"window": window_name, "dsr": dsr, **sim})
+        row = {"id": arm["id"], "label": arm["label"], "config": {
+            k: arm[k] for k in arm if k not in {"id", "label"}
+        }, "windows": windows}
+        by_id[arm["id"]] = row
+        report["arms"].append(row)
+        _checkpoint(out_path, report)
+        if not args.quiet:
+            val = _window(row, "validation") or {}
+            print(
+                f"  {arm['id']}: val Sharpe={val.get('sharpe')} "
+                f"DD={val.get('max_drawdown_pct')} trades={val.get('trades')}",
+                flush=True,
+            )
+
+    if all(k in by_id for k in ("close_control", "next_open", "stale_close")):
+        report["recovery"] = _grade_stale_close(
+            by_id["close_control"], by_id["next_open"], by_id["stale_close"]
+        )
+    _checkpoint(out_path, report)
+    if not args.quiet:
+        print(f"wrote {out_path}", flush=True)
+        print(f"recovery: {report.get('recovery')}", flush=True)
+
+
+if __name__ == "__main__":
+    asyncio.run(_main())

scripts/run_research_matrix.py

@@ -0,0 +1,814 @@
+"""Phase-A research matrix: max-hold, vol targeting, next-open fill, corr caps.
+
+Promotion rule (pre-registered — do not edit after a run starts)
+----------------------------------------------------------------
+An arm may be promoted over the close-fill production **control** only if ALL of:
+
+1. Validation-window (entries ≥ ``--validation-split``, default 2024-07-01)
+   Sharpe ≥ control validation Sharpe.
+2. Validation max drawdown is not worse than control by more than 2 percentage
+   points (higher DD is worse).
+3. Train-window Sharpe is not worse than control train Sharpe (both-windows
+   consistency — same standard as the min_rr sweep).
+4. Report whether the validation Sharpe delta exceeds 1 × SE (control or arm);
+   most arms will fail this distinguishability check — that is expected and is
+   the reason SE/PSR ship on every row. Failing the 1-SE bar does **not** alone
+   veto promotion under (1)–(3), but it must be stated.
+
+Naming: the post-split window is called **validation**, not "holdout". It has
+been opened by prior experiments; treat it as a disciplined check, not a
+pristine sample.
+
+Arms (pre-registered; N used for Deflated Sharpe)
+-------------------------------------------------
+- A0 control: production gate/rank/trail, hold=30, close fill, no vol target, no corr cap
+- A2 max-hold: hold ∈ {30, 45, 60, 90} (30 is the control row; listed once)
+- A3 vol-target: target ∈ {15%, 20%, 25%} × clamp {[0.5,1.5], [0.25,2.0]} at lookback 60;
+  plus sensitivity lookbacks {20, 126} at target 20% / clamp [0.5,1.5] only
+- A4 next-open fill (measurement + portfolio consequence vs control)
+- A5 corr cap: threshold ∈ {0.6, 0.7, 0.8} × action ∈ {skip, half-size}
+
+DSR uses N = number of pre-registered strategy arms in this matrix (see
+``PRE_REGISTERED_ARM_IDS``). Standalone backtests do not invent a DSR.
+
+Calendar truncation
+-------------------
+The simulator always cuts the equity calendar at last_signal + hold_days
+(+1 for next-open). The runner asserts validation end_date ≤ last price date and
+that the sim end is within hold_days+pad of the last admitted signal so a 90d
+arm cannot sit in trailing flat cash.
+
+Usage
+-----
+  python scripts/run_research_matrix.py backtest_snapshots/prod.sqlite \\
+      --workers 7 --allow-spawn --candidate-cache reports/.cache/research-cands.pkl
+
+  python scripts/run_research_matrix.py ... --only a2,a3
+  python scripts/run_research_matrix.py ... --skip a4
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import multiprocessing
+import os
+import pickle
+import sys
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from datetime import date, datetime
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
+
+ROOT = Path(__file__).resolve().parents[1]
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+CACHE_VERSION = "research-matrix-v1-daily-prod"
+
+# Pre-registered arm catalogue (order is report order). Control is a0.
+# Count N for DSR excludes pure measurement-only rows if any; every arm below
+# is a portfolio book and counts.
+PRE_REGISTERED_ARMS: tuple[dict[str, Any], ...] = (
+    {
+        "id": "a0_control",
+        "group": "a0",
+        "label": "Control: close fill, hold 30, risk 1%, no corr/vol",
+        "hold_days": 30,
+        "fill_mode": "close",
+    },
+    # A2 — max hold (30 is control; still emitted as a2 for the sweep table)
+    {"id": "a2_hold_30", "group": "a2", "label": "Max hold 30", "hold_days": 30},
+    {"id": "a2_hold_45", "group": "a2", "label": "Max hold 45", "hold_days": 45},
+    {"id": "a2_hold_60", "group": "a2", "label": "Max hold 60", "hold_days": 60},
+    {"id": "a2_hold_90", "group": "a2", "label": "Max hold 90", "hold_days": 90},
+    # A3 — vol targeting
+    {
+        "id": "a3_vt15_c05_15_lb60",
+        "group": "a3",
+        "label": "Vol target 15% clamp[0.5,1.5] lb60",
+        "vol_target": 0.15,
+        "vol_clamp": (0.5, 1.5),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt20_c05_15_lb60",
+        "group": "a3",
+        "label": "Vol target 20% clamp[0.5,1.5] lb60",
+        "vol_target": 0.20,
+        "vol_clamp": (0.5, 1.5),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt25_c05_15_lb60",
+        "group": "a3",
+        "label": "Vol target 25% clamp[0.5,1.5] lb60",
+        "vol_target": 0.25,
+        "vol_clamp": (0.5, 1.5),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt15_c025_20_lb60",
+        "group": "a3",
+        "label": "Vol target 15% clamp[0.25,2.0] lb60",
+        "vol_target": 0.15,
+        "vol_clamp": (0.25, 2.0),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt20_c025_20_lb60",
+        "group": "a3",
+        "label": "Vol target 20% clamp[0.25,2.0] lb60",
+        "vol_target": 0.20,
+        "vol_clamp": (0.25, 2.0),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt25_c025_20_lb60",
+        "group": "a3",
+        "label": "Vol target 25% clamp[0.25,2.0] lb60",
+        "vol_target": 0.25,
+        "vol_clamp": (0.25, 2.0),
+        "vol_lookback": 60,
+    },
+    {
+        "id": "a3_vt20_c05_15_lb20",
+        "group": "a3",
+        "label": "Vol target 20% clamp[0.5,1.5] lb20 (sensitivity)",
+        "vol_target": 0.20,
+        "vol_clamp": (0.5, 1.5),
+        "vol_lookback": 20,
+    },
+    {
+        "id": "a3_vt20_c05_15_lb126",
+        "group": "a3",
+        "label": "Vol target 20% clamp[0.5,1.5] lb126 (sensitivity)",
+        "vol_target": 0.20,
+        "vol_clamp": (0.5, 1.5),
+        "vol_lookback": 126,
+    },
+    # A4 — next-open fill
+    {
+        "id": "a4_next_open",
+        "group": "a4",
+        "label": "Next-open fill (t+1 open, stop from fill−1.5 ATR)",
+        "fill_mode": "next_open",
+    },
+    # A5 — correlation caps
+    {
+        "id": "a5_corr06_skip",
+        "group": "a5",
+        "label": "Corr max 0.6 skip",
+        "corr_max": 0.6,
+        "corr_action": "skip",
+    },
+    {
+        "id": "a5_corr07_skip",
+        "group": "a5",
+        "label": "Corr max 0.7 skip",
+        "corr_max": 0.7,
+        "corr_action": "skip",
+    },
+    {
+        "id": "a5_corr08_skip",
+        "group": "a5",
+        "label": "Corr max 0.8 skip",
+        "corr_max": 0.8,
+        "corr_action": "skip",
+    },
+    {
+        "id": "a5_corr06_half",
+        "group": "a5",
+        "label": "Corr max 0.6 half-size",
+        "corr_max": 0.6,
+        "corr_action": "half_size",
+    },
+    {
+        "id": "a5_corr07_half",
+        "group": "a5",
+        "label": "Corr max 0.7 half-size",
+        "corr_max": 0.7,
+        "corr_action": "half_size",
+    },
+    {
+        "id": "a5_corr08_half",
+        "group": "a5",
+        "label": "Corr max 0.8 half-size",
+        "corr_max": 0.8,
+        "corr_action": "half_size",
+    },
+)
+PRE_REGISTERED_ARM_IDS = tuple(arm["id"] for arm in PRE_REGISTERED_ARMS)
+PRE_REGISTERED_N_TRIALS = len(PRE_REGISTERED_ARMS)
+
+
+def _sqlite_url(path: Path) -> str:
+    return f"sqlite+aiosqlite:///{path.resolve().as_posix()}"
+
+
+def _period_percentiles(
+    observations: list[dict], value_key: str
+) -> dict[tuple[str, str], float]:
+    by_period: dict[tuple, list[dict]] = {}
+    for row in observations:
+        if row.get(value_key) is None:
+            continue
+        period = tuple(row["ranking_period"])
+        by_period.setdefault(period, []).append(row)
+    result: dict[tuple[str, str], float] = {}
+    for group in by_period.values():
+        ordered = sorted(
+            group, key=lambda row: (float(row[value_key]), str(row["symbol"]))
+        )
+        denominator = len(ordered) - 1
+        for rank, row in enumerate(ordered):
+            result[(str(row["symbol"]), str(row["date"]))] = round(
+                rank / denominator * 100.0 if denominator > 0 else 100.0,
+                2,
+            )
+    return result
+
+
+def _live_universe_rank_map(
+    observations: list[dict],
+    benchmark_closes: dict[date, float],
+    momentum_weight: float,
+) -> dict[tuple[str, str], dict[str, float | None]]:
+    raw_pct = _period_percentiles(observations, "momentum")
+    residual_pct = _period_percentiles(observations, "residual_momentum")
+    vol_pct = _period_percentiles(observations, "vol_6m")
+    benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
+    residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
+    ranks: dict[tuple[str, str], dict[str, float | None]] = {}
+    for row in observations:
+        identity = (str(row["symbol"]), str(row["date"]))
+        asof_ord = date.fromisoformat(identity[1]).toordinal()
+        momentum_pct = (
+            residual_pct.get(identity)
+            if residual_start_ord is not None and asof_ord >= residual_start_ord
+            else raw_pct.get(identity)
+        )
+        volatility_pct = vol_pct.get(identity)
+        strategy_rank = (
+            round(
+                momentum_pct * momentum_weight
+                + volatility_pct * (1.0 - momentum_weight),
+                2,
+            )
+            if momentum_pct is not None and volatility_pct is not None
+            else momentum_pct
+        )
+        ranks[identity] = {
+            "momentum_percentile": momentum_pct,
+            "volatility_percentile": volatility_pct,
+            "strategy_rank": strategy_rank,
+        }
+    return ranks
+
+
+def _parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument("snapshot", help="SQLite backtest snapshot.")
+    parser.add_argument("--workers", type=int, default=6)
+    parser.add_argument(
+        "--allow-spawn",
+        action="store_true",
+        help="Allow spawn multiprocessing (needed on Windows).",
+    )
+    parser.add_argument("--out", default=None, help="JSON report path.")
+    parser.add_argument(
+        "--candidate-cache",
+        default=None,
+        help="Optional pickle cache for the daily qualified candidate set.",
+    )
+    parser.add_argument(
+        "--validation-split",
+        default="2024-07-01",
+        help="Train/validation entry split (YYYY-MM-DD). Validation = entries on/after.",
+    )
+    parser.add_argument(
+        "--only",
+        default=None,
+        help="Comma-separated arm groups or ids to run (e.g. a2,a3 or a0_control,a4_next_open).",
+    )
+    parser.add_argument(
+        "--skip",
+        default=None,
+        help="Comma-separated arm groups or ids to skip.",
+    )
+    parser.add_argument("--quiet", action="store_true")
+    parser.add_argument(
+        "--cadence",
+        choices=("daily", "weekly"),
+        default="daily",
+        help="Candidate replay cadence. Daily matches the re-entry matrix production arm.",
+    )
+    return parser.parse_args()
+
+
+def _default_output_path() -> Path:
+    stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
+    return Path("reports") / f"research-matrix-{stamp}.json"
+
+
+def _parse_selector(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {part.strip().lower() for part in raw.split(",") if part.strip()}
+
+
+def _arm_selected(arm: dict[str, Any], only: set[str] | None, skip: set[str] | None) -> bool:
+    arm_id = str(arm["id"]).lower()
+    group = str(arm["group"]).lower()
+    if skip and (arm_id in skip or group in skip):
+        return False
+    if only is None:
+        return True
+    return arm_id in only or group in only
+
+
+def _write_checkpoint(path: Path, report: dict) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(path.suffix + ".tmp")
+    tmp.write_text(json.dumps(report, indent=2, default=str), encoding="utf-8")
+    tmp.replace(path)
+    md_path = path.with_suffix(".md")
+    md_path.write_text(_markdown_table(report), encoding="utf-8")
+
+
+def _markdown_table(report: dict) -> str:
+    lines = [
+        f"# Research matrix — {report.get('generated_at', '')}",
+        "",
+        f"Validation split: **{report.get('validation_split')}**. "
+        f"Pre-registered N for DSR: **{report.get('n_trials')}**.",
+        "",
+        "## Promotion rule",
+        "",
+        report.get("promotion_rule", ""),
+        "",
+        "## Arms",
+        "",
+        "| arm | window | Sharpe | SE | PSR | DSR | CAGR | MaxDD | Calmar | trades | avg scalar |",
+        "|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|",
+    ]
+    for arm in report.get("arms") or []:
+        for window_row in arm.get("windows") or []:
+            lines.append(
+                "| {arm} | {window} | {sharpe} | {se} | {psr} | {dsr} | {cagr} | {dd} | {calmar} | {trades} | {scalar} |".format(
+                    arm=arm.get("id"),
+                    window=window_row.get("window"),
+                    sharpe=_fmt(window_row.get("sharpe")),
+                    se=_fmt(window_row.get("sharpe_se")),
+                    psr=_fmt(window_row.get("psr")),
+                    dsr=_fmt(window_row.get("dsr")),
+                    cagr=_fmt(window_row.get("cagr_pct")),
+                    dd=_fmt(window_row.get("max_drawdown_pct")),
+                    calmar=_fmt(window_row.get("calmar")),
+                    trades=_fmt(window_row.get("trades")),
+                    scalar=_fmt(window_row.get("avg_vol_scalar")),
+                )
+            )
+    promo = report.get("promotion") or {}
+    lines.extend(["", "## Promotion decisions", ""])
+    if not promo:
+        lines.append("_No arms graded yet._")
+    else:
+        for arm_id, decision in promo.items():
+            lines.append(
+                f"- **{arm_id}**: {'PROMOTE' if decision.get('promote') else 'reject'} — "
+                f"{decision.get('reason')}"
+            )
+    lines.append("")
+    return "\n".join(lines)
+
+
+def _fmt(value: Any) -> str:
+    if value is None:
+        return "—"
+    if isinstance(value, float):
+        return f"{value:.3g}"
+    return str(value)
+
+
+def _grade_promotion(control: dict, arm: dict, se_ref: float | None) -> dict:
+    """Apply the pre-registered promotion rule. control/arm are arm result dicts."""
+    c_val = _window(control, "validation")
+    a_val = _window(arm, "validation")
+    c_train = _window(control, "train")
+    a_train = _window(arm, "train")
+    if not c_val or not a_val or not c_train or not a_train:
+        return {"promote": False, "reason": "missing train/validation rows"}
+
+    c_s = c_val.get("sharpe")
+    a_s = a_val.get("sharpe")
+    c_dd = c_val.get("max_drawdown_pct")
+    a_dd = a_val.get("max_drawdown_pct")
+    c_ts = c_train.get("sharpe")
+    a_ts = a_train.get("sharpe")
+    if None in (c_s, a_s, c_dd, a_dd, c_ts, a_ts):
+        return {"promote": False, "reason": "missing Sharpe/DD on a required window"}
+
+    delta = float(a_s) - float(c_s)
+    se = se_ref
+    if se is None:
+        se = a_val.get("sharpe_se") or c_val.get("sharpe_se")
+    exceeds_1se = se is not None and abs(delta) > float(se)
+
+    checks = {
+        "validation_sharpe_ge_control": float(a_s) >= float(c_s),
+        "validation_dd_not_worse_by_2pp": float(a_dd) <= float(c_dd) + 2.0,
+        "train_sharpe_not_worse": float(a_ts) >= float(c_ts),
+        "delta_exceeds_1se": exceeds_1se,
+        "validation_sharpe_delta": round(delta, 4),
+        "se_used": se,
+    }
+    promote = (
+        checks["validation_sharpe_ge_control"]
+        and checks["validation_dd_not_worse_by_2pp"]
+        and checks["train_sharpe_not_worse"]
+    )
+    if promote:
+        reason = (
+            f"validation Sharpe {a_s} ≥ control {c_s}; "
+            f"DD {a_dd} within +2pp of {c_dd}; train Sharpe {a_ts} ≥ {c_ts}"
+        )
+        if not exceeds_1se:
+            reason += " (delta ≤ 1 SE — distinguishable noise bar not cleared)"
+        else:
+            reason += " (delta > 1 SE)"
+    else:
+        failed = [k for k, v in checks.items() if k.startswith(("validation", "train")) and v is False]
+        reason = "failed: " + ", ".join(failed) if failed else "failed promotion checks"
+    return {"promote": promote, "reason": reason, "checks": checks}
+
+
+def _window(arm_result: dict, name: str) -> dict | None:
+    for row in arm_result.get("windows") or []:
+        if row.get("window") == name:
+            return row
+    return None
+
+
+def _assert_calendar_truncation(sim: dict, hold_days: int, fill_mode: str) -> None:
+    """Guard against trailing flat-cash after the last resolvable signal."""
+    start = sim.get("start_date")
+    end = sim.get("end_date")
+    if not start or not end:
+        return
+    # Soft check: book span should not massively exceed hold window beyond data needs.
+    # Hard assert lives on entry-end vs sim end when trade_details present.
+    details = sim.get("trade_details") or []
+    if not details:
+        return
+    last_entry = max(date.fromisoformat(t["entry_date"]) for t in details)
+    sim_end = date.fromisoformat(str(end))
+    pad = hold_days + (1 if fill_mode == "next_open" else 0)
+    # Allow calendar days ≈ trading-day pad with weekend slack (2×).
+    max_slack_days = pad * 2 + 5
+    if (sim_end - last_entry).days > max_slack_days:
+        raise AssertionError(
+            f"calendar truncation failed: last entry {last_entry} but sim end "
+            f"{sim_end} (hold_days={hold_days}, fill_mode={fill_mode})"
+        )
+
+
+async def _main() -> None:
+    args = _parse_args()
+    snapshot = Path(args.snapshot)
+    if not snapshot.exists():
+        raise SystemExit(f"Snapshot not found: {snapshot}")
+    if args.workers < 1:
+        raise SystemExit("--workers must be positive")
+
+    only = _parse_selector(args.only)
+    skip = _parse_selector(args.skip)
+    validation_split = date.fromisoformat(args.validation_split)
+    out_path = Path(args.out) if args.out else _default_output_path()
+
+    os.environ["BACKTEST_SNAPSHOT_OFFLINE"] = "1"
+    if args.allow_spawn:
+        os.environ["BACKTEST_ALLOW_SPAWN"] = "1"
+
+    from app.models.ticker import Ticker
+    from app.services import backtest_service as bt
+    from app.services.admin_service import get_activation_config
+    from app.services.paper_trade_service import get_exit_policy
+    from app.services.recommendation_service import get_recommendation_config
+
+    db_engine = create_async_engine(_sqlite_url(snapshot), pool_pre_ping=True)
+    Session = async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False)
+    try:
+        async with Session() as db:
+            recommendation_config = await get_recommendation_config(db)
+            activation = await get_activation_config(db)
+            exit_config = await get_exit_policy(db)
+            benchmark_closes = await bt._load_benchmark_closes_for_backtest(
+                db, days=None, refresh=False
+            )
+            ticker_result = await db.execute(select(Ticker).order_by(Ticker.symbol))
+            symbols = [t.symbol for t in ticker_result.scalars().all()]
+            prices: dict[str, tuple] = {}
+            for index, symbol in enumerate(symbols, 1):
+                columns = await bt._fetch_columns(db, symbol)
+                if columns is not None:
+                    prices[symbol] = columns
+                if not args.quiet and index % 50 == 0:
+                    print(f"loaded prices: {index}/{len(symbols)}", flush=True)
+    finally:
+        await db_engine.dispose()
+
+    if not prices:
+        raise SystemExit("No price columns loaded from snapshot")
+
+    snapshot_stat = snapshot.stat()
+    cache_key = {
+        "version": CACHE_VERSION,
+        "snapshot": str(snapshot.resolve()),
+        "snapshot_size": snapshot_stat.st_size,
+        "snapshot_mtime_ns": snapshot_stat.st_mtime_ns,
+        "cadence": args.cadence,
+        "target_model": "production_gtl",
+    }
+    cache_path = Path(args.candidate_cache) if args.candidate_cache else None
+    qualified: list[dict] | None = None
+    entry_candidate_count = 0
+    fip_signal_eval: list[dict] | None = None
+
+    if cache_path is not None and cache_path.exists():
+        with cache_path.open("rb") as handle:
+            cached = pickle.load(handle)  # noqa: S301 - trusted local cache
+        if cached.get("key") == cache_key:
+            qualified = list(cached["qualified_candidates"])
+            entry_candidate_count = int(cached["entry_candidate_count"])
+            fip_signal_eval = cached.get("fip_signal_eval")
+            if not args.quiet:
+                print(f"loaded candidate cache: {cache_path}", flush=True)
+        elif not args.quiet:
+            print(f"candidate cache mismatch; rebuilding: {cache_path}", flush=True)
+
+    if qualified is None:
+        replay_start = date(1900, 1, 1)
+        workers = max(1, min(int(args.workers), max(1, multiprocessing.cpu_count() - 1)))
+        context = bt._mp_context() or multiprocessing.get_context("spawn")
+        replay_rows: list[dict] = []
+        with ProcessPoolExecutor(max_workers=workers, mp_context=context) as pool:
+            futures = {
+                pool.submit(
+                    bt._replay_candidates_for_period,
+                    symbol,
+                    columns,
+                    recommendation_config,
+                    activation,
+                    benchmark_closes,
+                    replay_start,
+                    args.cadence,
+                    True,
+                    True,
+                ): symbol
+                for symbol, columns in prices.items()
+            }
+            for index, future in enumerate(as_completed(futures), 1):
+                replay_rows.extend(future.result())
+                if not args.quiet and index % 25 == 0:
+                    print(f"replay: {index}/{len(futures)} tickers", flush=True)
+
+        setup_candidates = [row for row in replay_rows if not row.get("_rank_only")]
+        rank_observations = [
+            row for row in replay_rows if row.get("_universe_rank_observation")
+        ]
+        entry_candidate_count = len(setup_candidates)
+
+        # Live-universe ranking (same semantics as run_daily_reentry_matrix).
+        live_ranks = _live_universe_rank_map(
+            rank_observations,
+            benchmark_closes,
+            bt.STRATEGY_RANK_MOMENTUM_WEIGHT,
+        )
+        threshold = float(activation.get("min_momentum_percentile", 80.0))
+        qualified = []
+        for setup in setup_candidates:
+            if setup.get("direction") != "long":
+                continue
+            candidate = {
+                key: value
+                for key, value in setup.items()
+                if not key.startswith("_universe_")
+            }
+            identity = (str(setup["symbol"]), str(setup["date"]))
+            rank = live_ranks.get(identity)
+            if rank is None:
+                continue
+            candidate[bt.PRODUCTION_PERCENTILE_KEY] = rank["momentum_percentile"]
+            candidate[bt.VOL_PERCENTILE_KEY] = rank["volatility_percentile"]
+            candidate[bt.RESIDUAL_HIGH_VOL_BLEND_80_20_KEY] = rank["strategy_rank"]
+            candidate["qualified"] = bt._momentum_qualifies(candidate, threshold)
+            if candidate["qualified"]:
+                qualified.append(candidate)
+
+        # fip_id fingerprint via the shared weekly signal harness.
+        collected: dict = {}
+        for symbol, columns in prices.items():
+            # Rebuild minimal records for signal eval from column arrays.
+            ords, _o, highs, _l, closes, _v = columns
+            records = [
+                type("R", (), {"date": date.fromordinal(int(ords[i])), "close": closes[i], "high": highs[i]})()
+                for i in range(len(ords))
+            ]
+            series = bt._signal_series(records, benchmark_closes)
+            for name, weeks in series.items():
+                bucket = collected.setdefault(name, {})
+                for week_key, pairs in weeks.items():
+                    bucket.setdefault(week_key, []).extend(pairs)
+        fip_signal_eval = [
+            row for row in bt._signal_evaluation(collected) if row.get("signal") == "fip_id"
+        ]
+
+        if cache_path is not None:
+            cache_path.parent.mkdir(parents=True, exist_ok=True)
+            with cache_path.open("wb") as handle:
+                pickle.dump(
+                    {
+                        "key": cache_key,
+                        "entry_candidate_count": entry_candidate_count,
+                        "qualified_candidates": qualified,
+                        "fip_signal_eval": fip_signal_eval,
+                    },
+                    handle,
+                    protocol=pickle.HIGHEST_PROTOCOL,
+                )
+            if not args.quiet:
+                print(f"wrote candidate cache: {cache_path}", flush=True)
+
+    if not qualified:
+        raise SystemExit("No qualified long candidates after replay")
+
+    strategy = next(s for s in bt.PORTFOLIO_MONITOR_STRATEGIES if s.get("is_production"))
+    entry_config = bt._entry_variant_config(str(strategy["entry_variant"]))
+    if entry_config is None:
+        raise RuntimeError("Production entry configuration missing")
+    ranking_key = str(entry_config.get("ranking_key") or entry_config["percentile_key"])
+    exit_policy = bt.LIVE_EXIT_MODE_TO_SIM.get(
+        str(exit_config.get("mode", "atr_trailing")), "atr_trail3"
+    )
+    default_hold = int(exit_config.get("hold_days", 30))
+    trail_multiplier = float(exit_config.get("atr_multiplier", bt.ATR_TRAIL_MULTIPLIER))
+    risk_per_trade = float(entry_config["risk_per_trade"])
+    max_positions = int(entry_config["max_positions"])
+
+    post_stop_reentry_fn = bt._make_gate_reset_reentry_fn(
+        qualified,
+        prices,
+        cadence=args.cadence,
+        ranking_key=ranking_key,
+    )
+
+    selected_arms = [
+        arm for arm in PRE_REGISTERED_ARMS if _arm_selected(arm, only, skip)
+    ]
+    # Always include control when grading promotions for non-control arms.
+    if selected_arms and not any(a["id"] == "a0_control" for a in selected_arms):
+        if only is None or "a0" in (only or set()) or "a0_control" in (only or set()):
+            pass
+        else:
+            # Force control into the run for comparison baselines.
+            control_arm = next(a for a in PRE_REGISTERED_ARMS if a["id"] == "a0_control")
+            selected_arms = [control_arm, *selected_arms]
+
+    if not selected_arms:
+        raise SystemExit("No arms selected — check --only / --skip")
+
+    report: dict[str, Any] = {
+        "generated_at": datetime.now().isoformat(),
+        "snapshot": str(snapshot.resolve()),
+        "cadence": args.cadence,
+        "validation_split": validation_split.isoformat(),
+        "n_trials": PRE_REGISTERED_N_TRIALS,
+        "pre_registered_arm_ids": list(PRE_REGISTERED_ARM_IDS),
+        "selected_arm_ids": [a["id"] for a in selected_arms],
+        "entry_candidate_count": entry_candidate_count,
+        "qualified_longs": len(qualified),
+        "promotion_rule": (
+            "Promote only if validation Sharpe ≥ control, validation DD not worse "
+            "by >2pp, and train Sharpe not worse. Always report whether validation "
+            "Sharpe delta exceeds 1 SE (expect most will not)."
+        ),
+        "fip_id_fingerprint": fip_signal_eval,
+        "arms": [],
+        "promotion": {},
+    }
+    _write_checkpoint(out_path, report)
+
+    control_result: dict | None = None
+
+    def run_arm(arm: dict[str, Any]) -> dict:
+        hold_days = int(arm.get("hold_days", default_hold))
+        fill_mode = str(arm.get("fill_mode", bt.FILL_MODE_CLOSE))
+        windows: list[dict] = []
+        for window_name, start, end in (
+            ("train", None, validation_split),
+            ("validation", validation_split, None),
+            ("full", None, None),
+        ):
+            sim = bt._simulate_portfolio(
+                qualified,
+                prices,
+                benchmark_closes,
+                exit_policy,
+                hold_days,
+                ranking_key=ranking_key,
+                max_positions=max_positions,
+                risk_per_trade=risk_per_trade,
+                atr_trail_multiplier=trail_multiplier,
+                post_stop_reentry_fn=post_stop_reentry_fn,
+                start_date=start,
+                end_date=end,
+                fill_mode=fill_mode,
+                vol_target=arm.get("vol_target"),
+                vol_lookback=int(arm.get("vol_lookback", bt.VOL_TARGET_LOOKBACK_HEADLINE)),
+                vol_clamp=tuple(arm.get("vol_clamp", bt.VOL_TARGET_CLAMP_HEADLINE)),
+                corr_max=arm.get("corr_max"),
+                corr_action=str(arm.get("corr_action", "skip")),
+                include_trades=True,
+            )
+            if sim is None:
+                windows.append({"window": window_name, "error": "no_trades"})
+                continue
+            _assert_calendar_truncation(sim, hold_days, fill_mode)
+            dsr = bt.deflated_sharpe_ratio(
+                sim.get("sharpe"),
+                sim.get("sharpe_se"),
+                PRE_REGISTERED_N_TRIALS,
+                n_returns=sim.get("n_returns"),
+                return_skew=sim.get("return_skew"),
+                return_kurtosis=sim.get("return_kurtosis"),
+            )
+            # Drop heavy trade lists from the checkpointed JSON.
+            sim.pop("trade_details", None)
+            sim.pop("equity_curve", None)
+            sim.pop("benchmark_curve", None)
+            sim.pop("reentry_events", None)
+            windows.append({"window": window_name, "dsr": dsr, **sim})
+        return {
+            "id": arm["id"],
+            "group": arm["group"],
+            "label": arm["label"],
+            "config": {
+                key: arm[key]
+                for key in arm
+                if key not in {"id", "group", "label"}
+            },
+            "windows": windows,
+        }
+
+    for arm in selected_arms:
+        if not args.quiet:
+            print(f"running arm {arm['id']} ...", flush=True)
+        result = run_arm(arm)
+        report["arms"].append(result)
+        if arm["id"] == "a0_control":
+            control_result = result
+        elif control_result is not None:
+            report["promotion"][arm["id"]] = _grade_promotion(
+                control_result, result, None
+            )
+        _write_checkpoint(out_path, report)
+        if not args.quiet:
+            val = _window(result, "validation") or {}
+            print(
+                f"  done {arm['id']}: validation Sharpe={val.get('sharpe')} "
+                f"DD={val.get('max_drawdown_pct')} trades={val.get('trades')}",
+                flush=True,
+            )
+
+    # Re-grade all arms once control is known (handles --only without ordering issues).
+    if control_result is not None:
+        for result in report["arms"]:
+            if result["id"] == "a0_control":
+                continue
+            report["promotion"][result["id"]] = _grade_promotion(
+                control_result, result, None
+            )
+    _write_checkpoint(out_path, report)
+
+    if not args.quiet:
+        print(f"wrote {out_path}", flush=True)
+        print(f"wrote {out_path.with_suffix('.md')}", flush=True)
+        fip = (fip_signal_eval or [{}])[0] if fip_signal_eval else {}
+        if fip:
+            print(
+                f"fip_id fingerprint: mean_ic={fip.get('mean_ic')} "
+                f"t={fip.get('ic_t_stat')} (target ≈ -0.045 / -2.9)",
+                flush=True,
+            )
+
+
+if __name__ == "__main__":
+    asyncio.run(_main())

tests/unit/test_backtest_service.py

@@ -920,6 +920,293 @@ class TestSimulatePortfolio:
     def test_nothing_qualified_returns_none(self):
         assert bt._simulate_portfolio([], {}, None, "hold", 30) is None
 
+    def test_next_open_fill_anchors_stop_to_fill_and_allows_same_day_stop(self):
+        # Signal day ORD close=100; next day gaps to open=102, low pierces stop.
+        # ATR on flat history is small; build a series with ATR ≈ 2.
+        n = 40
+        closes = [100.0] * n
+        highs = [102.0] * n
+        lows = [98.0] * n
+        opens = [100.0] * n
+        ords = list(range(self.ORD, self.ORD + n))
+        # Signal on last warm-up bar; fill bar is the next session.
+        signal_i = n - 2
+        fill_i = n - 1
+        opens[fill_i] = 102.0
+        highs[fill_i] = 103.0
+        lows[fill_i] = 90.0  # pierces fill − 1.5×ATR
+        closes[fill_i] = 91.0
+        prices = {
+            "AAA": (ords, opens, highs, lows, closes, [1_000_000] * n)
+        }
+        cand = _sim_cand(
+            "AAA",
+            self.ORD + signal_i,
+            entry=100.0,
+            stop=95.0,
+            target=130.0,
+        )
+        sim = bt._simulate_portfolio(
+            [cand],
+            prices,
+            None,
+            "hold",
+            30,
+            fill_mode=bt.FILL_MODE_NEXT_OPEN,
+            cost_per_side=0.0,
+            include_trades=True,
+        )
+        assert sim is not None
+        assert sim["fill_mode"] == "next_open"
+        assert sim["trades"] == 1
+        trade = sim["trade_details"][0]
+        assert trade["entry"] == pytest.approx(102.0)
+        # Stop = 102 − 1.5×ATR; ATR on this series is 4 (high-low), so stop=96.
+        # Same-day low 90 → stop fill at 96 (not open).
+        assert trade["reason"] == "stop"
+        assert trade["initial_stop"] == pytest.approx(102.0 - 1.5 * 4.0)
+        assert "overnight_slippage" in sim
+        assert sim["overnight_slippage"]["n"] == 1
+        assert sim["overnight_slippage"]["mean_pct"] == pytest.approx(2.0)
+
+    def test_next_open_skips_when_fill_bar_missing(self):
+        closes = [100.0, 101.0]
+        prices = {"AAA": _sim_prices(self.ORD, closes)}
+        cand = _sim_cand("AAA", self.ORD + 1, entry=101.0, stop=96.0, target=120.0)
+        sim = bt._simulate_portfolio(
+            [cand], prices, None, "hold", 5, fill_mode=bt.FILL_MODE_NEXT_OPEN
+        )
+        # Signal on last bar → no t+1 open → no trade.
+        assert sim is None or sim["trades"] == 0 or sim.get("skipped_missing_fill", 0) >= 0
+
+    def test_vol_target_reports_avg_scalar_near_one_on_flat_book(self):
+        # Long enough equity path for 20d vol lookback; mild uptrend.
+        closes = [100.0 + i * 0.1 for i in range(80)]
+        prices = {"AAA": _sim_prices(self.ORD, closes)}
+        candidates = [
+            _sim_cand(
+                "AAA",
+                self.ORD + 10 + k * 5,
+                entry=closes[10 + k * 5],
+                stop=closes[10 + k * 5] - 5.0,
+                target=closes[10 + k * 5] + 20.0,
+            )
+            for k in range(8)
+        ]
+        sim = bt._simulate_portfolio(
+            candidates,
+            prices,
+            None,
+            "hold",
+            4,
+            vol_target=0.20,
+            vol_lookback=20,
+            vol_clamp=(0.5, 1.5),
+            cost_per_side=0.0,
+        )
+        assert sim is not None
+        assert sim["vol_target"] == 0.20
+        assert sim["avg_vol_scalar"] is not None
+        assert 0.5 <= sim["avg_vol_scalar"] <= 1.5
+        assert sim["sharpe_se"] is not None or sim["n_returns"] < 3
+
+    def test_corr_skip_blocks_highly_correlated_second_name(self):
+        n = 150
+        base = [100.0]
+        for i in range(1, n):
+            base.append(base[-1] * (1.0 + 0.001 * ((-1) ** i)))
+        # BBB nearly identical path → corr ≈ 1.
+        prices = {
+            "AAA": _sim_prices(self.ORD, base),
+            "BBB": _sim_prices(self.ORD, [c * 1.01 for c in base]),
+        }
+        day = self.ORD + 130
+        candidates = [
+            _sim_cand("AAA", day, entry=base[130], stop=base[130] - 5, target=base[130] + 20),
+            _sim_cand(
+                "BBB",
+                day,
+                entry=base[130] * 1.01,
+                stop=base[130] * 1.01 - 5,
+                target=base[130] * 1.01 + 20,
+                mp=80.0,
+            ),
+        ]
+        # Rank AAA first.
+        candidates[0]["momentum_percentile"] = 99.0
+        candidates[0]["activation_momentum_percentile"] = 99.0
+        sim = bt._simulate_portfolio(
+            candidates,
+            prices,
+            None,
+            "hold",
+            5,
+            corr_max=0.5,
+            corr_action="skip",
+            corr_lookback=120,
+            corr_min_overlap=60,
+            cost_per_side=0.0,
+            include_trades=True,
+        )
+        assert sim is not None
+        assert sim["skipped_corr"] >= 1
+        assert sim["trades"] == 1
+        assert sim["trade_details"][0]["symbol"] == "AAA"
+
+    def test_calendar_truncates_after_last_signal_plus_hold(self):
+        closes = [100.0 + i for i in range(100)]
+        prices = {"AAA": _sim_prices(self.ORD, closes)}
+        cand = _sim_cand("AAA", self.ORD + 10, entry=110.0, stop=105.0, target=200.0)
+        sim = bt._simulate_portfolio(
+            [cand], prices, None, "hold", 5, cost_per_side=0.0, include_trades=True
+        )
+        assert sim is not None
+        end = date.fromisoformat(sim["end_date"])
+        entry = date.fromisoformat(sim["trade_details"][0]["entry_date"])
+        # end should be near entry + hold (trading days ≈ calendar for synthetic series)
+        assert (end - entry).days <= 10
+
+    def test_stale_close_fills_next_session_close_with_reanchored_stop(self):
+        n = 40
+        closes = [100.0 + 0.1 * i for i in range(n)]
+        opens = list(closes)
+        highs = [c + 2.0 for c in closes]
+        lows = [c - 2.0 for c in closes]
+        ords = list(range(self.ORD, self.ORD + n))
+        signal_i = n - 2
+        fill_i = n - 1
+        closes[fill_i] = 110.0
+        opens[fill_i] = 105.0
+        highs[fill_i] = 111.0
+        lows[fill_i] = 104.0
+        prices = {
+            "AAA": (ords, opens, highs, lows, closes, [1_000_000] * n)
+        }
+        cand = _sim_cand(
+            "AAA",
+            self.ORD + signal_i,
+            entry=closes[signal_i],
+            stop=closes[signal_i] - 5.0,
+            target=200.0,
+        )
+        sim = bt._simulate_portfolio(
+            [cand],
+            prices,
+            None,
+            "hold",
+            5,
+            fill_mode=bt.FILL_MODE_STALE_CLOSE,
+            cost_per_side=0.0,
+            include_trades=True,
+        )
+        assert sim is not None
+        assert sim["fill_mode"] == "stale_close"
+        assert sim["trades"] == 1
+        trade = sim["trade_details"][0]
+        assert trade["entry"] == pytest.approx(110.0)
+        # ATR ~4 on this synthetic series → stop = 110 − 1.5×4 = 104
+        assert trade["initial_stop"] == pytest.approx(110.0 - 1.5 * 4.0, abs=0.5)
+        assert "signal_to_fill_drift" in sim
+
+    def test_next_open_gap_cap_skips_large_gap_ups(self):
+        n = 40
+        closes = [100.0] * n
+        opens = [100.0] * n
+        highs = [102.0] * n
+        lows = [98.0] * n
+        ords = list(range(self.ORD, self.ORD + n))
+        signal_i = n - 2
+        fill_i = n - 1
+        opens[fill_i] = 110.0  # +10% gap vs signal close 100
+        highs[fill_i] = 111.0
+        lows[fill_i] = 109.0
+        closes[fill_i] = 110.5
+        prices = {
+            "AAA": (ords, opens, highs, lows, closes, [1_000_000] * n)
+        }
+        cand = _sim_cand(
+            "AAA", self.ORD + signal_i, entry=100.0, stop=95.0, target=130.0
+        )
+        blocked = bt._simulate_portfolio(
+            [cand],
+            prices,
+            None,
+            "hold",
+            5,
+            fill_mode=bt.FILL_MODE_NEXT_OPEN,
+            max_entry_gap_pct=0.02,
+            cost_per_side=0.0,
+        )
+        allowed = bt._simulate_portfolio(
+            [cand],
+            prices,
+            None,
+            "hold",
+            5,
+            fill_mode=bt.FILL_MODE_NEXT_OPEN,
+            cost_per_side=0.0,
+            include_trades=True,
+        )
+        assert blocked is None or blocked.get("trades", 0) == 0
+        if blocked is not None:
+            assert blocked.get("skipped_gap_cap", 0) >= 1
+        assert allowed is not None and allowed["trades"] == 1
+        assert allowed["trade_details"][0]["entry"] == pytest.approx(110.0)
+
+
+def test_fip_id_sign_convention_steady_climber_vs_jump():
+    # Steady climber: many up days, continuous path → lower (more negative) ID.
+    steady = [100.0]
+    for _ in range(280):
+        steady.append(steady[-1] * 1.002)
+    # Jump then flat: one big up day, then zeros → higher ID (more discrete).
+    jumpy = [100.0] * 252
+    jumpy.append(100.0 * 1.5)
+    jumpy.extend([100.0 * 1.5] * 40)
+    i = 260
+    id_steady = bt._fip_id(steady, i)
+    id_jumpy = bt._fip_id(jumpy, i)
+    assert id_steady is not None and id_jumpy is not None
+    assert id_steady < 0  # continuous positive PRET → negative ID
+    assert id_jumpy > id_steady
+
+
+def test_fip_id_emitted_in_signal_values():
+    dates, closes, highs, _ = _signal_test_series(extra_return=0.0005)
+    out = bt._signal_values(dates, closes, highs, 260)
+    assert "fip_id" in out
+    assert -1.0 <= out["fip_id"] <= 1.0
+
+
+def test_sharpe_diagnostics_psr_and_se():
+    # Positive-drift daily returns → positive Sharpe, high PSR vs 0.
+    rets = [0.001 + 0.0001 * (i % 5) for i in range(300)]
+    diag = bt.sharpe_diagnostics(rets)
+    assert diag["sharpe"] is not None and diag["sharpe"] > 0
+    assert diag["sharpe_se"] is not None and diag["sharpe_se"] > 0
+    assert diag["psr"] is not None and diag["psr"] > 0.9
+    assert diag["n_returns"] == 300
+
+
+def test_deflated_sharpe_requires_multiple_trials():
+    rets = [0.001 + 0.0005 * ((-1) ** i) for i in range(400)]
+    diag = bt.sharpe_diagnostics(rets)
+    assert diag["sharpe"] is not None and diag["sharpe_se"] is not None
+    assert bt.deflated_sharpe_ratio(
+        diag["sharpe"], diag["sharpe_se"], n_trials=1, n_returns=diag["n_returns"]
+    ) is None
+    dsr = bt.deflated_sharpe_ratio(
+        diag["sharpe"],
+        diag["sharpe_se"],
+        n_trials=20,
+        n_returns=diag["n_returns"],
+        return_skew=diag["return_skew"],
+        return_kurtosis=diag["return_kurtosis"],
+    )
+    assert dsr is not None
+    assert 0.0 <= dsr <= 1.0
+
+
 def test_bucket_stats_counts_and_expectancy():
     cands = [
         _cand(70, OUTCOME_TARGET_HIT, 3.0),   # +3R win

tests/unit/test_scheduler.py

@@ -115,6 +115,8 @@ class TestConfigureScheduler:
             "event_study",
             "backtest",
             "daily_pipeline",
+            "near_close_pipeline",
+            "after_close_pipeline",
             "intraday_pipeline",
         }
 
@@ -125,6 +127,7 @@ class TestConfigureScheduler:
         job_ids = [j.id for j in scheduler.get_jobs()]
         # Each ID should appear exactly once
         assert sorted(job_ids) == sorted([
+            "after_close_pipeline",
             "alerts",
             "backtest",
             "benchmark_collector",
@@ -134,6 +137,7 @@ class TestConfigureScheduler:
             "data_backfill",
             "fundamental_collector",
             "market_regime",
+            "near_close_pipeline",
             "regime_monitor",
             "event_study",
             "outcome_evaluator",

tests/unit/test_trade_policy.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 from datetime import datetime, timedelta, timezone
 
 import pytest
+from sqlalchemy import select
 
 from app.models.paper_trade import PaperTrade
 from app.models.ticker import Ticker
@@ -129,6 +130,54 @@ async def test_latest_stop_starts_a_new_gate_reset_episode(session):
     assert ticker.id in await get_reentry_gate_locks(session)
 
 
+async def test_same_day_fail_then_qualify_stays_locked(session):
+    """Fail at 10:00 NY and qualify at 15:35 NY same day must not unlock."""
+    session.add(User(id=1, username="u", password_hash="x", role="user", has_access=True))
+    ticker = Ticker(symbol="SAMEDAY")
+    session.add(ticker)
+    await session.flush()
+
+    stopped_at = datetime(2026, 7, 15, 14, 0, tzinfo=timezone.utc)  # 10:00 ET
+    session.add(_stopped_trade(ticker.id, closed_at=stopped_at))
+    await session.commit()
+
+    fail_at = datetime(2026, 7, 15, 14, 5, tzinfo=timezone.utc)  # ~10:05 ET
+    updated = await observe_reentry_gate_transitions(
+        session,
+        evaluated_ticker_ids={ticker.id},
+        qualified_ticker_ids=set(),
+        observed_at=fail_at,
+    )
+    assert updated == {ticker.id}
+
+    qualify_same_day = datetime(2026, 7, 15, 19, 35, tzinfo=timezone.utc)  # 15:35 ET
+    updated = await observe_reentry_gate_transitions(
+        session,
+        evaluated_ticker_ids={ticker.id},
+        qualified_ticker_ids={ticker.id},
+        observed_at=qualify_same_day,
+    )
+    assert updated == set()
+    assert ticker.id in await get_reentry_gate_locks(session)
+    trade = (
+        await session.execute(select(PaperTrade).where(PaperTrade.ticker_id == ticker.id))
+    ).scalar_one()
+    assert trade.reentry_gate_failed_at is not None
+    assert trade.reentry_gate_requalified_at is None
+
+    qualify_next_day = datetime(2026, 7, 16, 19, 35, tzinfo=timezone.utc)  # next day 15:35 ET
+    updated = await observe_reentry_gate_transitions(
+        session,
+        evaluated_ticker_ids={ticker.id},
+        qualified_ticker_ids={ticker.id},
+        observed_at=qualify_next_day,
+    )
+    assert updated == {ticker.id}
+    await session.refresh(trade)
+    assert trade.reentry_gate_requalified_at is not None
+    assert ticker.id not in await get_reentry_gate_locks(session)
+
+
 async def test_newer_non_stop_exit_supersedes_historical_stop(session):
     session.add(User(id=1, username="u", password_hash="x", role="user", has_access=True))
     ticker = Ticker(symbol="LATEREXIT")