From 5a61b164f6de2a32e1d1123de325247fa6563585 Mon Sep 17 00:00:00 2001 From: Dennis Thiessen Date: Sat, 18 Jul 2026 17:07:22 +0200 Subject: [PATCH] docs: lock execution-recovery decisions and ops constraints Record monotone fill-timing gradient, live [1.57,1.77] bracket, recover-flag interpretation, gap-cap as third tail-trim, and pre-scheduler ops checklist. --- docs/research/README.md | 35 +++++-- docs/research/execution-recovery.md | 146 ++++++++++++++++++++++++++++ docs/research/phase-a-matrix.md | 22 +++-- 3 files changed, 190 insertions(+), 13 deletions(-) create mode 100644 docs/research/execution-recovery.md diff --git a/docs/research/README.md b/docs/research/README.md index ca78ee6..396decb 100644 --- a/docs/research/README.md +++ b/docs/research/README.md @@ -115,10 +115,24 @@ Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** · | Max-hold {45,60,90} | **Note and move on** — validation glitter, train collapse (regime interaction) | | Equity-curve vol targeting | **Reject as edge** on this sample; park vt25 as optional DD insurance only | | Correlation caps | **Reject**; sector caps stay Phase B with reduced expectations | -| Next-open fill | **Discovery, not reject** — honest deployable ~Sharpe 1.2 / CAGR 30%. Decision baseline for future promotion = `next_open` | +| Next-open fill | **Discovery, not reject** — honest deployable ~Sharpe 1.2 / CAGR 30% under overnight scanner. Decision baseline until near-close ships = `next_open` | | `fip_id` re-derive | **Validated** (IC −0.045, t = −2.92) | -**Highest-leverage open work:** near-close execution recovery (scheduling, not a new signal). Simulator: `scripts/run_execution_recovery_matrix.py` (`stale_close` + gap-cap). +### Execution recovery (same day) — closed as evidence + +Full write-up: **[execution-recovery.md](execution-recovery.md)** · +`reports/execution_recovery_matrix.json`. + +| Finding | Decision | +|---|---| +| Monotone fill timing (next_open → stale → close) + DD recovery | **When you fill**, not decaying alpha | +| Live bracket **[1.57, 1.77]** full Sharpe | Near-close expected near top of bracket; no more fill-timing sim | +| Gap-cap | **Dead** — third tail-trim instance | +| Auto-`recover: false` | Not a null — bar hit the lower-bound arm by 0.03 train SE | + +**Highest-leverage open work:** **ops** — move the single daily R:R scan to +`America/New_York` near-close (checklist in execution-recovery.md). Not more research +knobs. --- @@ -126,10 +140,10 @@ Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** · | Lead | Why it's interesting | Blocker | |---|---|---| -| **Near-close / MOC execution** | Recovers the overnight momentum drift a 07:00-Berlin scanner leaves on the table (~0.5 Sharpe / ~18pp CAGR vs close-fill) | Prove with `stale_close` arm; then schedule change | -| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign; re-derived fingerprint matched Phase A | Doesn't improve *this* book. Revisit when the universe broadens | -| **Broader universe** (`nasdaq_all`) | Strengthens every week's cross-section and the IC t-stat | Also where `fip_id` could become tradeable | -| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time | +| **Near-close / MOC execution (ops)** | Recovers overnight momentum drift left on the table by a morning EU scan; evidence closed | Implement schedule + partial-bar scan path; one qualifying scan/day only | +| **`fip_id`** (information discreteness over the 12-1 window) | **Strongest cross-sectional signal measured on this universe** — IC −0.045, t = −2.91, correct sign; re-derived fingerprint matched Phase A | Doesn't improve *this* book. Revisit when the universe broadens — **after** execution path is decided | +| **Broader universe** (`nasdaq_all`) | Strengthens every week's cross-section and the IC t-stat | Grade under the fill mode you will trade | +| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships | | **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR | --- @@ -138,7 +152,8 @@ Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** · 1. **Nested lookback windows are NOT out-of-sample.** The clear-air result (#2) was clean, large, and consistent across five nested windows — and still died on a - proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT`. + proper train/test split by entry date. Use `BACKTEST_HOLDOUT_SPLIT` / a named + validation window — and do not pretend a repeatedly opened window is pristine. 2. **Check what population an ablation actually admits.** The blanket fallback (#3) looked like it tested the "resistance famine" hypothesis. It didn't — 65% of the setups it let in were a different population entirely, and they drove the result. @@ -148,6 +163,12 @@ Full write-up: **[phase-a-matrix.md](phase-a-matrix.md)** · 4. **The iron rule:** a signal earns its way into selection *only* through the factor harness — |mean IC| ≳ 0.03, consistent sign, `reliable: true` (≥ 12 non-overlapping windows). Never let an unvalidated score gate setups. +5. **Momentum filters are guilty of tail-trimming until proven otherwise.** + Independent failures: take-profit exits, FIP as an in-book filter, gap-up entry + caps. Cosmetic quality up, P&L down — the right tail *is* the edge. +6. **Fill timing is part of the strategy.** Close-fill reports are not deployable + numbers for an overnight scanner. Grade promotion under the fill mode you will + actually trade. --- diff --git a/docs/research/execution-recovery.md b/docs/research/execution-recovery.md new file mode 100644 index 0000000..be959ee --- /dev/null +++ b/docs/research/execution-recovery.md @@ -0,0 +1,146 @@ +# Execution recovery (2026-07-18) — when you fill is the edge you leave on the table + +Report: `reports/execution_recovery_matrix.json` / `.md` +Follows Phase A A4 ([phase-a-matrix.md](phase-a-matrix.md)). +Arms: `close_control` · `next_open` · `stale_close` · `next_open_gap2`. + +Mechanics verified before sign-off: `stale_close` re-anchors stop to +fill − 1.5×ATR(signal day); no same-day stop after a close fill; each t−1 +candidate keeps its own gate/rank cross-section (no lookahead). Numbers trusted. + +--- + +## Results + +| Arm | Full Sharpe | Full CAGR | Full DD | Train Sharpe | Val Sharpe | +|---|---:|---:|---:|---:|---:| +| **close_control** | **1.77** | **48.3%** | **21.6%** | 1.75 | 1.68 | +| **stale_close** | **1.57** | **40.6%** | **21.8%** | 1.36 | **1.74** | +| next_open | 1.20 | 30.0% | 28.2% | 0.94 | 1.44 | +| next_open_gap2 | 1.06 | 25.2% | 27.8% | 0.72 | 1.41 | + +### Monotonicity (the strongest evidence) + +In every window, Sharpe recovers as the fill moves **toward** the signal: + +| Window | next_open → stale_close → close | +|---|---| +| Train | 0.94 → 1.36 → 1.75 | +| Validation | 1.44 → 1.74 → 1.68* | +| Full | 1.20 → 1.57 → 1.77 | + +\*Val close 1.68 is within SE of stale 1.74 — not a break of the story. + +A dead edge does **not** produce a monotone gradient in fill timing. A live edge +that is progressively surrendered to execution delay does. Combined with full-period +**DD recovery** (28.2% → 21.8% ≈ close’s 21.6%), this confirms the diagnosis: +**when you fill**, not decaying alpha. + +### Auto-flag `recover: False` is not a null + +Pre-registered recovery applied the “≥ close − 0.5 SE” bar to **`stale_close`**, +the **lower-bound** arm (one full session of lag). Flags: + +| Flag | Result | +|---|---| +| near_close_control (val) | **True** (Δ +0.06) | +| beats_next_open | **True** (val Δ +0.30) | +| train_ok | **False** (1.36 vs need ≥ ~1.39 — miss by ~0.03) | + +The floor missed “≥ close − 0.5 SE” by 0.03 while the live design is expected to +sit **above** the floor. The flag worked correctly on the wrong object. + +**Log sentence:** *Partial recovery proven; full recovery needs same-day fill.* + +### Live outcome bracket + +| Bound | Arm | Meaning | +|---|---|---| +| Floor | stale_close ~1.57 full | One-session-stale close fill (conservative) | +| Ceiling | close_control ~1.77 full | Same-day close; optimistic only by final ~15 min of signal info | + +Real near-close execution (scan ~15:30–15:40 ET on a ~99% complete bar, MOC by +15:50/15:55) is signal-at-partial-bar filled at the same close the control uses. +**Live truth is bracketed [1.57, 1.77]** with residual uncertainty of ~15 minutes +of staleness, not 24 hours — expect near the **top** of the bracket. + +`stale_close` alone already justifies the schedule change. **No further fill-timing +simulation on this snapshot** — the bracket is the result. You cannot simulate +15:45 partial bars from daily data, and you do not need to. + +### Gap-cap — dead (third tail-trim instance) + +`next_open_gap2` worse than plain next_open on every window (full Sharpe 1.06 vs +1.20). **262** full-period gap-ups skipped — they were continuations. + +This is the **third independent instance** of the same lesson: + +1. Take-profit exits (gate target as TP) +2. FIP as an in-book filter +3. **Gap-up entry caps** + +Any rule that trims the right tail improves cosmetic quality metrics and destroys +P&L. **Standing method rule:** momentum filters must be presumed guilty of +tail-trimming until shown otherwise. + +--- + +## Decisions locked + +1. **Biggest lever is execution scheduling**, not a strategy rewrite. +2. **Until near-close execution is live:** grade strategy promotion under + `fill_mode=next_open`; keep close-fill as historical control. +3. **After near-close ships:** grade under a close-like / near-close fill mode + (actual live path). +4. **No more sim arms** on fill timing for this snapshot. +5. **Gap-cap:** do not ship. +6. **Strategy work** (nasdaq_all, fip_id, sector) waits until the execution path + is decided — those experiments must be graded under the fill mode you will trade. + +--- + +## Ops design — settle before touching the scheduler + +### 1. One canonical qualifying scan per day (non-negotiable) + +Post-stop gate-reset was validated on **one full-universe observation per day**. +Adding a near-close scan *alongside* the 07:00 scan would let fail→qualify +transitions complete twice as fast and **silently change** the validated re-entry +policy. + +**Move** the R:R scan to near-close. **Leave** sentiment / fundamentals / +OHLCV-backfill at 07:00 (or existing early slots). Preserve **scan before +Outcome Eval** (scan late session, eval after close) so the documented +strict-gate-reset live analogue is unchanged. + +### 2. Cron in `America/New_York`, not `Europe/Berlin` + +DST offsets shift on different dates. A Berlin-fixed wall time drifts ~1 hour +off the US close for a week or two twice a year. + +### 3. Fill mechanics and feed honesty + +- NYSE MOC cutoff **15:50 ET**; Nasdaq **15:55 ET**. +- Scan ~**15:30–15:40 ET** on a latest-price snapshot; place entries by 15:50. +- For now: paper-trade entries marked at the **actual close**. +- Document Alpaca entitlement: if only 15-minute-delayed SIP, a 15:35 scan sees + ~15:20 prices — immaterial for a 12-month signal, but write it down so nobody + treats it as a bug. + +### 4. Partial-bar plumbing + +- Scan synthesizes day-t’s in-progress bar from the snapshot. +- Nightly OHLCV job overwrites with the final bar via existing idempotent upsert. +- Forward paper record marks entries at the actual near-close fill so the live + track measures the new execution honestly. + +--- + +## Implementation status + +| Item | Status | +|---|---| +| Research evidence | **Closed** — this doc + matrix report | +| Scheduler move (R:R scan → NY near-close) | **Not started** — blocked on ops design above | +| Partial-bar scan path | **Not started** | +| Paper fill-at-close marking | **Not started** (may already mark at close; verify when shipping) | diff --git a/docs/research/phase-a-matrix.md b/docs/research/phase-a-matrix.md index 6e0fa54..f925005 100644 --- a/docs/research/phase-a-matrix.md +++ b/docs/research/phase-a-matrix.md @@ -74,9 +74,15 @@ Overnight gap on validation entries: mean **−0.52%**, median −0.18%, p05 − This is not “slippage noise.” It is largely the **overnight momentum drift** that close-fill earns and a 07:00-Berlin scanner (signal yesterday’s close → fill tomorrow’s open) **structurally cannot**. Honest deployable number under that schedule is ~Sharpe 1.2 / CAGR 30%, not 1.77 / 48%. -**Decision baseline going forward:** grade **promotion** under `fill_mode=next_open`; keep close-fill as the historical control for comparability with prior reports. +**Decision baseline going forward (until near-close ships):** grade **promotion** under +`fill_mode=next_open`; keep close-fill as the historical control for comparability +with prior reports. -**Highest-leverage follow-up (not a strategy change):** near-close / MOC-style execution (~15:45 ET) so live fills sit near the close the signal is built on. Simulator proof arm: `stale_close` (signal t−1 close, fill t close). Secondary: next-open **gap-cap** (skip open > +2% vs signal close) — measure, don’t assume. +**Follow-up (done):** execution recovery matrix — see +**[execution-recovery.md](execution-recovery.md)**. Short version: monotone fill-timing +gradient + DD recovery prove this is *when you fill*; live bracket **[1.57, 1.77]**; +gap-cap dead; no more fill-timing sim on this snapshot; ops move R:R scan to NY +near-close (one scan/day). ### `fip_id` re-derivation — **validated** @@ -101,10 +107,14 @@ No arm cleared ΔSharpe > 1 SE. - Regime-conditional max-hold as a “small” experiment - Treat validation-only max-hold glitter as a free CAGR lift - Ship vol targeting as edge without a vol-regime sample +- More fill-timing simulation on this snapshot (settled — see execution-recovery.md) +- Dual daily qualifying scans (would break gate-reset validation) +- Gap-up entry filters (third tail-trim failure) ## What to do next -1. **Execution recovery matrix** — `stale_close` vs close vs next_open; optional gap-cap under next_open (`scripts/run_execution_recovery_matrix.py`). -2. If `stale_close` ≈ close control: schedule scan near the US close (not a signal rewrite). -3. Until near-close execution ships live: **decision baseline = next_open**. -4. Phase B data work only when wanted: `nasdaq_all` + `fip_id`, earnings calendar, sector residual/caps. +1. **Ship near-close execution** — ops checklist in [execution-recovery.md](execution-recovery.md) + (one R:R scan/day in `America/New_York`, MOC window, partial-bar honesty). +2. Until that ships: **decision baseline = next_open**. +3. Strategy work (nasdaq_all, fip_id, sector) only **after** execution path is decided, + graded under the fill mode you will trade.