feat: near-close scan schedule and distinct-day gate reset
Move the only qualifying R:R scan to 15:30 ET with chained Telegram alerts, put outcome eval after a final-bar OHLCV fetch, enforce NY trading-day requalify semantics, stamp paper trades fill_mode=near_close, and migrate stored schedule_* keys to America/New_York.
This commit is contained in:
@@ -99,40 +99,79 @@ tail-trimming until shown otherwise.
|
||||
|
||||
---
|
||||
|
||||
## Ops design — settle before touching the scheduler
|
||||
## Ops design — implementation plan (code-checked)
|
||||
|
||||
### 1. One canonical qualifying scan per day (non-negotiable)
|
||||
Assumptions verified against current code before ship:
|
||||
|
||||
Post-stop gate-reset was validated on **one full-universe observation per day**.
|
||||
Adding a near-close scan *alongside* the 07:00 scan would let fail→qualify
|
||||
transitions complete twice as fast and **silently change** the validated re-entry
|
||||
policy.
|
||||
- Intraday pipeline already fetches/upserts the **in-progress day-t bar** all
|
||||
session (`fetch_ohlcv` end_date defaults to today). Near-close job =
|
||||
**OHLCV fetch → R:R scan** (no new snapshot synthesizer).
|
||||
- One global `schedule_timezone` (default `Europe/Berlin`); stored
|
||||
`SystemSetting` values shadow code defaults — **defaults alone do not
|
||||
migrate prod**.
|
||||
- `observe_reentry_gate_transitions` stamps timestamps with **no same-day
|
||||
guard** today — dual scans would accelerate fail→requalify unless fixed in
|
||||
`trade_policy`.
|
||||
|
||||
**Move** the R:R scan to near-close. **Leave** sentiment / fundamentals /
|
||||
OHLCV-backfill at 07:00 (or existing early slots). Preserve **scan before
|
||||
Outcome Eval** (scan late session, eval after close) so the documented
|
||||
strict-gate-reset live analogue is unchanged.
|
||||
### Semantic guard (ship step 1 — precondition)
|
||||
|
||||
### 2. Cron in `America/New_York`, not `Europe/Berlin`
|
||||
In `trade_policy` (not the scheduler):
|
||||
|
||||
DST offsets shift on different dates. A Berlin-fixed wall time drifts ~1 hour
|
||||
off the US close for a week or two twice a year.
|
||||
> `reentry_gate_requalified_at` may only be set when `reentry_gate_failed_at`
|
||||
> falls on an **earlier America/New_York trading date** than the current
|
||||
> observation.
|
||||
|
||||
### 3. Fill mechanics and feed honesty
|
||||
Manual mid-day scans stay allowed; same-day fail+qualify cannot unlock.
|
||||
Unit test: fail 10:00 / qualify 15:35 same day → still locked; qualify next day → unlocked.
|
||||
|
||||
- NYSE MOC cutoff **15:50 ET**; Nasdaq **15:55 ET**.
|
||||
- Scan ~**15:30–15:40 ET** on a latest-price snapshot; place entries by 15:50.
|
||||
- For now: paper-trade entries marked at the **actual close**.
|
||||
- Document Alpaca entitlement: if only 15-minute-delayed SIP, a 15:35 scan sees
|
||||
~15:20 prices — immaterial for a 12-month signal, but write it down so nobody
|
||||
treats it as a bug.
|
||||
### Schedule split
|
||||
|
||||
### 4. Partial-bar plumbing
|
||||
| Slot (America/New_York) | Jobs |
|
||||
|---|---|
|
||||
| Morning (~02:00) | OHLCV backfill, benchmark, sentiment, fundamentals — **no** qualifying R:R scan |
|
||||
| Near-close (~15:30 Mon–Fri) | OHLCV fetch (refresh day-t bar) → **R:R scan** (only daily qualifying observation) |
|
||||
| After close (~16:30–17:00) | **Outcome eval** on its own slot (not chained to the partial-bar scan) |
|
||||
| Intraday hourly | Unchanged in NY terms; last ~16:00 still mid-session under 15m feed |
|
||||
|
||||
- Scan synthesizes day-t’s in-progress bar from the snapshot.
|
||||
- Nightly OHLCV job overwrites with the final bar via existing idempotent upsert.
|
||||
- Forward paper record marks entries at the actual near-close fill so the live
|
||||
track measures the new execution honestly.
|
||||
- Near-close scan **1–5 only**; US-holiday no-ops are fine (stale identical data
|
||||
can’t flip gates) — comment only, no exchange calendar.
|
||||
- **Do not** run morning + near-close qualifying scans; move the scan, don’t add a second.
|
||||
|
||||
### Behavior change to document (not an accident)
|
||||
|
||||
With scan at ~15:35 ET, stops closed by **earlier same-day** intraday outcome
|
||||
evals can get a **same-day fail observation** — closer to the **promoted**
|
||||
`gate_reset` arm (stop-day close may establish failure) than today’s
|
||||
`strict_gate_reset` analogue (scan always before any eval). Stops after the
|
||||
bell still wait a day. Rewrite README “Live timing matters” / post-stop sections
|
||||
and a line here when shipping.
|
||||
|
||||
### Feed / paper honesty
|
||||
|
||||
- Document 15-minute delayed SIP: 15:35 scan may see ~15:20 prices; OK for 12-1.
|
||||
- Paper entry price ≈ scan entry (near close) is nearly automatic; add
|
||||
**`fill_mode=near_close` era tag** so Track Record can separate morning-scan /
|
||||
near-close / future broker-routed eras.
|
||||
- Morning sentiment staleness is display-only; gate is price-only (GTL parity /
|
||||
neutral-sentiment backtest). One doc line closes that.
|
||||
|
||||
### Stored settings migration (ship step 4)
|
||||
|
||||
Flip global default TZ to `America/New_York` and re-express crons in NY time.
|
||||
**Also** migration (or documented Admin rewrite) of stored `schedule_*` keys so
|
||||
prod does not keep 07:00 Berlin silently.
|
||||
|
||||
### Ship order
|
||||
|
||||
1. `trade_policy` distinct-day requalify guard + unit test
|
||||
2. Near-close job = existing fetch → scan; outcome eval own after-close slot
|
||||
3. Paper `fill_mode=near_close` era tag; verify entry marking
|
||||
4. Defaults + **stored settings migration** + README/research timing rewrite
|
||||
|
||||
### Out of scope
|
||||
|
||||
Broker MOC routing, more fill-timing sim, nasdaq_all / fip / sector (grade later
|
||||
under the fill mode you trade).
|
||||
|
||||
---
|
||||
|
||||
@@ -141,6 +180,13 @@ off the US close for a week or two twice a year.
|
||||
| Item | Status |
|
||||
|---|---|
|
||||
| Research evidence | **Closed** — this doc + matrix report |
|
||||
| Scheduler move (R:R scan → NY near-close) | **Not started** — blocked on ops design above |
|
||||
| Partial-bar scan path | **Not started** |
|
||||
| Paper fill-at-close marking | **Not started** (may already mark at close; verify when shipping) |
|
||||
| Distinct-day gate-reset guard | **Shipped** — `trade_policy` + unit test |
|
||||
| Schedule split + near-close scan→alert | **Shipped** — morning / near-close / after-close (fetch→outcome) |
|
||||
| Paper era tag | **Shipped** — `fill_mode=near_close` on new paper trades |
|
||||
| Settings migration + docs | **Shipped** — alembic 023 rewrites schedule_*; README updated |
|
||||
|
||||
### Shipped behavior change (not accidental)
|
||||
|
||||
Near-close scan at ~15:30 ET lets same-day fail observations after earlier
|
||||
intraday stop closes — closer to promoted `gate_reset` than the old
|
||||
morning-scan `strict_gate_reset` analogue. Documented in README.
|
||||
|
||||
Reference in New Issue
Block a user