diff --git a/docs/research/earnings-gap-and-sue.md b/docs/research/earnings-gap-and-sue.md index 4211c84..c2fbdae 100644 --- a/docs/research/earnings-gap-and-sue.md +++ b/docs/research/earnings-gap-and-sue.md @@ -1,202 +1,167 @@ # Earnings gap diagnostic + SUE / PEAD (Tier-1 alpha research) -**Status:** **PARK** (incomplete earnings coverage; SUE fails iron rule on available sample). -**Branch:** `research/earnings-gap-and-sue` -**Production impact:** none. Local research only. **No filters shipped from 2a.** -**Artifacts:** `reports/earnings-gap-sue-20260719-093129.json` (+ companion `.md`) +**Status:** **CLOSED — SUE DEAD**. +**Branch:** `research/earnings-gap-and-sue` +**Production impact:** none. Local research only; no earnings filter or SUE integration is shipped. --- -## Pre-registration (locked before first research run) +## Pre-registration (locked before the final research run) ### Data -- Historical earnings calendar for the production universe over the full snapshot - window (and deeper if the feed provides it). -- Preferred source: FMP **date-range earnings-calendar** (bulk). If unavailable on - free tier, fall back to per-symbol `/stable/earnings` with request accounting. -- Store in a real local table `earnings_events` (symbol + announce_date key). -- Point-in-time: a surprise is usable only from **announce date + 1 trading day** - onward. +- Historical earnings announcements for the production universe, stored in the + real `earnings_events` table and deduplicated on symbol + announcement date. +- The originally requested 2016 start is amended, with user approval, to the + public source's announcement coverage start of 2020-01-22. Earlier EPS-period + history may scale later surprises but may never activate a live signal. +- Report coverage, pairing, duplicates/restatements, annual-rate sanity, and + announcement-session quality before either experiment. +- Point-in-time: an earnings surprise is usable only from announcement date +1 + trading day. Same-day use is forbidden. ### Experiment 2a — earnings-gap risk (defense, report-only) -Join simulated production-config trades (`fill_mode=close`) with earnings dates. +Run the production-config book on the approximately 505-name production +universe with close fills and 0.001 transaction cost per side. Join simulated +trades to earnings by symbol and date. -**Pre-registered questions:** +1. Among closed trades with realized net R ≤ -1.0, report the fraction with an + announcement strictly after entry and before exit, alongside the base rate + for all trades. +2. Compare entries within three trading sessions before an announcement with + all other entries: count, mean/median R, win rate, p05, and p95. +3. Compare stops within one trading session after an announcement with all + other stops and exits. -1. What fraction of losses worse than **−1R** occur with an earnings announcement - **between entry and exit** (inclusive of the holding window)? -2. What is the mean R of entries taken within **3 trading days BEFORE** an - announcement vs all other entries — report **both tails** of the R - distribution (rule 4: any earnings-avoid entry filter is presumed guilty of - right-tail trimming until the win distribution shows otherwise)? - -**Output:** distributions and counts only. -**No filter is shipped.** If numbers argue for a filter → report and stop. +Verdict is always `INFORMATIONAL`. Report only: no filter arm, recommendation, +or implementation. The right tail must be shown alongside the left tail. ### Experiment 2b — SUE / PEAD (offense) Signal `sue_latest`: \[ -\text{SUE} = \frac{\text{actual} - \text{estimate}}{\sigma(\text{trailing 8 surprises})} +\text{SUE} = \frac{\text{actual} - \text{estimate}} +{\sigma(\text{trailing 8 surprises})} \] -Fallback if estimate history is thin: scale surprise by price. -Carry forward from announce+1 for **63 trading days**, else NaN (name drops out -of that cross-section). +Use at least four trailing surprises; if estimate history fails the registered +quality gate, use `(actual - estimate) / price` and name that fallback. Activate +at announcement date +1 trading day, carry for 63 trading days, then drop the +symbol from the cross-section. -**Iron rule (IC harness):** mean weekly Spearman IC on non-overlapping weeks; -\|mean IC\| ≥ ~0.03, **positive** sign (drift), `reliable: true` (≥12 windows). +Evaluate mean weekly Spearman IC on the existing non-overlapping-window harness. +Always report `sue_latest`, `mom_12_1`, and `mom_12_1_resid` on identical +week-symbol-forward-return cells, plus SUE inside the top momentum quintile. -Always side-by-side with `mom_12_1` and `mom_12_1_resid` on **identical** -cross-sections. +### Mechanical verdict rule -Also report **momentum-conditional** IC (within top momentum quintile). - -**If it passes iron rule:** STOP and report. Book-integration design is a -separate human-approved step — do not wire. - -### Verdict labels - -| label | meaning | -|---|---| -| **PROMOTE** | (2b only) iron rule cleared → human designs tilt/gate | -| **PARK** | Interesting but incomplete / weak | -| **DEAD** | No edge / diagnostic argues against action | -| **REPORT-ONLY** | (2a) always — never auto-filter | - ---- - -## Data provenance - -| item | result | -|---|---| -| Snapshot | `backtest_snapshots/prod.sqlite` (506 names) | -| FMP bulk `earnings-calendar` | **402 Premium** — not available on free tier | -| FMP per-symbol `/stable/earnings` | used; hit daily rate limit ~225 reqs | -| Alpha Vantage `EARNINGS` | used for +24 symbols (announce = `reportedDate`) | -| Symbols with events | **48 / 506 (9.5%)** | -| Total events | 5,612 (5,018 with actual+estimate) | -| Announce range | 1985-08-31 → 2026-07-16 | -| FMP requests (first day) | 260 FMP + 25 AV (see `reports/earnings-backfill-status.json`) | - -**Incomplete backfill is first-class.** 2a under-detects earnings overlaps; 2b SUE -cross-section averages **~47 names**, not ~500. Resume: - -```bash -# Day N (FMP free ~250/day; AV free ~25/day — prefer FMP after reset) -python scripts/backfill_earnings_events.py \ - --snapshot backtest_snapshots/prod.sqlite \ - --provider fmp --force-symbol --limit 250 --sleep 0.4 - -# When done==506: -python scripts/run_earnings_research.py \ - --snapshot backtest_snapshots/prod.sqlite \ - --workers 6 --allow-spawn -``` +- **PASS** only if unconditional `sue_latest` has mean IC ≥ +0.03, + `reliable: true` (at least 12 windows), and positive signs in both the pre-2021 + and post-2021 eras. +- **FAIL** otherwise, with terminal verdict `SUE DEAD for this stack`. +- PASS stops at `SUE PASS→PENDING_HUMAN`; integration design remains a separate + human decision. FAIL is terminal and no variants are proposed. --- ## Results -Generated: `2026-07-19T09:31:29` +### Data quality gate -### 2a — Earnings-gap risk (report-only) +Approved earnings window: 2020-01-22 to 2026-07-17. Source mode: dolthub_public_bulk_clone. -Production book sim: Sharpe 2.09 (SE 0.497), CAGR 51.6%, max DD 21.4%, **322 trades**, -`fill_mode=close`. - -#### Q1 — Losses worse than −1R with earnings in hold - -| metric | value | +| check | result | |---|---:| -| n losses < −1R | 28 | -| of which earnings in hold | **1** | -| fraction | **3.6%** | -| all trades with earnings in hold | 14 / 322 (4.4%) | +| Prod symbols requested / tradable | 506 / 505 | +| Manifest complete + live counts match | True | +| Prod symbols with pre-2021 bars | 491 (97.2%) | +| SPY benchmark depth | 2649 rows, 2016-01-04 to 2026-07-17 | +| Snapshot depth gate | True | +| Bulk source windows / requests logged | 1/1 / 1 | +| Source repository / pinned commit | https://www.dolthub.com/repositories/post-no-preference/earnings @ 9n0et3hpj9j7vue8f3qsldon3qa5sdjj | +| Source license / upstream provider documented | CC-BY-SA-4.0 / False | +| Existing-source conflicts preserved | 940 rows / 1526 fields | +| Symbols with >=8 announcements | 498 (98.6%) | +| Symbols with >=8 paired announcements | 495 (98.0%) | +| Events with estimate + actual | 12311/12414 (99.2%) | +| Duplicate rows in keyed table | 0 | +| Duplicate / restated payload rows fetched | 0 / 940 | +| Mean announcements per active symbol-year | 4.08 (expected about 4) | +| Symbols far off (<2 or >6/year, incl. zero) | 1 | +| Recognised BMO/AMC/during | 92.8% (reliable=True) | +| Point-in-time policy | announce_date_plus_1_trading_day_for_all_events | +| SUE price fallback | not_used | -**Read:** On incomplete earnings labels this is a **lower bound** on earnings -overlap, not a clean “earnings rarely hurt.” Do **not** conclude earnings risk is -immaterial until coverage ≥ ~95% of the book’s names. +Deduplication: UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment -#### Q2 — Entry within 3 trading days before announce (both tails) +Far-off announcement-rate symbols: SPCX -| cohort | n | mean R | win rate | p05 | p50 | p95 | max | -|---|---:|---:|---:|---:|---:|---:|---:| -| pre-earn (≤3d before) | **4** | 1.94 | 50% | −1.24 | 1.12 | 6.26 | 6.84 | -| other | 318 | 0.70 | 37% | −1.11 | −0.83 | 6.08 | **12.87** | -| all | 322 | 0.71 | 37% | −1.12 | −0.83 | 6.22 | 12.87 | +### Experiment 2a - earnings-gap risk diagnostic -**Tail-trim presumption:** n=4 is not a sample. Point estimate does **not** show -right-tail destruction of pre-earn entries (p95 similar; max actually higher in -“other”). **No earnings-avoid filter is supported.** Re-run after full backfill. +Verdict: **INFORMATIONAL**. Report-only; no filter arm or implementation. ---- +Trade cohort is restricted to the approved earnings-coverage window 2020-01-22 to 2026-07-17; 0 simulated trades outside that window were excluded. -### 2b — SUE / PEAD IC +| cohort | count | fraction | +|---|---:|---:| +| Realized net R <= -1.0 | 266 | - | +| Losses with announcement strictly inside hold | 23 | 0.0865 | +| All trades with announcement strictly inside hold | 115 | 0.2003 | -#### Full-universe harness (mom on ~500; SUE only where labeled) +| Entry cohort | count | mean R | median R | win rate | p05 R | p95 R | +|---|---:|---:|---:|---:|---:|---:| +| Within 3 sessions before earnings | 27 | 0.4837 | -1.0265 | 0.3333 | -1.1463 | 5.981 | +| All other entries | 547 | 0.2734 | -0.8316 | 0.3565 | -1.1228 | 4.5888 | -| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable | -|---|---:|---:|---:|---:|---| -| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | true | -| mom_12_1_resid | 0.0552 | 1.98 | 35 | 497.7 | true | -| mom_12_1 | 0.0531 | 1.61 | 35 | 497.7 | true | -| **sue_latest** | **0.0172** | **0.6** | 44 | **47.4** | true | -| fip_id | −0.045 | −2.91 | 35 | 497.7 | true | +Tail deltas (pre minus other): p05=-0.0235, p95=1.3922. -#### Identical SUE subset (fair side-by-side — use this while coverage is thin) +Registered directional tail condition is not present. -| signal | mean_ic | ic_t_stat | weeks | avg_N | -|---|---:|---:|---:|---:| -| sue_latest | 0.0172 | 0.6 | 44 | 47.4 | -| mom_12_1 | −0.0174 | −0.42 | 35 | 47.3 | -| mom_12_1_resid | −0.0104 | −0.27 | 35 | 47.3 | +| Exit cohort | count | mean R | median R | win rate | p05 R | p95 R | +|---|---:|---:|---:|---:|---:|---:| +| Stops within 1 session after earnings | 26 | -0.6434 | -0.9753 | 0.2308 | -2.4614 | 1.1142 | +| All other stops | 433 | -0.5035 | -1.0278 | 0.1963 | -1.1373 | 1.3591 | +| All other exits | 548 | 0.3273 | -0.8361 | 0.3613 | -1.0644 | 4.7224 | -On the thin labeled subset, momentum itself is noise — so the subset is not yet -a meaningful PEAD test. +### Experiment 2b - SUE / post-earnings drift -#### Momentum-conditional SUE (top mom quintile) +Mechanical verdict: **FAIL** - SUE DEAD for this stack -| metric | value | -|---|---:| -| mean IC | **−0.0065** | -| t | −0.1 | -| weeks | 35 | +Identical cross-sections: -Wrong sign vs “ride positive surprises inside the momentum gate.” +| signal | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| sue_latest | 0.0148 | 1.27 | 56 | 450.9 | 51.8 | true | +| mom_12_1 | 0.0195 | 0.74 | 56 | 450.9 | 58.9 | true | +| mom_12_1_resid | 0.0262 | 1.07 | 56 | 450.9 | 55.4 | true | -**Iron rule:** fail (\|IC\| 0.017 < 0.03; t 0.6). **No promote.** +Unconditional SUE grade row: ---- +| signal | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| sue_latest | 0.0151 | 1.29 | 56 | 451.4 | 51.8 | true | -## Verdict +Era stability: -| piece | verdict | -|---|---| -| **2a earnings-gap** | **REPORT-ONLY** — no filter. Coverage too thin for risk claims; tails do not argue for an avoid-filter on n=4. | -| **2b SUE** | **PARK** (effectively not green). Mild positive IC on ~48 names; fails iron bar; mom-conditional flat/negative. Re-score after full backfill before DEAD. | -| **Production** | **no change** | +| era | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| pre-2021 | 0.0286 | 0.7 | 9 | 398.3 | 55.6 | false | +| post-2021 | 0.0172 | 1.34 | 48 | 461.9 | 64.6 | true | ---- +Coverage: 501 symbols with live SUE; avg weekly N=453.1; scored non-overlap avg N=451.4. -## What a human must decide next +Cross-section is not flagged thin at the registered <100-name read. -1. Resume multi-day earnings backfill to **506/506**, then re-run - `run_earnings_research.py` (heavy — MacBook OK). -2. Do **not** ship an earnings-avoid entry filter from 2a. -3. Do **not** wire SUE until a full-coverage IC clears the iron rule (and - preferably mom-conditional > 0). -4. Do not merge into main strategy docs without review. +Momentum-conditional top-quintile SUE: mean IC=0.0213, t=1.3, windows=56, avg N=89.8. ---- +## Artifacts -## Implementation notes +- `reports/earnings-2a-gap-20260720-dolthub-final.json` and companion Markdown +- `reports/earnings-2b-sue-20260720-dolthub-final.json` and companion Markdown +- `reports/earnings-backfill-status.json` -| piece | role | -|---|---| -| `scripts/backfill_earnings_events.py` | bulk attempt → FMP/AV per-symbol; `earnings_events` + meta on snapshot | -| `scripts/run_earnings_research.py` | 2a trade join + 2b SUE IC / mom-conditional | -| Snapshot table `earnings_events` | real table (not SystemSetting JSON) | +Production changes: **none**. No earnings filter or SUE integration was implemented. + +## Final status: **Task 2 CLOSED (SUE DEAD)** diff --git a/reports/earnings-2a-gap-20260720-dolthub-final.json b/reports/earnings-2a-gap-20260720-dolthub-final.json new file mode 100644 index 0000000..725aa9e --- /dev/null +++ b/reports/earnings-2a-gap-20260720-dolthub-final.json @@ -0,0 +1,348 @@ +{ + "generated_at": "2026-07-20T07:02:10.934892+00:00", + "snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite", + "snapshot_depth": { + "manifest": { + "schema_version": 1, + "snapshot": "research.sqlite", + "snapshot_resolved": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite", + "complete": true, + "finished_at": "2026-07-19T14:22:15.706192+00:00", + "ticker_count": 4650, + "ohlcv_row_count": 5081073, + "rank_only_count": 4144, + "sources": { + "pool": "source_snapshot" + }, + "history_days": 5000, + "min_bars": 1262, + "fetch_ok": 505, + "fetch_fail": 1, + "limit": null, + "extra": { + "prod_symbols_at_start": 506, + "pool_size": 506, + "to_fetch": 506, + "source_symbols_only": true, + "benchmark_spy_rows": 2649 + }, + "live_counts": { + "ticker_count": 4650, + "ohlcv_row_count": 5081073, + "rank_only_count": 4144 + } + }, + "requested_symbols": 506, + "tradable_symbols": 505, + "missing_symbols": [], + "zero_bar_symbols": [ + "RHM" + ], + "bar_count": { + "min": 24, + "median": 2649, + "max": 2649 + }, + "symbols_with_pre2021_bars": 491, + "symbols_with_pre2021_bars_pct": 97.2, + "shallow_symbols_lt_1000_bars": [ + { + "symbol": "SPCX", + "first_bar": "2026-06-12", + "last_bar": "2026-07-17", + "bars": 24 + }, + { + "symbol": "Q", + "first_bar": "2025-11-03", + "last_bar": "2026-07-17", + "bars": 176 + }, + { + "symbol": "PSKY", + "first_bar": "2025-08-07", + "last_bar": "2026-07-17", + "bars": 237 + }, + { + "symbol": "SNDK", + "first_bar": "2025-02-13", + "last_bar": "2026-07-17", + "bars": 357 + }, + { + "symbol": "GEV", + "first_bar": "2024-04-02", + "last_bar": "2026-07-17", + "bars": 575 + }, + { + "symbol": "SOLV", + "first_bar": "2024-04-01", + "last_bar": "2026-07-17", + "bars": 576 + }, + { + "symbol": "VLTO", + "first_bar": "2023-10-02", + "last_bar": "2026-07-17", + "bars": 700 + }, + { + "symbol": "KVUE", + "first_bar": "2023-05-04", + "last_bar": "2026-07-17", + "bars": 803 + }, + { + "symbol": "GEHC", + "first_bar": "2022-12-15", + "last_bar": "2026-07-17", + "bars": 898 + } + ], + "price_window": { + "min": "2016-01-04", + "max": "2026-07-17" + }, + "benchmark_spy": { + "rows": 2649, + "min": "2016-01-04", + "max": "2026-07-17" + }, + "gate_threshold": { + "min_tradable_symbols": 505, + "max_missing_or_zero_bar": 1, + "min_symbols_with_pre2021_bars_pct": 80.0, + "benchmark_min_rows": 1000, + "benchmark_must_begin_pre2021": true + }, + "gate_pass": true + }, + "data_quality": { + "window": { + "from": "2020-01-22", + "to": "2026-07-17" + }, + "prod_symbols": 505, + "events": 12414, + "symbols_with_any_event": 504, + "symbols_with_ge8_announcements": 498, + "symbols_with_ge8_announcements_pct": 98.6, + "symbols_with_ge8_paired_announcements": 495, + "symbols_with_ge8_paired_announcements_pct": 98.0, + "events_with_actual_and_estimate": 12311, + "events_with_actual_and_estimate_pct": 99.2, + "duplicate_rows_in_table": 0, + "duplicate_rows_fetched": 0, + "restated_rows_fetched": 940, + "dedupe_policy": "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment", + "events_per_symbol_year": { + "mean_active_span_rate": 4.08, + "expected": "approximately 4", + "far_off_rule": "active-span rate <2 or >6, plus zero-event symbols", + "far_off_count": 1, + "far_off_symbols": [ + { + "symbol": "SPCX", + "events": 0, + "events_per_year": 0.0 + } + ] + }, + "announcement_session": { + "recognised_bmo_amc_or_during": 11520, + "recognised_pct": 92.8, + "reliable": true, + "assessment": "usable" + }, + "point_in_time_policy": "announce_date_plus_1_trading_day_for_all_events", + "sue_scaling": { + "primary": "eps_surprise_over_stdev_of_prior_8_surprises_min_4", + "fallback_trigger": "paired event coverage <50% or symbols with >=8 paired events <50%", + "fallback_needed": false, + "fallback_name": "not_used" + }, + "backfill": { + "mode": "dolthub_public_bulk_clone", + "window": { + "from": "2020-01-22", + "to": "2026-07-17" + }, + "coverage_amendment": { + "approved_by_user": true, + "reason": "FMP free tier blocks historical bulk earnings", + "original_start": "2016-01-04", + "amended_announcement_start": "2020-01-22" + }, + "source": { + "repository": "https://www.dolthub.com/repositories/post-no-preference/earnings", + "commit": "9n0et3hpj9j7vue8f3qsldon3qa5sdjj", + "license": "CC-BY-SA-4.0", + "upstream_provider_documented": false + }, + "bulk_windows_total": 1, + "bulk_windows_done": 1, + "bulk_requests_logged_total": 1, + "bulk_exports": 2, + "calendar": { + "raw_rows": 117482, + "universe_rows_in_window": 12342, + "deduped_rows_in_window": 12342, + "duplicate_rows": 0, + "restated_rows": 0 + }, + "eps_history": { + "raw_rows": 165050, + "universe_rows": 18515, + "deduped_rows": 18515, + "duplicate_rows": 0, + "restated_rows": 0, + "complete_actual_and_estimate": 18304 + }, + "pairing": { + "method": "minimum-cost monotonic alignment per symbol", + "allowed_announce_minus_period_end_days": [ + -14, + 90 + ], + "matched_calendar_events": 12271, + "unmatched_calendar_events": 71, + "unmatched_periods_in_pairing_window": 538, + "announce_minus_period_end_days": { + "min": -10, + "median": 30, + "max": 89 + }, + "pre_2020_eps_history_use": "trailing_surprise_stdev_only; never treated as an announcement or live signal event" + }, + "duplicate_rows_logged_total": 0, + "restated_rows_logged_total": 940, + "conflicting_existing_rows": 940, + "conflicting_existing_fields": 1526, + "preserved_existing_fields": 2945, + "existing_enrichment_events_not_in_dolthub_calendar": 72, + "dedupe_policy": "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment", + "events_in_window": 12414, + "events_with_actual_and_estimate": 12311, + "symbols_done": 506, + "symbols_universe": 506, + "symbols_with_dolthub_calendar": 504, + "symbols_without_dolthub_calendar": [ + "RHM", + "SPCX" + ], + "announce_date_range": { + "min": "2020-01-22", + "max": "2026-07-17" + }, + "complete": true + } + }, + "production_impact": "none", + "experiment": "2a", + "result": { + "verdict": "INFORMATIONAL", + "costs": { + "per_side": 0.001, + "r_is_net_of_round_trip_costs": true + }, + "closed_trades": 574, + "q1_loss_concentration": { + "loss_definition": "realized_net_R <= -1.0", + "holding_period_definition": "announcement strictly after entry and before exit", + "losses_count": 266, + "losses_with_announcement_count": 23, + "losses_with_announcement_fraction": 0.0865, + "all_trades_with_announcement_count": 115, + "all_trades_with_announcement_fraction": 0.2003 + }, + "q2_entries_within_3_trading_days_before_announcement": { + "pre_earnings": { + "count": 27, + "mean_r": 0.4837, + "median_r": -1.0265, + "win_rate": 0.3333, + "p05_r": -1.1463, + "p95_r": 5.981, + "min_r": -1.2449, + "max_r": 8.8996 + }, + "all_other_entries": { + "count": 547, + "mean_r": 0.2734, + "median_r": -0.8316, + "win_rate": 0.3565, + "p05_r": -1.1228, + "p95_r": 4.5888, + "min_r": -6.0161, + "max_r": 19.98 + }, + "tail_deltas_pre_minus_other": { + "p05_r": -0.0235, + "p95_r": 1.3922 + }, + "directional_tail_condition_present": false, + "tail_read": "Registered directional tail condition is not present." + }, + "q3_stop_exits_within_1_trading_day_after_announcement": { + "stops_after_earnings": { + "count": 26, + "mean_r": -0.6434, + "median_r": -0.9753, + "win_rate": 0.2308, + "p05_r": -2.4614, + "p95_r": 1.1142, + "min_r": -2.6413, + "max_r": 1.7012 + }, + "all_other_stops": { + "count": 433, + "mean_r": -0.5035, + "median_r": -1.0278, + "win_rate": 0.1963, + "p05_r": -1.1373, + "p95_r": 1.3591, + "min_r": -6.0161, + "max_r": 19.98 + }, + "all_other_exits": { + "count": 548, + "mean_r": 0.3273, + "median_r": -0.8361, + "win_rate": 0.3613, + "p05_r": -1.0644, + "p95_r": 4.7224, + "min_r": -6.0161, + "max_r": 19.98 + } + }, + "implementation": "REPORT_ONLY_NO_FILTER_ARM_NO_FILTER_CHANGE", + "analysis_window": { + "from": "2020-01-22", + "to": "2026-07-17", + "rule": "entry_on_or_after_start_and_exit_on_or_before_end", + "simulation_trades_total": 574, + "trades_excluded_outside_earnings_coverage": 0 + }, + "run_config": { + "universe_symbols": 505, + "fill_mode": "close", + "cost_per_side": 0.001, + "momentum_cutoff": 80.0, + "exit_policy": "atr_trail3", + "hold_days": 30, + "max_positions": 10, + "risk_per_trade": 0.01 + }, + "sim_summary": { + "start_date": "2020-01-22", + "end_date": "2026-07-13", + "trades": 574, + "sharpe": 1.03, + "cagr_pct": 22.9, + "max_drawdown_pct": 26.6, + "total_return_pct": 280.6 + } + } +} diff --git a/reports/earnings-2a-gap-20260720-dolthub-final.md b/reports/earnings-2a-gap-20260720-dolthub-final.md new file mode 100644 index 0000000..b735255 --- /dev/null +++ b/reports/earnings-2a-gap-20260720-dolthub-final.md @@ -0,0 +1,58 @@ +# Earnings Task 2a - gap diagnostic + +### Data quality gate + +Approved earnings window: 2020-01-22 to 2026-07-17. Source mode: dolthub_public_bulk_clone. + +| check | result | +|---|---:| +| Prod symbols requested / tradable | 506 / 505 | +| Manifest complete + live counts match | True | +| Prod symbols with pre-2021 bars | 491 (97.2%) | +| SPY benchmark depth | 2649 rows, 2016-01-04 to 2026-07-17 | +| Snapshot depth gate | True | +| Bulk source windows / requests logged | 1/1 / 1 | +| Source repository / pinned commit | https://www.dolthub.com/repositories/post-no-preference/earnings @ 9n0et3hpj9j7vue8f3qsldon3qa5sdjj | +| Source license / upstream provider documented | CC-BY-SA-4.0 / False | +| Existing-source conflicts preserved | 940 rows / 1526 fields | +| Symbols with >=8 announcements | 498 (98.6%) | +| Symbols with >=8 paired announcements | 495 (98.0%) | +| Events with estimate + actual | 12311/12414 (99.2%) | +| Duplicate rows in keyed table | 0 | +| Duplicate / restated payload rows fetched | 0 / 940 | +| Mean announcements per active symbol-year | 4.08 (expected about 4) | +| Symbols far off (<2 or >6/year, incl. zero) | 1 | +| Recognised BMO/AMC/during | 92.8% (reliable=True) | +| Point-in-time policy | announce_date_plus_1_trading_day_for_all_events | +| SUE price fallback | not_used | + +Deduplication: UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment + +Far-off announcement-rate symbols: SPCX + +### Experiment 2a - earnings-gap risk diagnostic + +Verdict: **INFORMATIONAL**. Report-only; no filter arm or implementation. + +Trade cohort is restricted to the approved earnings-coverage window 2020-01-22 to 2026-07-17; 0 simulated trades outside that window were excluded. + +| cohort | count | fraction | +|---|---:|---:| +| Realized net R <= -1.0 | 266 | - | +| Losses with announcement strictly inside hold | 23 | 0.0865 | +| All trades with announcement strictly inside hold | 115 | 0.2003 | + +| Entry cohort | count | mean R | median R | win rate | p05 R | p95 R | +|---|---:|---:|---:|---:|---:|---:| +| Within 3 sessions before earnings | 27 | 0.4837 | -1.0265 | 0.3333 | -1.1463 | 5.981 | +| All other entries | 547 | 0.2734 | -0.8316 | 0.3565 | -1.1228 | 4.5888 | + +Tail deltas (pre minus other): p05=-0.0235, p95=1.3922. + +Registered directional tail condition is not present. + +| Exit cohort | count | mean R | median R | win rate | p05 R | p95 R | +|---|---:|---:|---:|---:|---:|---:| +| Stops within 1 session after earnings | 26 | -0.6434 | -0.9753 | 0.2308 | -2.4614 | 1.1142 | +| All other stops | 433 | -0.5035 | -1.0278 | 0.1963 | -1.1373 | 1.3591 | +| All other exits | 548 | 0.3273 | -0.8361 | 0.3613 | -1.0644 | 4.7224 | diff --git a/reports/earnings-2b-sue-20260720-dolthub-final.json b/reports/earnings-2b-sue-20260720-dolthub-final.json new file mode 100644 index 0000000..ef3c22b --- /dev/null +++ b/reports/earnings-2b-sue-20260720-dolthub-final.json @@ -0,0 +1,349 @@ +{ + "generated_at": "2026-07-20T07:02:10.934892+00:00", + "snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite", + "snapshot_depth": { + "manifest": { + "schema_version": 1, + "snapshot": "research.sqlite", + "snapshot_resolved": "C:\\Workspace\\signal-platform\\backtest_snapshots\\research.sqlite", + "complete": true, + "finished_at": "2026-07-19T14:22:15.706192+00:00", + "ticker_count": 4650, + "ohlcv_row_count": 5081073, + "rank_only_count": 4144, + "sources": { + "pool": "source_snapshot" + }, + "history_days": 5000, + "min_bars": 1262, + "fetch_ok": 505, + "fetch_fail": 1, + "limit": null, + "extra": { + "prod_symbols_at_start": 506, + "pool_size": 506, + "to_fetch": 506, + "source_symbols_only": true, + "benchmark_spy_rows": 2649 + }, + "live_counts": { + "ticker_count": 4650, + "ohlcv_row_count": 5081073, + "rank_only_count": 4144 + } + }, + "requested_symbols": 506, + "tradable_symbols": 505, + "missing_symbols": [], + "zero_bar_symbols": [ + "RHM" + ], + "bar_count": { + "min": 24, + "median": 2649, + "max": 2649 + }, + "symbols_with_pre2021_bars": 491, + "symbols_with_pre2021_bars_pct": 97.2, + "shallow_symbols_lt_1000_bars": [ + { + "symbol": "SPCX", + "first_bar": "2026-06-12", + "last_bar": "2026-07-17", + "bars": 24 + }, + { + "symbol": "Q", + "first_bar": "2025-11-03", + "last_bar": "2026-07-17", + "bars": 176 + }, + { + "symbol": "PSKY", + "first_bar": "2025-08-07", + "last_bar": "2026-07-17", + "bars": 237 + }, + { + "symbol": "SNDK", + "first_bar": "2025-02-13", + "last_bar": "2026-07-17", + "bars": 357 + }, + { + "symbol": "GEV", + "first_bar": "2024-04-02", + "last_bar": "2026-07-17", + "bars": 575 + }, + { + "symbol": "SOLV", + "first_bar": "2024-04-01", + "last_bar": "2026-07-17", + "bars": 576 + }, + { + "symbol": "VLTO", + "first_bar": "2023-10-02", + "last_bar": "2026-07-17", + "bars": 700 + }, + { + "symbol": "KVUE", + "first_bar": "2023-05-04", + "last_bar": "2026-07-17", + "bars": 803 + }, + { + "symbol": "GEHC", + "first_bar": "2022-12-15", + "last_bar": "2026-07-17", + "bars": 898 + } + ], + "price_window": { + "min": "2016-01-04", + "max": "2026-07-17" + }, + "benchmark_spy": { + "rows": 2649, + "min": "2016-01-04", + "max": "2026-07-17" + }, + "gate_threshold": { + "min_tradable_symbols": 505, + "max_missing_or_zero_bar": 1, + "min_symbols_with_pre2021_bars_pct": 80.0, + "benchmark_min_rows": 1000, + "benchmark_must_begin_pre2021": true + }, + "gate_pass": true + }, + "data_quality": { + "window": { + "from": "2020-01-22", + "to": "2026-07-17" + }, + "prod_symbols": 505, + "events": 12414, + "symbols_with_any_event": 504, + "symbols_with_ge8_announcements": 498, + "symbols_with_ge8_announcements_pct": 98.6, + "symbols_with_ge8_paired_announcements": 495, + "symbols_with_ge8_paired_announcements_pct": 98.0, + "events_with_actual_and_estimate": 12311, + "events_with_actual_and_estimate_pct": 99.2, + "duplicate_rows_in_table": 0, + "duplicate_rows_fetched": 0, + "restated_rows_fetched": 940, + "dedupe_policy": "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment", + "events_per_symbol_year": { + "mean_active_span_rate": 4.08, + "expected": "approximately 4", + "far_off_rule": "active-span rate <2 or >6, plus zero-event symbols", + "far_off_count": 1, + "far_off_symbols": [ + { + "symbol": "SPCX", + "events": 0, + "events_per_year": 0.0 + } + ] + }, + "announcement_session": { + "recognised_bmo_amc_or_during": 11520, + "recognised_pct": 92.8, + "reliable": true, + "assessment": "usable" + }, + "point_in_time_policy": "announce_date_plus_1_trading_day_for_all_events", + "sue_scaling": { + "primary": "eps_surprise_over_stdev_of_prior_8_surprises_min_4", + "fallback_trigger": "paired event coverage <50% or symbols with >=8 paired events <50%", + "fallback_needed": false, + "fallback_name": "not_used" + }, + "backfill": { + "mode": "dolthub_public_bulk_clone", + "window": { + "from": "2020-01-22", + "to": "2026-07-17" + }, + "coverage_amendment": { + "approved_by_user": true, + "reason": "FMP free tier blocks historical bulk earnings", + "original_start": "2016-01-04", + "amended_announcement_start": "2020-01-22" + }, + "source": { + "repository": "https://www.dolthub.com/repositories/post-no-preference/earnings", + "commit": "9n0et3hpj9j7vue8f3qsldon3qa5sdjj", + "license": "CC-BY-SA-4.0", + "upstream_provider_documented": false + }, + "bulk_windows_total": 1, + "bulk_windows_done": 1, + "bulk_requests_logged_total": 1, + "bulk_exports": 2, + "calendar": { + "raw_rows": 117482, + "universe_rows_in_window": 12342, + "deduped_rows_in_window": 12342, + "duplicate_rows": 0, + "restated_rows": 0 + }, + "eps_history": { + "raw_rows": 165050, + "universe_rows": 18515, + "deduped_rows": 18515, + "duplicate_rows": 0, + "restated_rows": 0, + "complete_actual_and_estimate": 18304 + }, + "pairing": { + "method": "minimum-cost monotonic alignment per symbol", + "allowed_announce_minus_period_end_days": [ + -14, + 90 + ], + "matched_calendar_events": 12271, + "unmatched_calendar_events": 71, + "unmatched_periods_in_pairing_window": 538, + "announce_minus_period_end_days": { + "min": -10, + "median": 30, + "max": 89 + }, + "pre_2020_eps_history_use": "trailing_surprise_stdev_only; never treated as an announcement or live signal event" + }, + "duplicate_rows_logged_total": 0, + "restated_rows_logged_total": 940, + "conflicting_existing_rows": 940, + "conflicting_existing_fields": 1526, + "preserved_existing_fields": 2945, + "existing_enrichment_events_not_in_dolthub_calendar": 72, + "dedupe_policy": "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment", + "events_in_window": 12414, + "events_with_actual_and_estimate": 12311, + "symbols_done": 506, + "symbols_universe": 506, + "symbols_with_dolthub_calendar": 504, + "symbols_without_dolthub_calendar": [ + "RHM", + "SPCX" + ], + "announce_date_range": { + "min": "2020-01-22", + "max": "2026-07-17" + }, + "complete": true + } + }, + "production_impact": "none", + "experiment": "2b", + "result": { + "verdict": "FAIL", + "verdict_detail": "SUE DEAD for this stack", + "grade_rule": { + "mean_ic_ge_0_03_positive": false, + "reliable_ge_12_windows": true, + "positive_sign_pre_and_post_2021": true, + "pass": false + }, + "sue_unconditional": { + "signal": "sue_latest", + "weeks": 56, + "avg_cross_section": 451.4, + "mean_ic": 0.0151, + "ic_t_stat": 1.29, + "ic_positive_pct": 51.8, + "mean_quintile_spread": 0.0041, + "reliable": true + }, + "era_split": { + "pre_2021": { + "signal": "sue_latest", + "weeks": 9, + "avg_cross_section": 398.3, + "mean_ic": 0.0286, + "ic_t_stat": 0.7, + "ic_positive_pct": 55.6, + "mean_quintile_spread": 0.0084, + "reliable": false + }, + "post_2021": { + "signal": "sue_latest", + "weeks": 48, + "avg_cross_section": 461.9, + "mean_ic": 0.0172, + "ic_t_stat": 1.34, + "ic_positive_pct": 64.6, + "mean_quintile_spread": 0.0038, + "reliable": true + } + }, + "signal_eval_identical_cross_sections": { + "sue_latest": { + "signal": "sue_latest", + "weeks": 56, + "avg_cross_section": 450.9, + "mean_ic": 0.0148, + "ic_t_stat": 1.27, + "ic_positive_pct": 51.8, + "mean_quintile_spread": 0.004, + "reliable": true + }, + "mom_12_1": { + "signal": "mom_12_1", + "weeks": 56, + "avg_cross_section": 450.9, + "mean_ic": 0.0195, + "ic_t_stat": 0.74, + "ic_positive_pct": 58.9, + "mean_quintile_spread": 0.0104, + "reliable": true + }, + "mom_12_1_resid": { + "signal": "mom_12_1_resid", + "weeks": 56, + "avg_cross_section": 450.9, + "mean_ic": 0.0262, + "ic_t_stat": 1.07, + "ic_positive_pct": 55.4, + "mean_quintile_spread": 0.0114, + "reliable": true + } + }, + "identical_cross_section_definition": "same week-symbol-forward-return cells where sue_latest, mom_12_1, and mom_12_1_resid are all non-null", + "momentum_conditional_top_quintile": { + "mean_ic": 0.0213, + "ic_t_stat": 1.3, + "weeks": 56, + "avg_cross_section": 89.8, + "population": "top_mom_12_1_quintile_only" + }, + "coverage": { + "symbols_with_live_sue": 501, + "avg_weekly_live_n_all_weeks": 453.1, + "avg_cross_section_n_scored_nonoverlap": 451.4, + "thin_cross_section_lt_100": false, + "warning": null + }, + "scaling": { + "method": "eps_surprise_over_stdev_of_prior_8_surprises_min_4", + "fallback": "not_used", + "counts": { + "standard_scaled_events": 12149, + "events_scaled_from_period_history": 12149, + "price_fallback_events": 0, + "dropped_insufficient_trailing_history": 95, + "dropped_missing_period_alignment": 67, + "dropped_zero_stdev": 0 + }, + "pre_coverage_history_policy": "period-end EPS surprises may scale later events but are never treated as live signals without an announcement date", + "availability": "announce_date_plus_1_trading_day", + "carry_trading_days": 63 + }, + "universe_symbols": 505 + } +} diff --git a/reports/earnings-2b-sue-20260720-dolthub-final.md b/reports/earnings-2b-sue-20260720-dolthub-final.md new file mode 100644 index 0000000..c961567 --- /dev/null +++ b/reports/earnings-2b-sue-20260720-dolthub-final.md @@ -0,0 +1,62 @@ +# Earnings Task 2b - SUE / PEAD + +### Data quality gate + +Approved earnings window: 2020-01-22 to 2026-07-17. Source mode: dolthub_public_bulk_clone. + +| check | result | +|---|---:| +| Prod symbols requested / tradable | 506 / 505 | +| Manifest complete + live counts match | True | +| Prod symbols with pre-2021 bars | 491 (97.2%) | +| SPY benchmark depth | 2649 rows, 2016-01-04 to 2026-07-17 | +| Snapshot depth gate | True | +| Bulk source windows / requests logged | 1/1 / 1 | +| Source repository / pinned commit | https://www.dolthub.com/repositories/post-no-preference/earnings @ 9n0et3hpj9j7vue8f3qsldon3qa5sdjj | +| Source license / upstream provider documented | CC-BY-SA-4.0 / False | +| Existing-source conflicts preserved | 940 rows / 1526 fields | +| Symbols with >=8 announcements | 498 (98.6%) | +| Symbols with >=8 paired announcements | 495 (98.0%) | +| Events with estimate + actual | 12311/12414 (99.2%) | +| Duplicate rows in keyed table | 0 | +| Duplicate / restated payload rows fetched | 0 / 940 | +| Mean announcements per active symbol-year | 4.08 (expected about 4) | +| Symbols far off (<2 or >6/year, incl. zero) | 1 | +| Recognised BMO/AMC/during | 92.8% (reliable=True) | +| Point-in-time policy | announce_date_plus_1_trading_day_for_all_events | +| SUE price fallback | not_used | + +Deduplication: UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment + +Far-off announcement-rate symbols: SPCX + +### Experiment 2b - SUE / post-earnings drift + +Mechanical verdict: **FAIL** - SUE DEAD for this stack + +Identical cross-sections: + +| signal | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| sue_latest | 0.0148 | 1.27 | 56 | 450.9 | 51.8 | true | +| mom_12_1 | 0.0195 | 0.74 | 56 | 450.9 | 58.9 | true | +| mom_12_1_resid | 0.0262 | 1.07 | 56 | 450.9 | 55.4 | true | + +Unconditional SUE grade row: + +| signal | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| sue_latest | 0.0151 | 1.29 | 56 | 451.4 | 51.8 | true | + +Era stability: + +| era | mean IC | t | windows | avg N | IC positive % | reliable | +|---|---:|---:|---:|---:|---:|---| +| pre-2021 | 0.0286 | 0.7 | 9 | 398.3 | 55.6 | false | +| post-2021 | 0.0172 | 1.34 | 48 | 461.9 | 64.6 | true | + +Coverage: 501 symbols with live SUE; avg weekly N=453.1; scored non-overlap avg N=451.4. + +Cross-section is not flagged thin at the registered <100-name read. + +Momentum-conditional top-quintile SUE: mean IC=0.0213, t=1.3, windows=56, avg N=89.8. diff --git a/reports/earnings-backfill-status.json b/reports/earnings-backfill-status.json index 7b22f25..1043641 100644 --- a/reports/earnings-backfill-status.json +++ b/reports/earnings-backfill-status.json @@ -1,15 +1,75 @@ { - "mode": "per_symbol", - "fmp_requests": 25, - "events_written_this_run": 2541, - "total_events": 5612, - "symbols_done": 48, - "symbols_universe": 506, - "announce_date_range": { - "min": "1985-08-31", - "max": "2026-07-16" + "mode": "dolthub_public_bulk_clone", + "window": { + "from": "2020-01-22", + "to": "2026-07-17" }, - "events_with_actual_and_estimate": 5018, - "budget": 25, - "complete": false + "coverage_amendment": { + "approved_by_user": true, + "reason": "FMP free tier blocks historical bulk earnings", + "original_start": "2016-01-04", + "amended_announcement_start": "2020-01-22" + }, + "source": { + "repository": "https://www.dolthub.com/repositories/post-no-preference/earnings", + "commit": "9n0et3hpj9j7vue8f3qsldon3qa5sdjj", + "license": "CC-BY-SA-4.0", + "upstream_provider_documented": false + }, + "bulk_windows_total": 1, + "bulk_windows_done": 1, + "bulk_requests_logged_total": 1, + "bulk_exports": 2, + "calendar": { + "raw_rows": 117482, + "universe_rows_in_window": 12342, + "deduped_rows_in_window": 12342, + "duplicate_rows": 0, + "restated_rows": 0 + }, + "eps_history": { + "raw_rows": 165050, + "universe_rows": 18515, + "deduped_rows": 18515, + "duplicate_rows": 0, + "restated_rows": 0, + "complete_actual_and_estimate": 18304 + }, + "pairing": { + "method": "minimum-cost monotonic alignment per symbol", + "allowed_announce_minus_period_end_days": [ + -14, + 90 + ], + "matched_calendar_events": 12271, + "unmatched_calendar_events": 71, + "unmatched_periods_in_pairing_window": 538, + "announce_minus_period_end_days": { + "min": -10, + "median": 30, + "max": 89 + }, + "pre_2020_eps_history_use": "trailing_surprise_stdev_only; never treated as an announcement or live signal event" + }, + "duplicate_rows_logged_total": 0, + "restated_rows_logged_total": 940, + "conflicting_existing_rows": 940, + "conflicting_existing_fields": 1526, + "preserved_existing_fields": 2945, + "existing_enrichment_events_not_in_dolthub_calendar": 72, + "dedupe_policy": "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one calendar row per key; preserve existing non-null session/EPS values from the prior FMP/Alpha Vantage partial backfill, then fill nulls and all remaining symbols from DoltHub; attach DoltHub period-end alignment", + "events_in_window": 12414, + "events_with_actual_and_estimate": 12311, + "symbols_done": 506, + "symbols_universe": 506, + "symbols_with_dolthub_calendar": 504, + "symbols_without_dolthub_calendar": [ + "RHM", + "SPCX" + ], + "announce_date_range": { + "min": "2020-01-22", + "max": "2026-07-17" + }, + "complete": true } diff --git a/reports/earnings-gap-sue-20260719-093129.json b/reports/earnings-gap-sue-20260719-093129.json deleted file mode 100644 index 57dc35f..0000000 --- a/reports/earnings-gap-sue-20260719-093129.json +++ /dev/null @@ -1,331 +0,0 @@ -{ - "generated_at": "2026-07-19T09:31:29.078611", - "data_provenance": { - "snapshot": "C:\\Workspace\\signal-platform\\backtest_snapshots\\prod.sqlite", - "n_earnings_events": 5612, - "backfill_meta": { - "done": 48, - "universe_tickers": 506 - }, - "announce_range": { - "min": "1985-08-31", - "max": "2026-07-16" - }, - "with_actual_and_estimate": 5018 - }, - "experiment_2a": { - "sim_summary": { - "sharpe": 2.09, - "sharpe_se": 0.497, - "cagr_pct": 51.6, - "max_drawdown_pct": 21.4, - "trades": 322, - "total_return_pct": 424.6 - }, - "n_trades_parsed": 322, - "q1_losses_worse_than_minus_1r": { - "n_losses_lt_minus_1r": 28, - "n_with_earnings_in_hold": 1, - "fraction_with_earnings": 0.0357, - "all_trades_with_earnings_in_hold": 14, - "fraction_all_trades_with_earnings": 0.0435 - }, - "q2_entry_within_3d_before_announce": { - "pre_earn_entries": { - "n": 4, - "mean": 1.9379, - "win_rate": 0.5, - "p05": -1.2428, - "p25": -0.8833, - "p50": 1.1209, - "p75": 3.942, - "p95": 6.2623, - "min": -1.3327, - "max": 6.8424 - }, - "other_entries": { - "n": 318, - "mean": 0.6965, - "win_rate": 0.3711, - "p05": -1.1052, - "p25": -1.0, - "p50": -0.8259, - "p75": 2.1053, - "p95": 6.077, - "min": -3.2587, - "max": 12.8654 - }, - "all_entries": { - "n": 322, - "mean": 0.7119, - "win_rate": 0.3727, - "p05": -1.1209, - "p25": -1.0, - "p50": -0.8251, - "p75": 2.1595, - "p95": 6.2246, - "min": -3.2587, - "max": 12.8654 - }, - "tail_trim_note": "Compare p95/max and mean of pre_earn vs other. Rising win_rate with falling mean/p95 = right-tail trim red flag." - }, - "note": "REPORT-ONLY \u2014 no filter shipped." - }, - "experiment_2b": { - "signal_eval_side_by_side": { - "mom_12_1": { - "signal": "mom_12_1", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0531, - "ic_t_stat": 1.61, - "ic_positive_pct": 65.7, - "mean_quintile_spread": 0.0206, - "reliable": true - }, - "mom_12_1_resid": { - "signal": "mom_12_1_resid", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0552, - "ic_t_stat": 1.98, - "ic_positive_pct": 60.0, - "mean_quintile_spread": 0.0207, - "reliable": true - }, - "mom_12_1_sector_resid": { - "signal": "mom_12_1_sector_resid", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0578, - "ic_t_stat": 2.34, - "ic_positive_pct": 65.7, - "mean_quintile_spread": 0.0245, - "reliable": true - }, - "mom_12_1_sector_demeaned": { - "signal": "mom_12_1_sector_demeaned", - "weeks": 35, - "avg_cross_section": 496.7, - "mean_ic": 0.034, - "ic_t_stat": 1.32, - "ic_positive_pct": 62.9, - "mean_quintile_spread": 0.0154, - "reliable": true - }, - "sue_latest": { - "signal": "sue_latest", - "weeks": 44, - "avg_cross_section": 47.4, - "mean_ic": 0.0172, - "ic_t_stat": 0.6, - "ic_positive_pct": 47.7, - "mean_quintile_spread": 0.0064, - "reliable": true - }, - "fip_id": { - "signal": "fip_id", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": -0.045, - "ic_t_stat": -2.91, - "ic_positive_pct": 25.7, - "mean_quintile_spread": -0.0168, - "reliable": true - } - }, - "signal_eval_identical_sue_subset": { - "mom_12_1": { - "signal": "mom_12_1", - "weeks": 35, - "avg_cross_section": 47.3, - "mean_ic": -0.0174, - "ic_t_stat": -0.42, - "ic_positive_pct": 45.7, - "mean_quintile_spread": 0.0077, - "reliable": true - }, - "mom_12_1_resid": { - "signal": "mom_12_1_resid", - "weeks": 35, - "avg_cross_section": 47.3, - "mean_ic": -0.0104, - "ic_t_stat": -0.27, - "ic_positive_pct": 51.4, - "mean_quintile_spread": 0.0075, - "reliable": true - }, - "sue_latest": { - "signal": "sue_latest", - "weeks": 44, - "avg_cross_section": 47.4, - "mean_ic": 0.0172, - "ic_t_stat": 0.6, - "ic_positive_pct": 47.7, - "mean_quintile_spread": 0.0064, - "reliable": true - } - }, - "identical_subset_note": "Mom baselines re-scored only on (week, symbol) cells where SUE exists. Use this table when backfill is incomplete \u2014 full-universe mom N is not comparable.", - "full_signal_eval": [ - { - "signal": "vol_6m", - "weeks": 39, - "avg_cross_section": 498.2, - "mean_ic": 0.0609, - "ic_t_stat": 1.48, - "ic_positive_pct": 64.1, - "mean_quintile_spread": 0.0337, - "reliable": true - }, - { - "signal": "mom_12_1_sector_resid", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0578, - "ic_t_stat": 2.34, - "ic_positive_pct": 65.7, - "mean_quintile_spread": 0.0245, - "reliable": true - }, - { - "signal": "mom_12_1_resid", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0552, - "ic_t_stat": 1.98, - "ic_positive_pct": 60.0, - "mean_quintile_spread": 0.0207, - "reliable": true - }, - { - "signal": "mom_12_1", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": 0.0531, - "ic_t_stat": 1.61, - "ic_positive_pct": 65.7, - "mean_quintile_spread": 0.0206, - "reliable": true - }, - { - "signal": "mom_12_1_sector_demeaned", - "weeks": 35, - "avg_cross_section": 496.7, - "mean_ic": 0.034, - "ic_t_stat": 1.32, - "ic_positive_pct": 62.9, - "mean_quintile_spread": 0.0154, - "reliable": true - }, - { - "signal": "sue_latest", - "weeks": 44, - "avg_cross_section": 47.4, - "mean_ic": 0.0172, - "ic_t_stat": 0.6, - "ic_positive_pct": 47.7, - "mean_quintile_spread": 0.0064, - "reliable": true - }, - { - "signal": "trend_200", - "weeks": 37, - "avg_cross_section": 497.9, - "mean_ic": 0.0161, - "ic_t_stat": 0.44, - "ic_positive_pct": 59.5, - "mean_quintile_spread": 0.006, - "reliable": true - }, - { - "signal": "reversal_1m", - "weeks": 43, - "avg_cross_section": 498.7, - "mean_ic": 0.0059, - "ic_t_stat": 0.22, - "ic_positive_pct": 53.5, - "mean_quintile_spread": 0.0053, - "reliable": true - }, - { - "signal": "mom_6_1", - "weeks": 39, - "avg_cross_section": 498.2, - "mean_ic": 0.0051, - "ic_t_stat": 0.21, - "ic_positive_pct": 56.4, - "mean_quintile_spread": 0.0087, - "reliable": true - }, - { - "signal": "mom_3_1", - "weeks": 42, - "avg_cross_section": 498.5, - "mean_ic": -0.0064, - "ic_t_stat": -0.25, - "ic_positive_pct": 50.0, - "mean_quintile_spread": 0.0046, - "reliable": true - }, - { - "signal": "high_52w", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": -0.0086, - "ic_t_stat": -0.26, - "ic_positive_pct": 54.3, - "mean_quintile_spread": -0.0088, - "reliable": true - }, - { - "signal": "fip_id", - "weeks": 35, - "avg_cross_section": 497.7, - "mean_ic": -0.045, - "ic_t_stat": -2.91, - "ic_positive_pct": 25.7, - "mean_quintile_spread": -0.0168, - "reliable": true - } - ], - "sue_grade": { - "green": false, - "checks": { - "mean_ic": 0.0172, - "sign_positive": true, - "abs_ge_0_03": false, - "reliable": true, - "ic_t_stat": 0.6, - "weeks": 44 - }, - "reason": "iron rule not met", - "row": { - "signal": "sue_latest", - "weeks": 44, - "avg_cross_section": 47.4, - "mean_ic": 0.0172, - "ic_t_stat": 0.6, - "ic_positive_pct": 47.7, - "mean_quintile_spread": 0.0064, - "reliable": true - } - }, - "momentum_conditional_sue": { - "mean_ic": -0.0065, - "ic_t_stat": -0.1, - "weeks": 35, - "note": "IC of sue_latest within top mom_12_1 quintile (non-overlapping weeks)" - }, - "sue_coverage": { - "symbols_with_sue": 48, - "avg_weeks_with_sue": 47.1, - "weeks_with_min_cross_section": 256 - } - }, - "verdict": "PARK", - "verdict_detail": "SUE IC=0.0172 below iron bar or unreliable; keep data, no wire.", - "human_next": "- No SUE book change.\n- Read 2a tails before considering any earnings-avoid filter.", - "report_path": "reports/earnings-gap-sue-20260719-093129.json", - "fmp_note": "Bulk earnings-calendar is paid (402 on free tier). Backfill used per-symbol /stable/earnings; see earnings-backfill-status.json." -} diff --git a/reports/earnings-gap-sue-20260719-093129.md b/reports/earnings-gap-sue-20260719-093129.md deleted file mode 100644 index 4211c84..0000000 --- a/reports/earnings-gap-sue-20260719-093129.md +++ /dev/null @@ -1,202 +0,0 @@ -# Earnings gap diagnostic + SUE / PEAD (Tier-1 alpha research) - -**Status:** **PARK** (incomplete earnings coverage; SUE fails iron rule on available sample). -**Branch:** `research/earnings-gap-and-sue` -**Production impact:** none. Local research only. **No filters shipped from 2a.** -**Artifacts:** `reports/earnings-gap-sue-20260719-093129.json` (+ companion `.md`) - ---- - -## Pre-registration (locked before first research run) - -### Data - -- Historical earnings calendar for the production universe over the full snapshot - window (and deeper if the feed provides it). -- Preferred source: FMP **date-range earnings-calendar** (bulk). If unavailable on - free tier, fall back to per-symbol `/stable/earnings` with request accounting. -- Store in a real local table `earnings_events` (symbol + announce_date key). -- Point-in-time: a surprise is usable only from **announce date + 1 trading day** - onward. - -### Experiment 2a — earnings-gap risk (defense, report-only) - -Join simulated production-config trades (`fill_mode=close`) with earnings dates. - -**Pre-registered questions:** - -1. What fraction of losses worse than **−1R** occur with an earnings announcement - **between entry and exit** (inclusive of the holding window)? -2. What is the mean R of entries taken within **3 trading days BEFORE** an - announcement vs all other entries — report **both tails** of the R - distribution (rule 4: any earnings-avoid entry filter is presumed guilty of - right-tail trimming until the win distribution shows otherwise)? - -**Output:** distributions and counts only. -**No filter is shipped.** If numbers argue for a filter → report and stop. - -### Experiment 2b — SUE / PEAD (offense) - -Signal `sue_latest`: - -\[ -\text{SUE} = \frac{\text{actual} - \text{estimate}}{\sigma(\text{trailing 8 surprises})} -\] - -Fallback if estimate history is thin: scale surprise by price. -Carry forward from announce+1 for **63 trading days**, else NaN (name drops out -of that cross-section). - -**Iron rule (IC harness):** mean weekly Spearman IC on non-overlapping weeks; -\|mean IC\| ≥ ~0.03, **positive** sign (drift), `reliable: true` (≥12 windows). - -Always side-by-side with `mom_12_1` and `mom_12_1_resid` on **identical** -cross-sections. - -Also report **momentum-conditional** IC (within top momentum quintile). - -**If it passes iron rule:** STOP and report. Book-integration design is a -separate human-approved step — do not wire. - -### Verdict labels - -| label | meaning | -|---|---| -| **PROMOTE** | (2b only) iron rule cleared → human designs tilt/gate | -| **PARK** | Interesting but incomplete / weak | -| **DEAD** | No edge / diagnostic argues against action | -| **REPORT-ONLY** | (2a) always — never auto-filter | - ---- - -## Data provenance - -| item | result | -|---|---| -| Snapshot | `backtest_snapshots/prod.sqlite` (506 names) | -| FMP bulk `earnings-calendar` | **402 Premium** — not available on free tier | -| FMP per-symbol `/stable/earnings` | used; hit daily rate limit ~225 reqs | -| Alpha Vantage `EARNINGS` | used for +24 symbols (announce = `reportedDate`) | -| Symbols with events | **48 / 506 (9.5%)** | -| Total events | 5,612 (5,018 with actual+estimate) | -| Announce range | 1985-08-31 → 2026-07-16 | -| FMP requests (first day) | 260 FMP + 25 AV (see `reports/earnings-backfill-status.json`) | - -**Incomplete backfill is first-class.** 2a under-detects earnings overlaps; 2b SUE -cross-section averages **~47 names**, not ~500. Resume: - -```bash -# Day N (FMP free ~250/day; AV free ~25/day — prefer FMP after reset) -python scripts/backfill_earnings_events.py \ - --snapshot backtest_snapshots/prod.sqlite \ - --provider fmp --force-symbol --limit 250 --sleep 0.4 - -# When done==506: -python scripts/run_earnings_research.py \ - --snapshot backtest_snapshots/prod.sqlite \ - --workers 6 --allow-spawn -``` - ---- - -## Results - -Generated: `2026-07-19T09:31:29` - -### 2a — Earnings-gap risk (report-only) - -Production book sim: Sharpe 2.09 (SE 0.497), CAGR 51.6%, max DD 21.4%, **322 trades**, -`fill_mode=close`. - -#### Q1 — Losses worse than −1R with earnings in hold - -| metric | value | -|---|---:| -| n losses < −1R | 28 | -| of which earnings in hold | **1** | -| fraction | **3.6%** | -| all trades with earnings in hold | 14 / 322 (4.4%) | - -**Read:** On incomplete earnings labels this is a **lower bound** on earnings -overlap, not a clean “earnings rarely hurt.” Do **not** conclude earnings risk is -immaterial until coverage ≥ ~95% of the book’s names. - -#### Q2 — Entry within 3 trading days before announce (both tails) - -| cohort | n | mean R | win rate | p05 | p50 | p95 | max | -|---|---:|---:|---:|---:|---:|---:|---:| -| pre-earn (≤3d before) | **4** | 1.94 | 50% | −1.24 | 1.12 | 6.26 | 6.84 | -| other | 318 | 0.70 | 37% | −1.11 | −0.83 | 6.08 | **12.87** | -| all | 322 | 0.71 | 37% | −1.12 | −0.83 | 6.22 | 12.87 | - -**Tail-trim presumption:** n=4 is not a sample. Point estimate does **not** show -right-tail destruction of pre-earn entries (p95 similar; max actually higher in -“other”). **No earnings-avoid filter is supported.** Re-run after full backfill. - ---- - -### 2b — SUE / PEAD IC - -#### Full-universe harness (mom on ~500; SUE only where labeled) - -| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable | -|---|---:|---:|---:|---:|---| -| mom_12_1_sector_resid | 0.0578 | 2.34 | 35 | 497.7 | true | -| mom_12_1_resid | 0.0552 | 1.98 | 35 | 497.7 | true | -| mom_12_1 | 0.0531 | 1.61 | 35 | 497.7 | true | -| **sue_latest** | **0.0172** | **0.6** | 44 | **47.4** | true | -| fip_id | −0.045 | −2.91 | 35 | 497.7 | true | - -#### Identical SUE subset (fair side-by-side — use this while coverage is thin) - -| signal | mean_ic | ic_t_stat | weeks | avg_N | -|---|---:|---:|---:|---:| -| sue_latest | 0.0172 | 0.6 | 44 | 47.4 | -| mom_12_1 | −0.0174 | −0.42 | 35 | 47.3 | -| mom_12_1_resid | −0.0104 | −0.27 | 35 | 47.3 | - -On the thin labeled subset, momentum itself is noise — so the subset is not yet -a meaningful PEAD test. - -#### Momentum-conditional SUE (top mom quintile) - -| metric | value | -|---|---:| -| mean IC | **−0.0065** | -| t | −0.1 | -| weeks | 35 | - -Wrong sign vs “ride positive surprises inside the momentum gate.” - -**Iron rule:** fail (\|IC\| 0.017 < 0.03; t 0.6). **No promote.** - ---- - -## Verdict - -| piece | verdict | -|---|---| -| **2a earnings-gap** | **REPORT-ONLY** — no filter. Coverage too thin for risk claims; tails do not argue for an avoid-filter on n=4. | -| **2b SUE** | **PARK** (effectively not green). Mild positive IC on ~48 names; fails iron bar; mom-conditional flat/negative. Re-score after full backfill before DEAD. | -| **Production** | **no change** | - ---- - -## What a human must decide next - -1. Resume multi-day earnings backfill to **506/506**, then re-run - `run_earnings_research.py` (heavy — MacBook OK). -2. Do **not** ship an earnings-avoid entry filter from 2a. -3. Do **not** wire SUE until a full-coverage IC clears the iron rule (and - preferably mom-conditional > 0). -4. Do not merge into main strategy docs without review. - ---- - -## Implementation notes - -| piece | role | -|---|---| -| `scripts/backfill_earnings_events.py` | bulk attempt → FMP/AV per-symbol; `earnings_events` + meta on snapshot | -| `scripts/run_earnings_research.py` | 2a trade join + 2b SUE IC / mom-conditional | -| Snapshot table `earnings_events` | real table (not SystemSetting JSON) | diff --git a/scripts/backfill_earnings_events.py b/scripts/backfill_earnings_events.py index 665eabc..570e117 100644 --- a/scripts/backfill_earnings_events.py +++ b/scripts/backfill_earnings_events.py @@ -1,15 +1,13 @@ -"""Backfill historical earnings into a snapshot ``earnings_events`` table. +"""Bulk-only historical earnings backfill for a local SQLite snapshot. -Prefers FMP bulk date-range ``earnings-calendar`` (one request per window). -On free-tier 402/403, falls back to per-symbol ``/stable/earnings`` with -resume support and request counting (≈250 req/day free tier). +The job uses FMP's date-range earnings-calendar endpoint. One request covers all +symbols in a date window; per-symbol endpoints are intentionally not available +in this task runner. Successful windows are committed independently so a later +run resumes after a daily quota boundary without repeating completed windows. -Research only — writes to the local snapshot SQLite, never production Postgres. - -Example -------- - python scripts/backfill_earnings_events.py \\ - --snapshot backtest_snapshots/prod.sqlite --limit 250 +Example: + python scripts/backfill_earnings_events.py --snapshot backtest_snapshots/prod.sqlite \ + --from-date 2012-01-01 --window-days 30 --limit 250 """ from __future__ import annotations @@ -17,10 +15,11 @@ from __future__ import annotations import argparse import asyncio import json +import math import sys -import time from datetime import date, datetime, timedelta, timezone from pathlib import Path +from typing import Any import httpx from sqlalchemy import create_engine, text @@ -34,7 +33,7 @@ from app.ssl_bootstrap import bootstrap_ssl # noqa: E402 bootstrap_ssl() FMP_STABLE = "https://financialmodelingprep.com/stable" -DDL = """ +EVENTS_DDL = """ CREATE TABLE IF NOT EXISTS earnings_events ( id INTEGER PRIMARY KEY, symbol TEXT NOT NULL, @@ -49,7 +48,6 @@ CREATE TABLE IF NOT EXISTS earnings_events ( UNIQUE(symbol, announce_date) ) """ -# Side table tracks which symbols have been fully pulled (resume). META_DDL = """ CREATE TABLE IF NOT EXISTS earnings_backfill_meta ( symbol TEXT PRIMARY KEY, @@ -59,239 +57,238 @@ CREATE TABLE IF NOT EXISTS earnings_backfill_meta ( note TEXT ) """ +WINDOW_DDL = """ +CREATE TABLE IF NOT EXISTS earnings_backfill_windows ( + from_date TEXT NOT NULL, + to_date TEXT NOT NULL, + status TEXT NOT NULL, + requests INTEGER NOT NULL DEFAULT 0, + rows_raw INTEGER NOT NULL DEFAULT 0, + rows_universe INTEGER NOT NULL DEFAULT 0, + duplicate_rows INTEGER NOT NULL DEFAULT 0, + restated_rows INTEGER NOT NULL DEFAULT 0, + updated_at TEXT NOT NULL, + note TEXT, + PRIMARY KEY(from_date, to_date) +) +""" def _parse_args() -> argparse.Namespace: - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("--snapshot", default="backtest_snapshots/prod.sqlite") - p.add_argument( - "--from-date", - default="2020-01-01", - help="Bulk calendar window start (also filters per-symbol rows).", - ) - p.add_argument( - "--to-date", - default=None, - help="Bulk calendar window end (default: today).", - ) - p.add_argument( - "--limit", - type=int, - default=250, - help="Max FMP requests this run (free-tier cushion).", - ) - p.add_argument("--sleep", type=float, default=0.35) - p.add_argument( - "--force-symbol", + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--snapshot", default="backtest_snapshots/prod.sqlite") + parser.add_argument("--from-date", default="2012-01-01") + parser.add_argument("--to-date", default=None) + parser.add_argument("--window-days", type=int, default=30) + parser.add_argument("--limit", type=int, default=250) + parser.add_argument("--sleep", type=float, default=0.35) + parser.add_argument( + "--refetch-windows", action="store_true", - help="Skip bulk attempt; go straight to per-symbol.", + help="Re-fetch date windows already logged as done.", ) - p.add_argument( - "--refetch-done", - action="store_true", - help="Re-fetch symbols already marked done.", - ) - p.add_argument( - "--provider", - choices=("fmp", "alpha_vantage", "auto"), - default="auto", - help="Earnings provider. auto tries FMP bulk then FMP/AV per-symbol.", - ) - return p.parse_args() + return parser.parse_args() def _ensure_tables(engine) -> None: with engine.begin() as conn: - conn.execute(text(DDL)) + conn.execute(text(EVENTS_DDL)) conn.execute(text(META_DDL)) + conn.execute(text(WINDOW_DDL)) -def _upsert_events(conn, rows: list[dict], source: str) -> int: - if not rows: - return 0 - now = datetime.now(timezone.utc).isoformat() - written = 0 - for r in rows: - conn.execute( - text( - """ - INSERT INTO earnings_events ( - symbol, announce_date, announce_time, - eps_estimate, eps_actual, revenue_estimate, revenue_actual, - source, fetched_at - ) VALUES ( - :symbol, :announce_date, :announce_time, - :eps_estimate, :eps_actual, :revenue_estimate, :revenue_actual, - :source, :fetched_at - ) - ON CONFLICT(symbol, announce_date) DO UPDATE SET - announce_time=excluded.announce_time, - eps_estimate=excluded.eps_estimate, - eps_actual=excluded.eps_actual, - revenue_estimate=excluded.revenue_estimate, - revenue_actual=excluded.revenue_actual, - source=excluded.source, - fetched_at=excluded.fetched_at - """ - ), - { - "symbol": r["symbol"], - "announce_date": r["announce_date"], - "announce_time": r.get("announce_time"), - "eps_estimate": r.get("eps_estimate"), - "eps_actual": r.get("eps_actual"), - "revenue_estimate": r.get("revenue_estimate"), - "revenue_actual": r.get("revenue_actual"), - "source": source, - "fetched_at": now, - }, - ) - written += 1 - return written +def _number(value: Any) -> float | None: + if value is None or value == "": + return None + try: + result = float(value) + except (TypeError, ValueError): + return None + return result if math.isfinite(result) else None + + +def _normalise_session(value: Any) -> str | None: + if value is None: + return None + cleaned = str(value).strip().lower().replace("_", " ").replace("-", " ") + aliases = { + "bmo": "bmo", + "before market open": "bmo", + "before open": "bmo", + "amc": "amc", + "after market close": "amc", + "after close": "amc", + "during market hours": "during", + "dmh": "during", + } + return aliases.get(cleaned, cleaned or None) def _parse_bulk_item(item: dict) -> dict | None: - sym = (item.get("symbol") or "").strip().upper() - d = item.get("date") or item.get("earningsDate") - if not sym or not d: + symbol = str(item.get("symbol") or "").strip().upper().replace(".", "-") + raw_date = item.get("date") or item.get("earningsDate") + if not symbol or not raw_date: return None return { - "symbol": sym.replace(".", "-"), - "announce_date": str(d)[:10], - "announce_time": item.get("time") or item.get("announceTime"), - "eps_estimate": _f(item.get("epsEstimated") or item.get("estimatedEarning")), - "eps_actual": _f(item.get("epsActual") or item.get("eps")), - "revenue_estimate": _f(item.get("revenueEstimated")), - "revenue_actual": _f(item.get("revenueActual")), + "symbol": symbol, + "announce_date": str(raw_date)[:10], + "announce_time": _normalise_session( + item.get("time") or item.get("announceTime") + ), + "eps_estimate": _number( + item.get("epsEstimated") + if item.get("epsEstimated") is not None + else item.get("estimatedEarning") + ), + "eps_actual": _number( + item.get("epsActual") + if item.get("epsActual") is not None + else item.get("eps") + ), + "revenue_estimate": _number(item.get("revenueEstimated")), + "revenue_actual": _number(item.get("revenueActual")), } -def _parse_symbol_item(item: dict, symbol: str) -> dict | None: - d = item.get("date") - if not d: - return None - return { - "symbol": symbol.replace(".", "-").upper(), - "announce_date": str(d)[:10], - "announce_time": item.get("time"), - "eps_estimate": _f(item.get("epsEstimated")), - "eps_actual": _f(item.get("epsActual")), - "revenue_estimate": _f(item.get("revenueEstimated")), - "revenue_actual": _f(item.get("revenueActual")), - } +def _windows(start: date, end: date, window_days: int) -> list[tuple[date, date]]: + if window_days < 1: + raise ValueError("window_days must be positive") + result: list[tuple[date, date]] = [] + cursor = start + while cursor <= end: + window_end = min(end, cursor + timedelta(days=window_days - 1)) + result.append((cursor, window_end)) + cursor = window_end + timedelta(days=1) + return result -def _f(v) -> float | None: - if v is None or v == "": - return None - try: - return float(v) - except (TypeError, ValueError): - return None +def _dedupe_bulk_rows(rows: list[dict]) -> tuple[list[dict], int, int]: + """Prefer the most complete duplicate; use the later row as the tie-break.""" + fields = ( + "announce_time", + "eps_estimate", + "eps_actual", + "revenue_estimate", + "revenue_actual", + ) + chosen: dict[tuple[str, str], dict] = {} + duplicate_extras = 0 + restated = 0 + for row in rows: + key = (str(row["symbol"]), str(row["announce_date"])) + previous = chosen.get(key) + if previous is None: + chosen[key] = row + continue + duplicate_extras += 1 + if any( + previous.get(field) is not None + and row.get(field) is not None + and previous.get(field) != row.get(field) + for field in fields + ): + restated += 1 + previous_score = sum(previous.get(field) is not None for field in fields) + new_score = sum(row.get(field) is not None for field in fields) + if new_score >= previous_score: + chosen[key] = row + return list(chosen.values()), duplicate_extras, restated -async def _try_bulk( - client: httpx.AsyncClient, - api_key: str, +def _upsert_events(conn, rows: list[dict]) -> int: + if not rows: + return 0 + fetched_at = datetime.now(timezone.utc).isoformat() + statement = text( + """ + INSERT INTO earnings_events ( + symbol, announce_date, announce_time, eps_estimate, eps_actual, + revenue_estimate, revenue_actual, source, fetched_at + ) VALUES ( + :symbol, :announce_date, :announce_time, :eps_estimate, :eps_actual, + :revenue_estimate, :revenue_actual, 'fmp_earnings_calendar', :fetched_at + ) + ON CONFLICT(symbol, announce_date) DO UPDATE SET + announce_time=COALESCE(excluded.announce_time, earnings_events.announce_time), + eps_estimate=COALESCE(excluded.eps_estimate, earnings_events.eps_estimate), + eps_actual=COALESCE(excluded.eps_actual, earnings_events.eps_actual), + revenue_estimate=COALESCE(excluded.revenue_estimate, earnings_events.revenue_estimate), + revenue_actual=COALESCE(excluded.revenue_actual, earnings_events.revenue_actual), + source=excluded.source, + fetched_at=excluded.fetched_at + """ + ) + conn.execute(statement, [{**row, "fetched_at": fetched_at} for row in rows]) + return len(rows) + + +async def _fetch_bulk_window( + client: httpx.AsyncClient, api_key: str, start: date, end: date +) -> tuple[list[dict], int, str | None]: + response = await client.get( + f"{FMP_STABLE}/earnings-calendar", + params={"from": start.isoformat(), "to": end.isoformat(), "apikey": api_key}, + ) + if response.status_code in (402, 403): + return [], response.status_code, "bulk_endpoint_unavailable" + if response.status_code == 429: + return [], response.status_code, "daily_limit_reached" + response.raise_for_status() + payload = response.json() + if not isinstance(payload, list): + return [], response.status_code, f"unexpected_payload:{type(payload).__name__}" + rows = [] + for item in payload: + if isinstance(item, dict): + parsed = _parse_bulk_item(item) + if parsed: + rows.append(parsed) + return rows, response.status_code, None + + +def _write_window_status( + engine, + *, start: date, end: date, - *, - window_days: int = 30, -) -> tuple[list[dict], int, str | None]: - """Return (rows, requests_used, error_note).""" - rows: list[dict] = [] - reqs = 0 - cur = start - while cur <= end: - win_end = min(end, cur + timedelta(days=window_days - 1)) - resp = await client.get( - f"{FMP_STABLE}/earnings-calendar", - params={ - "from": cur.isoformat(), - "to": win_end.isoformat(), - "apikey": api_key, + status: str, + raw_n: int = 0, + universe_n: int = 0, + duplicate_n: int = 0, + restated_n: int = 0, + note: str | None = None, +) -> None: + with engine.begin() as conn: + conn.execute( + text( + """ + INSERT INTO earnings_backfill_windows( + from_date, to_date, status, requests, rows_raw, rows_universe, + duplicate_rows, restated_rows, updated_at, note + ) VALUES (:a, :b, :status, 1, :raw, :uni, :dup, :rest, :now, :note) + ON CONFLICT(from_date, to_date) DO UPDATE SET + status=excluded.status, + requests=earnings_backfill_windows.requests + 1, + rows_raw=excluded.rows_raw, + rows_universe=excluded.rows_universe, + duplicate_rows=excluded.duplicate_rows, + restated_rows=excluded.restated_rows, + updated_at=excluded.updated_at, + note=excluded.note + """ + ), + { + "a": start.isoformat(), + "b": end.isoformat(), + "status": status, + "raw": raw_n, + "uni": universe_n, + "dup": duplicate_n, + "rest": restated_n, + "now": datetime.now(timezone.utc).isoformat(), + "note": note, }, ) - reqs += 1 - if resp.status_code in (402, 403): - return [], reqs, f"bulk_unavailable status={resp.status_code}" - if resp.status_code == 429: - return rows, reqs, "rate_limited" - resp.raise_for_status() - data = resp.json() - if not isinstance(data, list): - return [], reqs, f"unexpected bulk payload type={type(data)}" - for item in data: - if isinstance(item, dict): - parsed = _parse_bulk_item(item) - if parsed: - rows.append(parsed) - cur = win_end + timedelta(days=1) - return rows, reqs, None - - -async def _fetch_symbol( - client: httpx.AsyncClient, api_key: str, symbol: str -) -> list[dict]: - resp = await client.get( - f"{FMP_STABLE}/earnings", - params={"symbol": symbol, "apikey": api_key}, - ) - if resp.status_code == 429: - raise RuntimeError("rate_limited") - if resp.status_code == 402: - return [] - resp.raise_for_status() - data = resp.json() - if not isinstance(data, list): - return [] - out: list[dict] = [] - for item in data: - if isinstance(item, dict): - parsed = _parse_symbol_item(item, symbol) - if parsed: - out.append(parsed) - return out - - -async def _fetch_symbol_alpha_vantage( - client: httpx.AsyncClient, api_key: str, symbol: str -) -> list[dict]: - """Alpha Vantage EARNINGS — includes reportedDate (announce) + estimate/actual.""" - resp = await client.get( - "https://www.alphavantage.co/query", - params={"function": "EARNINGS", "symbol": symbol, "apikey": api_key}, - ) - if resp.status_code == 429: - raise RuntimeError("rate_limited") - resp.raise_for_status() - data = resp.json() - if not isinstance(data, dict): - return [] - note = str(data.get("Note") or data.get("Information") or "") - if "rate limit" in note.lower() or "Thank you for using Alpha Vantage" in note: - raise RuntimeError("rate_limited") - if data.get("Error Message"): - return [] - quarterly = data.get("quarterlyEarnings") or [] - out: list[dict] = [] - for item in quarterly: - if not isinstance(item, dict): - continue - # Prefer announce (reportedDate); fall back to fiscal end (worse PIT). - ad = item.get("reportedDate") or item.get("fiscalDateEnding") - if not ad: - continue - out.append({ - "symbol": symbol.replace(".", "-").upper(), - "announce_date": str(ad)[:10], - "announce_time": item.get("reportTime"), - "eps_estimate": _f(item.get("estimatedEPS")), - "eps_actual": _f(item.get("reportedEPS")), - "revenue_estimate": None, - "revenue_actual": None, - }) - return out async def _main() -> None: @@ -304,228 +301,214 @@ async def _main() -> None: if not settings.fmp_api_key: raise SystemExit("FMP_API_KEY required") - start = date.fromisoformat(args.from_date) end = date.fromisoformat(args.to_date) if args.to_date else date.today() - engine = create_engine( - f"sqlite:///{snapshot.resolve().as_posix()}", - future=True, - ) - _ensure_tables(engine) + if start > end: + raise SystemExit("--from-date must not be after --to-date") + engine = create_engine(f"sqlite:///{snapshot.resolve().as_posix()}", future=True) + _ensure_tables(engine) + all_windows = _windows(start, end, int(args.window_days)) with engine.connect() as conn: symbols = [ - str(r[0]).upper().replace(".", "-") - for r in conn.execute(text("SELECT symbol FROM tickers ORDER BY symbol")) + str(row[0]).upper().replace(".", "-") + for row in conn.execute(text("SELECT symbol FROM tickers ORDER BY symbol")) ] - done = set() - if not args.refetch_done: - done = { - str(r[0]) - for r in conn.execute( - text( - "SELECT symbol FROM earnings_backfill_meta " - "WHERE status='done' AND n_events > 0" - ) + completed = { + (str(row[0]), str(row[1])) + for row in conn.execute( + text( + "SELECT from_date, to_date FROM earnings_backfill_windows " + "WHERE status='done'" ) - } - - pending = [s for s in symbols if s not in done] - print(f"Snapshot: {snapshot}") - print(f"Universe: {len(symbols)}; pending: {len(pending)}; done: {len(done)}") - print(f"Window filter: {start} → {end}") - print(f"Provider: {args.provider}") - - req_budget = int(args.limit) - reqs_used = 0 - events_written = 0 - mode = "per_symbol" - use_av = args.provider in ("alpha_vantage", "auto") and bool( - getattr(settings, "alpha_vantage_api_key", "") - ) - use_fmp = args.provider in ("fmp", "auto") and bool(settings.fmp_api_key) - - async with httpx.AsyncClient(timeout=60.0) as client: - if ( - not args.force_symbol - and req_budget > 0 - and use_fmp - and args.provider != "alpha_vantage" - ): - print("Attempting bulk earnings-calendar…") - bulk_rows, bulk_reqs, err = await _try_bulk( - client, settings.fmp_api_key, start, end ) - reqs_used += bulk_reqs - if err: - print(f" Bulk unavailable: {err} (requests={bulk_reqs})") - else: - # Filter to universe. - uni = set(symbols) - bulk_rows = [r for r in bulk_rows if r["symbol"] in uni] - with engine.begin() as conn: - events_written += _upsert_events(conn, bulk_rows, "fmp_earnings_calendar") - for sym in symbols: - n = conn.execute( - text( - "SELECT COUNT(*) FROM earnings_events WHERE symbol=:s" - ), - {"s": sym}, - ).scalar_one() - conn.execute( - text( - """ - INSERT INTO earnings_backfill_meta(symbol, status, n_events, updated_at, note) - VALUES (:s, 'done', :n, :t, 'bulk') - ON CONFLICT(symbol) DO UPDATE SET - status='done', n_events=excluded.n_events, - updated_at=excluded.updated_at, note=excluded.note - """ - ), - { - "s": sym, - "n": int(n), - "t": datetime.now(timezone.utc).isoformat(), - }, - ) - mode = "bulk" - print(f" Bulk wrote {events_written} events; requests={bulk_reqs}") - pending = [] + } + pending = [ + window + for window in all_windows + if args.refetch_windows + or (window[0].isoformat(), window[1].isoformat()) not in completed + ] + universe = set(symbols) + print(f"Snapshot: {snapshot}") + print(f"Universe: {len(symbols)} symbols") + print(f"Window: {start} -> {end}") + print( + f"Bulk windows: {len(all_windows)} total; " + f"{len(all_windows) - len(pending)} done; {len(pending)} pending" + ) + print("Provider: FMP bulk earnings-calendar only") - # Per-symbol fallback / completion. - fmp_limited = False - for sym in pending: - if reqs_used >= req_budget: - print(f"Request budget exhausted ({req_budget}). Resume later.") + requests_this_run = 0 + rows_upserted = 0 + duplicate_rows = 0 + restated_rows = 0 + stop_note: str | None = None + async with httpx.AsyncClient(timeout=60.0) as client: + for index, (window_start, window_end) in enumerate(pending, 1): + if requests_this_run >= int(args.limit): + stop_note = "request_budget_exhausted" break - items: list[dict] = [] - source = "fmp_earnings" - note = "per_symbol" try: - if use_fmp and not fmp_limited and args.provider != "alpha_vantage": - items = await _fetch_symbol(client, settings.fmp_api_key, sym) - source = "fmp_earnings" - note = "fmp_per_symbol" - # Empty list may mean soft-limit or no data — try AV if available. - if not items and use_av: - items = await _fetch_symbol_alpha_vantage( - client, settings.alpha_vantage_api_key, sym - ) - source = "alpha_vantage_earnings" - note = "av_after_fmp_empty" - reqs_used += 1 # count AV call separately below too - elif use_av: - items = await _fetch_symbol_alpha_vantage( - client, settings.alpha_vantage_api_key, sym - ) - source = "alpha_vantage_earnings" - note = "av_per_symbol" - else: - raise RuntimeError("no provider available") + raw_rows, status_code, error = await _fetch_bulk_window( + client, settings.fmp_api_key, window_start, window_end + ) except Exception as exc: - msg = str(exc) - print(f" FAIL {sym}: {msg}") - reqs_used += 1 - if "rate_limited" in msg and note.startswith("fmp"): - fmp_limited = True - with engine.begin() as conn: + raw_rows, status_code = [], 0 + error = f"request_error:{type(exc).__name__}:{exc}" + requests_this_run += 1 + if error: + _write_window_status( + engine, + start=window_start, + end=window_end, + status="error", + note=f"http={status_code} {error}"[:300], + ) + stop_note = error + print( + f"STOP {window_start}..{window_end}: {error} " + f"(http={status_code}, request={requests_this_run})" + ) + break + + in_universe = [row for row in raw_rows if row["symbol"] in universe] + deduped, duplicate_n, restated_n = _dedupe_bulk_rows(in_universe) + with engine.begin() as conn: + rows_upserted += _upsert_events(conn, deduped) + _write_window_status( + engine, + start=window_start, + end=window_end, + status="done", + raw_n=len(raw_rows), + universe_n=len(deduped), + duplicate_n=duplicate_n, + restated_n=restated_n, + note="bulk", + ) + duplicate_rows += duplicate_n + restated_rows += restated_n + if index == 1 or index % 10 == 0 or index == len(pending): + print( + f"progress windows={index}/{len(pending)} " + f"requests={requests_this_run}/{args.limit} " + f"last={window_start}..{window_end} rows={len(deduped)}" + ) + if args.sleep > 0: + await asyncio.sleep(float(args.sleep)) + + with engine.begin() as conn: + windows_done = int( + conn.execute( + text( + "SELECT COUNT(*) FROM earnings_backfill_windows " + "WHERE status='done' AND from_date >= :a AND to_date <= :b" + ), + {"a": start.isoformat(), "b": end.isoformat()}, + ).scalar_one() + ) + complete = windows_done >= len(all_windows) + if complete: + now = datetime.now(timezone.utc).isoformat() + for symbol in symbols: + count = int( conn.execute( text( - """ - INSERT INTO earnings_backfill_meta(symbol, status, n_events, updated_at, note) - VALUES (:s, 'error', 0, :t, :n) - ON CONFLICT(symbol) DO UPDATE SET - status='error', updated_at=excluded.updated_at, note=excluded.note - """ + "SELECT COUNT(*) FROM earnings_events " + "WHERE symbol=:symbol AND announce_date BETWEEN :a AND :b" ), - { - "s": sym, - "t": datetime.now(timezone.utc).isoformat(), - "n": msg[:200], - }, - ) - if args.sleep > 0: - await asyncio.sleep(args.sleep) - continue - - reqs_used += 1 - # Keep all rows with dates on/before end — SUE needs trailing history. - filtered = [ - r for r in items if r["announce_date"] <= end.isoformat() - ] - # Do NOT mark empty as done — leave pending for another provider/day. - status = "done" if filtered else "empty" - with engine.begin() as conn: - n_w = _upsert_events(conn, filtered, source) if filtered else 0 - events_written += n_w + {"symbol": symbol, "a": start.isoformat(), "b": end.isoformat()}, + ).scalar_one() + ) conn.execute( text( """ INSERT INTO earnings_backfill_meta(symbol, status, n_events, updated_at, note) - VALUES (:s, :st, :n, :t, :note) + VALUES (:symbol, 'done', :count, :now, 'bulk_complete') ON CONFLICT(symbol) DO UPDATE SET - status=excluded.status, n_events=excluded.n_events, - updated_at=excluded.updated_at, note=excluded.note + status='done', n_events=excluded.n_events, + updated_at=excluded.updated_at, note=excluded.note """ ), - { - "s": sym, - "st": status, - "n": len(filtered), - "t": datetime.now(timezone.utc).isoformat(), - "note": note, - }, + {"symbol": symbol, "count": count, "now": now}, ) - if reqs_used % 10 == 0 or reqs_used == 1: - print( - f" progress reqs={reqs_used}/{req_budget} last={sym} " - f"events_batch={len(filtered)} src={source}" - ) - # AV free tier is ~5/min or 25/day — be polite when using it. - sleep_s = float(args.sleep) - if source.startswith("alpha_vantage"): - sleep_s = max(sleep_s, 12.0) - if sleep_s > 0: - await asyncio.sleep(sleep_s) - - with engine.connect() as conn: + params = {"a": start.isoformat(), "b": end.isoformat()} total_events = int( - conn.execute(text("SELECT COUNT(*) FROM earnings_events")).scalar_one() + conn.execute( + text( + "SELECT COUNT(*) FROM earnings_events " + "WHERE symbol IN (SELECT symbol FROM tickers) " + "AND announce_date BETWEEN :a AND :b" + ), + params, + ).scalar_one() ) - done_n = int( + paired_events = int( + conn.execute( + text( + "SELECT COUNT(*) FROM earnings_events " + "WHERE symbol IN (SELECT symbol FROM tickers) " + "AND announce_date BETWEEN :a AND :b " + "AND eps_actual IS NOT NULL AND eps_estimate IS NOT NULL" + ), + params, + ).scalar_one() + ) + date_range = conn.execute( + text( + "SELECT MIN(announce_date), MAX(announce_date) FROM earnings_events " + "WHERE symbol IN (SELECT symbol FROM tickers) " + "AND announce_date BETWEEN :a AND :b" + ), + params, + ).fetchone() + done_symbols = int( conn.execute( text("SELECT COUNT(*) FROM earnings_backfill_meta WHERE status='done'") ).scalar_one() ) - d_range = conn.execute( - text("SELECT MIN(announce_date), MAX(announce_date) FROM earnings_events") + totals = conn.execute( + text( + "SELECT COALESCE(SUM(requests),0), COALESCE(SUM(duplicate_rows),0), " + "COALESCE(SUM(restated_rows),0) FROM earnings_backfill_windows " + "WHERE from_date >= :a AND to_date <= :b" + ), + params, ).fetchone() - with_actual = int( - conn.execute( - text( - "SELECT COUNT(*) FROM earnings_events " - "WHERE eps_actual IS NOT NULL AND eps_estimate IS NOT NULL" - ) - ).scalar_one() - ) summary = { - "mode": mode, - "fmp_requests": reqs_used, - "events_written_this_run": events_written, - "total_events": total_events, - "symbols_done": done_n, + "mode": "fmp_bulk_date_range_only", + "window": {"from": start.isoformat(), "to": end.isoformat()}, + "window_days": int(args.window_days), + "bulk_windows_total": len(all_windows), + "bulk_windows_done": windows_done, + "bulk_requests_this_run": requests_this_run, + "bulk_requests_logged_total": int(totals[0]), + "rows_upserted_this_run": rows_upserted, + "duplicate_rows_this_run": duplicate_rows, + "restated_rows_this_run": restated_rows, + "duplicate_rows_logged_total": int(totals[1]), + "restated_rows_logged_total": int(totals[2]), + "dedupe_policy": ( + "UNIQUE(symbol, announce_date); prefer more non-null fields, then " + "the provider's later occurrence; non-null bulk fields replace prior " + "values while null bulk fields retain existing values" + ), + "events_in_window": total_events, + "events_with_actual_and_estimate": paired_events, + "symbols_done": done_symbols, "symbols_universe": len(symbols), - "announce_date_range": {"min": d_range[0], "max": d_range[1]}, - "events_with_actual_and_estimate": with_actual, - "budget": req_budget, - "complete": done_n >= len(symbols), + "announce_date_range": {"min": date_range[0], "max": date_range[1]}, + "request_budget": int(args.limit), + "stop_note": stop_note, + "complete": complete, } + output = Path("reports/earnings-backfill-status.json") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") print(json.dumps(summary, indent=2)) - out = Path("reports") / "earnings-backfill-status.json" - out.parent.mkdir(parents=True, exist_ok=True) - out.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") - print(f"Wrote {out}") + print(f"Wrote {output}") if __name__ == "__main__": diff --git a/scripts/extend_snapshot_universe.py b/scripts/extend_snapshot_universe.py index 8f10b94..21a6ec8 100644 --- a/scripts/extend_snapshot_universe.py +++ b/scripts/extend_snapshot_universe.py @@ -97,6 +97,14 @@ def _parse_args() -> argparse.Namespace: default=5, help="Retries per symbol on RateLimitError.", ) + p.add_argument( + "--source-symbols-only", + action="store_true", + help=( + "Refresh only symbols present in --source. Useful for repairing " + "per-symbol depth without re-fetching the broad rank-only pool." + ), + ) p.add_argument("--quiet", action="store_true") return p.parse_args() @@ -218,6 +226,15 @@ async def _main() -> None: if not source.exists(): raise SystemExit(f"Source snapshot not found: {source}") + source_engine = create_engine( + f"sqlite:///{source.resolve().as_posix()}", future=True + ) + with source_engine.connect() as conn: + source_symbols = { + str(row[0]) for row in conn.execute(text("SELECT symbol FROM tickers")) + } + source_engine.dispose() + # Any rebuild/update invalidates prior completion until we finish cleanly. clear_manifest(output) @@ -241,7 +258,12 @@ async def _main() -> None: start = end - timedelta(days=int(args.history_days)) print("Resolving universe pool (nasdaq_all ∪ sp500)…") - pool, sources = await _resolve_pool() + if args.source_symbols_only: + pool = sorted(source_symbols) + sources = {"pool": "source_snapshot"} + print(" source snapshot: symbol pool selected") + else: + pool, sources = await _resolve_pool() print(f"Pool size: {len(pool)} (sources={sources})") # Sync sqlite via raw SQL — one short transaction per symbol so a failed @@ -257,7 +279,7 @@ async def _main() -> None: text("SELECT id, symbol FROM tickers") ).fetchall() existing_ids = {str(sym): int(tid) for tid, sym in existing_rows} - prod_symbols = set(existing_ids) + prod_symbols = set(source_symbols) bar_counts: dict[str, int] = {} for sym, tid in existing_ids.items(): @@ -363,7 +385,7 @@ async def _main() -> None: for b in bars ], ) - if is_new: + if is_new and sym not in prod_symbols: write.execute( text( "INSERT OR REPLACE INTO research_rank_only " @@ -385,6 +407,39 @@ async def _main() -> None: f"elapsed={elapsed/60:.1f}m last={sym} bars={len(bars)}" ) + benchmark_rows = 0 + try: + benchmark_bars = await _fetch_symbol_bars( + provider, + "SPY", + start, + end, + max_retries=args.max_retries, + sleep_s=args.sleep, + ) + with engine.begin() as write: + write.execute( + text( + "DELETE FROM benchmark_prices WHERE symbol='SPY' " + "AND date >= :start AND date <= :end" + ), + {"start": start.isoformat(), "end": end.isoformat()}, + ) + if benchmark_bars: + write.execute( + text( + "INSERT INTO benchmark_prices(symbol, date, close) " + "VALUES ('SPY', :date, :close)" + ), + [ + {"date": bar.date.isoformat(), "close": float(bar.close)} + for bar in benchmark_bars + ], + ) + benchmark_rows = len(benchmark_bars) + except Exception as exc: + print(f" benchmark SPY refresh FAIL {exc}") + rank_only_n = conn.execute( text("SELECT COUNT(*) FROM research_rank_only") ).scalar_one() @@ -409,6 +464,8 @@ async def _main() -> None: "prod_symbols_at_start": len(prod_symbols), "pool_size": len(pool), "to_fetch": len(to_fetch), + "source_symbols_only": bool(args.source_symbols_only), + "benchmark_spy_rows": benchmark_rows, }, ) diff --git a/scripts/import_dolthub_earnings.py b/scripts/import_dolthub_earnings.py new file mode 100644 index 0000000..a2d4cf4 --- /dev/null +++ b/scripts/import_dolthub_earnings.py @@ -0,0 +1,657 @@ +"""Import the public post-no-preference/earnings DoltHub database. + +The earnings calendar and EPS history are separate tables in the source. This +importer aligns them monotonically per symbol, keeps every calendar event for +the defensive gap study, and stores the longer EPS history separately for SUE +scaling. EPS history without an announcement date is never exposed as a live +signal event. +""" + +from __future__ import annotations + +import argparse +import csv +import json +import math +import sqlite3 +from collections import defaultdict +from datetime import date, datetime, timedelta, timezone +from pathlib import Path +from typing import Any + + +EVENTS_DDL = """ +CREATE TABLE IF NOT EXISTS earnings_events ( + id INTEGER PRIMARY KEY, + symbol TEXT NOT NULL, + announce_date TEXT NOT NULL, + announce_time TEXT, + eps_estimate REAL, + eps_actual REAL, + revenue_estimate REAL, + revenue_actual REAL, + source TEXT NOT NULL, + fetched_at TEXT NOT NULL, + period_end_date TEXT, + UNIQUE(symbol, announce_date) +) +""" +META_DDL = """ +CREATE TABLE IF NOT EXISTS earnings_backfill_meta ( + symbol TEXT PRIMARY KEY, + status TEXT NOT NULL, + n_events INTEGER NOT NULL DEFAULT 0, + updated_at TEXT NOT NULL, + note TEXT +) +""" +SURPRISE_HISTORY_DDL = """ +CREATE TABLE IF NOT EXISTS earnings_surprise_history ( + symbol TEXT NOT NULL, + period_end_date TEXT NOT NULL, + eps_estimate REAL, + eps_actual REAL, + source TEXT NOT NULL, + fetched_at TEXT NOT NULL, + PRIMARY KEY(symbol, period_end_date) +) +""" + +SKIP_EVENT_COST = 45.0 +SKIP_PERIOD_COST = 45.0 + + +def _parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--snapshot", default="backtest_snapshots/prod.sqlite") + parser.add_argument("--calendar-csv", required=True) + parser.add_argument("--history-csv", required=True) + parser.add_argument("--from-date", default="2020-01-22") + parser.add_argument("--to-date", required=True) + parser.add_argument("--source-commit", required=True) + parser.add_argument( + "--source-url", + default="https://www.dolthub.com/repositories/post-no-preference/earnings", + ) + parser.add_argument("--max-period-lag-days", type=int, default=90) + parser.add_argument("--max-period-lead-days", type=int, default=14) + parser.add_argument( + "--status-output", default="reports/earnings-backfill-status.json" + ) + return parser.parse_args() + + +def _normalise_symbol(value: Any) -> str: + return str(value or "").strip().upper().replace(".", "-") + + +def _normalise_session(value: Any) -> str | None: + cleaned = str(value or "").strip().lower().replace("_", " ").replace("-", " ") + aliases = { + "before market open": "bmo", + "before open": "bmo", + "bmo": "bmo", + "after market close": "amc", + "after close": "amc", + "amc": "amc", + "during market hours": "during", + "dmh": "during", + } + return aliases.get(cleaned, cleaned or None) + + +def _number(value: Any) -> float | None: + if value is None or str(value).strip() == "": + return None + try: + result = float(value) + except (TypeError, ValueError): + return None + return result if math.isfinite(result) else None + + +def _read_calendar( + path: Path, + universe: set[str], + start: date, + end: date, +) -> tuple[dict[str, list[dict[str, Any]]], dict[str, int]]: + by_key: dict[tuple[str, date], dict[str, Any]] = {} + raw_rows = 0 + universe_rows = 0 + duplicate_rows = 0 + restated_rows = 0 + with path.open(newline="", encoding="utf-8-sig") as handle: + for raw in csv.DictReader(handle): + raw_rows += 1 + symbol = _normalise_symbol(raw.get("act_symbol")) + raw_date = str(raw.get("date") or "")[:10] + if symbol not in universe or not raw_date: + continue + event_date = date.fromisoformat(raw_date) + if not start <= event_date <= end: + continue + universe_rows += 1 + row = { + "symbol": symbol, + "announce_date": event_date, + "announce_time": _normalise_session(raw.get("when")), + } + key = (symbol, event_date) + previous = by_key.get(key) + if previous is not None: + duplicate_rows += 1 + if ( + previous.get("announce_time") is not None + and row.get("announce_time") is not None + and previous["announce_time"] != row["announce_time"] + ): + restated_rows += 1 + if row.get("announce_time") is not None: + by_key[key] = row + else: + by_key[key] = row + grouped: dict[str, list[dict[str, Any]]] = defaultdict(list) + for row in by_key.values(): + grouped[row["symbol"]].append(row) + for rows in grouped.values(): + rows.sort(key=lambda item: item["announce_date"]) + return grouped, { + "raw_rows": raw_rows, + "universe_rows_in_window": universe_rows, + "deduped_rows_in_window": len(by_key), + "duplicate_rows": duplicate_rows, + "restated_rows": restated_rows, + } + + +def _read_history( + path: Path, universe: set[str] +) -> tuple[dict[str, list[dict[str, Any]]], dict[str, int]]: + by_key: dict[tuple[str, date], dict[str, Any]] = {} + raw_rows = 0 + universe_rows = 0 + duplicate_rows = 0 + restated_rows = 0 + fields = ("eps_actual", "eps_estimate") + with path.open(newline="", encoding="utf-8-sig") as handle: + for raw in csv.DictReader(handle): + raw_rows += 1 + symbol = _normalise_symbol(raw.get("act_symbol")) + raw_date = str(raw.get("period_end_date") or "")[:10] + if symbol not in universe or not raw_date: + continue + universe_rows += 1 + period_end = date.fromisoformat(raw_date) + row = { + "symbol": symbol, + "period_end_date": period_end, + "eps_actual": _number(raw.get("reported")), + "eps_estimate": _number(raw.get("estimate")), + } + key = (symbol, period_end) + previous = by_key.get(key) + if previous is not None: + duplicate_rows += 1 + if any( + previous.get(field) is not None + and row.get(field) is not None + and previous[field] != row[field] + for field in fields + ): + restated_rows += 1 + previous_score = sum(previous.get(field) is not None for field in fields) + row_score = sum(row.get(field) is not None for field in fields) + if row_score >= previous_score: + by_key[key] = row + else: + by_key[key] = row + grouped: dict[str, list[dict[str, Any]]] = defaultdict(list) + for row in by_key.values(): + grouped[row["symbol"]].append(row) + for rows in grouped.values(): + rows.sort(key=lambda item: item["period_end_date"]) + return grouped, { + "raw_rows": raw_rows, + "universe_rows": universe_rows, + "deduped_rows": len(by_key), + "duplicate_rows": duplicate_rows, + "restated_rows": restated_rows, + } + + +def _match_cost(event: dict[str, Any], period: dict[str, Any]) -> float: + delta = (event["announce_date"] - period["period_end_date"]).days + missing_session_penalty = 3.0 if event.get("announce_time") is None else 0.0 + return float(abs(delta - 30)) + missing_session_penalty + + +def _align_symbol( + events: list[dict[str, Any]], + periods: list[dict[str, Any]], + *, + max_lag_days: int, + max_lead_days: int, +) -> tuple[list[tuple[int, int]], list[int], list[int]]: + """Return a minimum-cost monotonic calendar-to-period alignment.""" + n_events = len(events) + n_periods = len(periods) + scores = [[0.0] * (n_periods + 1) for _ in range(n_events + 1)] + choices = [[""] * (n_periods + 1) for _ in range(n_events + 1)] + for event_index in range(n_events - 1, -1, -1): + scores[event_index][n_periods] = ( + scores[event_index + 1][n_periods] + SKIP_EVENT_COST + ) + choices[event_index][n_periods] = "event" + for period_index in range(n_periods - 1, -1, -1): + scores[n_events][period_index] = ( + scores[n_events][period_index + 1] + SKIP_PERIOD_COST + ) + choices[n_events][period_index] = "period" + + for event_index in range(n_events - 1, -1, -1): + for period_index in range(n_periods - 1, -1, -1): + options = [ + ( + scores[event_index + 1][period_index] + SKIP_EVENT_COST, + 2, + "event", + ), + ( + scores[event_index][period_index + 1] + SKIP_PERIOD_COST, + 1, + "period", + ), + ] + delta = ( + events[event_index]["announce_date"] + - periods[period_index]["period_end_date"] + ).days + if -max_lead_days <= delta <= max_lag_days: + options.append( + ( + scores[event_index + 1][period_index + 1] + + _match_cost(events[event_index], periods[period_index]), + 0, + "match", + ) + ) + score, _, choice = min(options) + scores[event_index][period_index] = score + choices[event_index][period_index] = choice + + matches: list[tuple[int, int]] = [] + unmatched_events: list[int] = [] + unmatched_periods: list[int] = [] + event_index = 0 + period_index = 0 + while event_index < n_events or period_index < n_periods: + if event_index >= n_events: + unmatched_periods.extend(range(period_index, n_periods)) + break + if period_index >= n_periods: + unmatched_events.extend(range(event_index, n_events)) + break + choice = choices[event_index][period_index] + if choice == "match": + matches.append((event_index, period_index)) + event_index += 1 + period_index += 1 + elif choice == "period": + unmatched_periods.append(period_index) + period_index += 1 + else: + unmatched_events.append(event_index) + event_index += 1 + return matches, unmatched_events, unmatched_periods + + +def _ensure_schema(connection: sqlite3.Connection) -> None: + connection.execute(EVENTS_DDL) + columns = { + str(row[1]) + for row in connection.execute("PRAGMA table_info(earnings_events)") + } + if "period_end_date" not in columns: + connection.execute("ALTER TABLE earnings_events ADD COLUMN period_end_date TEXT") + connection.execute(META_DDL) + connection.execute(SURPRISE_HISTORY_DDL) + + +def _main() -> None: + args = _parse_args() + snapshot = Path(args.snapshot) + calendar_csv = Path(args.calendar_csv) + history_csv = Path(args.history_csv) + for path in (snapshot, calendar_csv, history_csv): + if not path.exists(): + raise SystemExit(f"Missing input: {path}") + start = date.fromisoformat(args.from_date) + end = date.fromisoformat(args.to_date) + if start > end: + raise SystemExit("--from-date must not be after --to-date") + + connection = sqlite3.connect(snapshot) + try: + universe = { + _normalise_symbol(row[0]) + for row in connection.execute("SELECT symbol FROM tickers") + } + finally: + connection.close() + calendar, calendar_stats = _read_calendar(calendar_csv, universe, start, end) + history, history_stats = _read_history(history_csv, universe) + + aligned_events: list[dict[str, Any]] = [] + pairing_deltas: list[int] = [] + unmatched_calendar = 0 + unmatched_periods_in_pairing_window = 0 + matched = 0 + for symbol in sorted(universe): + events = calendar.get(symbol, []) + lower = start - timedelta(days=int(args.max_period_lag_days)) + upper = end + timedelta(days=int(args.max_period_lead_days)) + periods = [ + row + for row in history.get(symbol, []) + if lower <= row["period_end_date"] <= upper + ] + matches, unmatched_events, unmatched_periods = _align_symbol( + events, + periods, + max_lag_days=int(args.max_period_lag_days), + max_lead_days=int(args.max_period_lead_days), + ) + matched_by_event = {event_index: period_index for event_index, period_index in matches} + matched += len(matches) + unmatched_calendar += len(unmatched_events) + unmatched_periods_in_pairing_window += len(unmatched_periods) + for event_index, event in enumerate(events): + row = dict(event) + period_index = matched_by_event.get(event_index) + if period_index is None: + row.update( + { + "period_end_date": None, + "eps_actual": None, + "eps_estimate": None, + } + ) + else: + period = periods[period_index] + row.update( + { + "period_end_date": period["period_end_date"], + "eps_actual": period["eps_actual"], + "eps_estimate": period["eps_estimate"], + } + ) + pairing_deltas.append( + (event["announce_date"] - period["period_end_date"]).days + ) + aligned_events.append(row) + + now = datetime.now(timezone.utc).isoformat() + source = f"dolthub_post_no_preference@{args.source_commit}" + conflicting_existing_rows = 0 + conflicting_existing_fields = 0 + preserved_existing_fields = 0 + incoming_keys = { + (row["symbol"], row["announce_date"].isoformat()) for row in aligned_events + } + connection = sqlite3.connect(snapshot) + try: + _ensure_schema(connection) + existing = { + (str(row[0]), str(row[1])): row + for row in connection.execute( + """ + SELECT symbol, announce_date, announce_time, eps_estimate, + eps_actual, period_end_date, source + FROM earnings_events + WHERE announce_date BETWEEN ? AND ? + """, + (start.isoformat(), end.isoformat()), + ) + } + upsert = """ + INSERT INTO earnings_events( + symbol, announce_date, announce_time, eps_estimate, eps_actual, + revenue_estimate, revenue_actual, source, fetched_at, period_end_date + ) VALUES (?, ?, ?, ?, ?, NULL, NULL, ?, ?, ?) + ON CONFLICT(symbol, announce_date) DO UPDATE SET + announce_time=COALESCE(earnings_events.announce_time, excluded.announce_time), + eps_estimate=COALESCE(earnings_events.eps_estimate, excluded.eps_estimate), + eps_actual=COALESCE(earnings_events.eps_actual, excluded.eps_actual), + period_end_date=COALESCE(excluded.period_end_date, earnings_events.period_end_date), + source=excluded.source, + fetched_at=excluded.fetched_at + """ + for row in aligned_events: + key = (row["symbol"], row["announce_date"].isoformat()) + old = existing.get(key) + retained = 0 + conflicts = 0 + if old is not None: + old_values = { + "announce_time": old[2], + "eps_estimate": old[3], + "eps_actual": old[4], + "period_end_date": old[5], + } + new_values = { + "announce_time": row.get("announce_time"), + "eps_estimate": row.get("eps_estimate"), + "eps_actual": row.get("eps_actual"), + "period_end_date": ( + row["period_end_date"].isoformat() + if row.get("period_end_date") + else None + ), + } + for field, new_value in new_values.items(): + old_value = old_values[field] + if field != "period_end_date" and old_value is not None: + retained += 1 + if new_value is not None and old_value is not None: + if field in {"eps_estimate", "eps_actual"}: + differs = not math.isclose( + float(new_value), float(old_value), rel_tol=0.0, abs_tol=1e-9 + ) + else: + differs = str(new_value) != str(old_value) + conflicts += int(differs) + conflicting_existing_rows += int(conflicts > 0) + conflicting_existing_fields += conflicts + preserved_existing_fields += retained + row_source = source + if retained and old is not None: + row_source = f"{old[6]}+calendar:{source}" + connection.execute( + upsert, + ( + row["symbol"], + row["announce_date"].isoformat(), + row.get("announce_time"), + row.get("eps_estimate"), + row.get("eps_actual"), + row_source, + now, + ( + row["period_end_date"].isoformat() + if row.get("period_end_date") + else None + ), + ), + ) + + history_upsert = """ + INSERT INTO earnings_surprise_history( + symbol, period_end_date, eps_estimate, eps_actual, source, fetched_at + ) VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT(symbol, period_end_date) DO UPDATE SET + eps_estimate=COALESCE(excluded.eps_estimate, earnings_surprise_history.eps_estimate), + eps_actual=COALESCE(excluded.eps_actual, earnings_surprise_history.eps_actual), + source=excluded.source, + fetched_at=excluded.fetched_at + """ + for symbol, rows in history.items(): + connection.executemany( + history_upsert, + [ + ( + symbol, + row["period_end_date"].isoformat(), + row.get("eps_estimate"), + row.get("eps_actual"), + source, + now, + ) + for row in rows + ], + ) + + for symbol in sorted(universe): + count = int( + connection.execute( + """ + SELECT COUNT(*) FROM earnings_events + WHERE symbol=? AND announce_date BETWEEN ? AND ? + """, + (symbol, start.isoformat(), end.isoformat()), + ).fetchone()[0] + ) + connection.execute( + """ + INSERT INTO earnings_backfill_meta(symbol, status, n_events, updated_at, note) + VALUES (?, 'done', ?, ?, 'dolthub_bulk_complete') + ON CONFLICT(symbol) DO UPDATE SET + status='done', n_events=excluded.n_events, + updated_at=excluded.updated_at, note=excluded.note + """, + (symbol, count, now), + ) + connection.commit() + + params = (start.isoformat(), end.isoformat()) + total_events = int( + connection.execute( + """ + SELECT COUNT(*) FROM earnings_events + WHERE symbol IN (SELECT symbol FROM tickers) + AND announce_date BETWEEN ? AND ? + """, + params, + ).fetchone()[0] + ) + paired_events = int( + connection.execute( + """ + SELECT COUNT(*) FROM earnings_events + WHERE symbol IN (SELECT symbol FROM tickers) + AND announce_date BETWEEN ? AND ? + AND eps_actual IS NOT NULL AND eps_estimate IS NOT NULL + """, + params, + ).fetchone()[0] + ) + date_range = connection.execute( + """ + SELECT MIN(announce_date), MAX(announce_date) FROM earnings_events + WHERE symbol IN (SELECT symbol FROM tickers) + AND announce_date BETWEEN ? AND ? + """, + params, + ).fetchone() + source_symbols = set(calendar) + history_complete = int( + connection.execute( + """ + SELECT COUNT(*) FROM earnings_surprise_history + WHERE symbol IN (SELECT symbol FROM tickers) + AND eps_actual IS NOT NULL AND eps_estimate IS NOT NULL + """ + ).fetchone()[0] + ) + finally: + connection.close() + + deltas = sorted(pairing_deltas) + summary = { + "mode": "dolthub_public_bulk_clone", + "window": {"from": start.isoformat(), "to": end.isoformat()}, + "coverage_amendment": { + "approved_by_user": True, + "reason": "FMP free tier blocks historical bulk earnings", + "original_start": "2016-01-04", + "amended_announcement_start": start.isoformat(), + }, + "source": { + "repository": args.source_url, + "commit": args.source_commit, + "license": "CC-BY-SA-4.0", + "upstream_provider_documented": False, + }, + "bulk_windows_total": 1, + "bulk_windows_done": 1, + "bulk_requests_logged_total": 1, + "bulk_exports": 2, + "calendar": calendar_stats, + "eps_history": {**history_stats, "complete_actual_and_estimate": history_complete}, + "pairing": { + "method": "minimum-cost monotonic alignment per symbol", + "allowed_announce_minus_period_end_days": [ + -int(args.max_period_lead_days), + int(args.max_period_lag_days), + ], + "matched_calendar_events": matched, + "unmatched_calendar_events": unmatched_calendar, + "unmatched_periods_in_pairing_window": unmatched_periods_in_pairing_window, + "announce_minus_period_end_days": { + "min": min(deltas) if deltas else None, + "median": deltas[len(deltas) // 2] if deltas else None, + "max": max(deltas) if deltas else None, + }, + "pre_2020_eps_history_use": ( + "trailing_surprise_stdev_only; never treated as an announcement " + "or live signal event" + ), + }, + "duplicate_rows_logged_total": ( + calendar_stats["duplicate_rows"] + history_stats["duplicate_rows"] + ), + "restated_rows_logged_total": ( + calendar_stats["restated_rows"] + + history_stats["restated_rows"] + + conflicting_existing_rows + ), + "conflicting_existing_rows": conflicting_existing_rows, + "conflicting_existing_fields": conflicting_existing_fields, + "preserved_existing_fields": preserved_existing_fields, + "existing_enrichment_events_not_in_dolthub_calendar": max( + 0, total_events - len(incoming_keys) + ), + "dedupe_policy": ( + "UNIQUE(symbol, announce_date); normalise dot/dash symbols; retain one " + "calendar row per key; preserve existing non-null session/EPS values from " + "the prior FMP/Alpha Vantage partial backfill, then fill nulls and all " + "remaining symbols from DoltHub; attach DoltHub period-end alignment" + ), + "events_in_window": total_events, + "events_with_actual_and_estimate": paired_events, + "symbols_done": len(universe), + "symbols_universe": len(universe), + "symbols_with_dolthub_calendar": len(source_symbols), + "symbols_without_dolthub_calendar": sorted(universe - source_symbols), + "announce_date_range": {"min": date_range[0], "max": date_range[1]}, + "complete": True, + } + output = Path(args.status_output) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") + print(json.dumps(summary, indent=2)) + print(f"Wrote {output}") + + +if __name__ == "__main__": + _main() diff --git a/scripts/run_earnings_research.py b/scripts/run_earnings_research.py index bfc353e..1f71620 100644 --- a/scripts/run_earnings_research.py +++ b/scripts/run_earnings_research.py @@ -1,27 +1,29 @@ -"""Earnings gap diagnostic (2a) + SUE IC (2b). Local research only. +"""Run Task 2 earnings-gap (2a) and SUE/PEAD (2b) research. -Requires ``earnings_events`` on the snapshot (see backfill_earnings_events.py). - -Example -------- - python scripts/run_earnings_research.py \\ - --snapshot backtest_snapshots/prod.sqlite --workers 6 --allow-spawn +Both experiments use the manifest-guarded research snapshot restricted to the +production symbol set. Earnings data remain in the real earnings_events table +on the production snapshot. This runner is research-only and never changes +production configuration or integrates a signal/filter. """ from __future__ import annotations import argparse import asyncio +import bisect import json import math import os +import sqlite3 import sys from collections import defaultdict -from datetime import date, datetime, timedelta +from concurrent.futures import ProcessPoolExecutor +from datetime import date, datetime, timezone from pathlib import Path +from types import SimpleNamespace from typing import Any -from sqlalchemy import create_engine, text +from sqlalchemy import create_engine, select, text from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine ROOT = Path(__file__).resolve().parents[1] @@ -33,290 +35,654 @@ from app.ssl_bootstrap import bootstrap_ssl # noqa: E402 bootstrap_ssl() IRON_IC_BAR = 0.03 -MIN_RELIABLE = 12 SUE_CARRY_DAYS = 63 SUE_TRAIL = 8 +SUE_MIN_TRAIL = 4 +COST_PER_SIDE = 0.001 +MIN_PRE2021_DEPTH_PCT = 80.0 + + +def _parse_args() -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--snapshot", default="backtest_snapshots/research.sqlite") + parser.add_argument( + "--universe-snapshot", default="backtest_snapshots/prod.sqlite" + ) + parser.add_argument( + "--earnings-snapshot", default="backtest_snapshots/prod.sqlite" + ) + parser.add_argument("--workers", type=int, default=6) + parser.add_argument("--allow-spawn", action="store_true") + parser.add_argument("--quiet", action="store_true") + parser.add_argument("--stamp", default=None) + return parser.parse_args() def _sqlite_url(path: Path) -> str: return f"sqlite+aiosqlite:///{path.resolve().as_posix()}" -def _parse_args() -> argparse.Namespace: - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("--snapshot", default="backtest_snapshots/prod.sqlite") - p.add_argument("--workers", type=int, default=6) - p.add_argument("--allow-spawn", action="store_true") - p.add_argument("--skip-2a", action="store_true") - p.add_argument("--skip-2b", action="store_true") - p.add_argument("--quiet", action="store_true") - p.add_argument("--out", default=None) - return p.parse_args() - - -def _load_earnings(snapshot: Path) -> list[dict]: - engine = create_engine( - f"sqlite:///{snapshot.resolve().as_posix()}", - future=True, - ) +def _read_symbols(snapshot: Path) -> list[str]: + engine = create_engine(f"sqlite:///{snapshot.resolve().as_posix()}", future=True) + try: + with engine.connect() as conn: + return [ + str(row[0]).upper().replace(".", "-") + for row in conn.execute(text("SELECT symbol FROM tickers ORDER BY symbol")) + ] + finally: + engine.dispose() + + +def _snapshot_depth(snapshot: Path, symbols: set[str]) -> dict[str, Any]: + from scripts.research_snapshot_manifest import assert_research_snapshot_complete + + manifest = assert_research_snapshot_complete(snapshot) + connection = sqlite3.connect(snapshot) + try: + rows = connection.execute( + """ + SELECT t.symbol, MIN(o.date), MAX(o.date), COUNT(o.id) + FROM tickers t + LEFT JOIN ohlcv_records o ON o.ticker_id=t.id + GROUP BY t.id, t.symbol + """ + ).fetchall() + benchmark = connection.execute( + "SELECT COUNT(*), MIN(date), MAX(date) FROM benchmark_prices " + "WHERE symbol='SPY'" + ).fetchone() + finally: + connection.close() + + by_symbol = {str(row[0]).upper(): row for row in rows} + selected = [by_symbol[symbol] for symbol in sorted(symbols) if symbol in by_symbol] + missing = sorted(symbols - set(by_symbol)) + usable = [row for row in selected if int(row[3] or 0) > 0] + zero_bar = sorted(str(row[0]) for row in selected if int(row[3] or 0) == 0) + counts = sorted(int(row[3]) for row in usable) + pre2021 = [row for row in usable if row[1] and str(row[1]) < "2021-01-01"] + pre_pct = round(len(pre2021) / max(1, len(usable)) * 100.0, 1) + shallow = [ + { + "symbol": str(row[0]), + "first_bar": row[1], + "last_bar": row[2], + "bars": int(row[3] or 0), + } + for row in sorted(usable, key=lambda item: int(item[3] or 0)) + if int(row[3] or 0) < 1000 + ] + gate_pass = ( + len(usable) >= 505 + and len(missing) + len(zero_bar) <= 1 + and pre_pct >= MIN_PRE2021_DEPTH_PCT + and int(benchmark[0] or 0) >= 1000 + and benchmark[1] is not None + and str(benchmark[1]) < "2021-01-01" + ) + return { + "manifest": manifest, + "requested_symbols": len(symbols), + "tradable_symbols": len(usable), + "missing_symbols": missing, + "zero_bar_symbols": zero_bar, + "bar_count": { + "min": min(counts) if counts else 0, + "median": counts[len(counts) // 2] if counts else 0, + "max": max(counts) if counts else 0, + }, + "symbols_with_pre2021_bars": len(pre2021), + "symbols_with_pre2021_bars_pct": pre_pct, + "shallow_symbols_lt_1000_bars": shallow, + "price_window": { + "min": min(str(row[1]) for row in usable if row[1]), + "max": max(str(row[2]) for row in usable if row[2]), + }, + "benchmark_spy": { + "rows": int(benchmark[0] or 0), + "min": benchmark[1], + "max": benchmark[2], + }, + "gate_threshold": { + "min_tradable_symbols": 505, + "max_missing_or_zero_bar": 1, + "min_symbols_with_pre2021_bars_pct": MIN_PRE2021_DEPTH_PCT, + "benchmark_min_rows": 1000, + "benchmark_must_begin_pre2021": True, + }, + "gate_pass": gate_pass, + } + + +def _load_backfill_status() -> dict[str, Any]: + path = Path("reports/earnings-backfill-status.json") + if not path.exists(): + raise SystemExit(f"Missing backfill status: {path}") + return json.loads(path.read_text(encoding="utf-8")) + + +def _load_earnings(snapshot: Path, symbols: set[str]) -> list[dict[str, Any]]: + engine = create_engine(f"sqlite:///{snapshot.resolve().as_posix()}", future=True) try: with engine.connect() as conn: - # Table must exist. tables = { - r[0] - for r in conn.execute( + str(row[0]) + for row in conn.execute( text("SELECT name FROM sqlite_master WHERE type='table'") ) } if "earnings_events" not in tables: - raise SystemExit( - "earnings_events table missing — run scripts/backfill_earnings_events.py" - ) + raise SystemExit("earnings_events table missing") + event_columns = { + str(row[1]) + for row in conn.execute(text("PRAGMA table_info(earnings_events)")) + } + period_expression = ( + "period_end_date" if "period_end_date" in event_columns else "NULL" + ) rows = conn.execute( text( - """ - SELECT symbol, announce_date, announce_time, - eps_estimate, eps_actual, revenue_estimate, revenue_actual + f""" + SELECT symbol, announce_date, announce_time, eps_estimate, + eps_actual, revenue_estimate, revenue_actual, source, + {period_expression} AS period_end_date FROM earnings_events ORDER BY symbol, announce_date """ ) ).fetchall() - meta = {} - if "earnings_backfill_meta" in tables: - meta = { - "done": int( - conn.execute( - text( - "SELECT COUNT(*) FROM earnings_backfill_meta " - "WHERE status='done'" - ) - ).scalar_one() - ), - "universe_tickers": int( - conn.execute(text("SELECT COUNT(*) FROM tickers")).scalar_one() - ), - } finally: engine.dispose() + result = [] + for row in rows: + symbol = str(row[0]).upper().replace(".", "-") + if symbol not in symbols: + continue + result.append( + { + "symbol": symbol, + "announce_date": date.fromisoformat(str(row[1])[:10]), + "announce_time": str(row[2]).lower() if row[2] else None, + "eps_estimate": row[3], + "eps_actual": row[4], + "revenue_estimate": row[5], + "revenue_actual": row[6], + "source": row[7], + "period_end_date": ( + date.fromisoformat(str(row[8])[:10]) if row[8] else None + ), + } + ) + return result - events = [ - { - "symbol": str(r[0]).upper(), - "announce_date": date.fromisoformat(str(r[1])[:10]), - "announce_time": r[2], - "eps_estimate": r[3], - "eps_actual": r[4], - "revenue_estimate": r[5], - "revenue_actual": r[6], - } - for r in rows + +def _load_surprise_history( + snapshot: Path, symbols: set[str] +) -> dict[str, list[dict[str, Any]]]: + engine = create_engine(f"sqlite:///{snapshot.resolve().as_posix()}", future=True) + try: + with engine.connect() as conn: + tables = { + str(row[0]) + for row in conn.execute( + text("SELECT name FROM sqlite_master WHERE type='table'") + ) + } + if "earnings_surprise_history" not in tables: + return {} + rows = conn.execute( + text( + """ + SELECT symbol, period_end_date, eps_estimate, eps_actual + FROM earnings_surprise_history + ORDER BY symbol, period_end_date + """ + ) + ).fetchall() + finally: + engine.dispose() + result: dict[str, list[dict[str, Any]]] = defaultdict(list) + for row in rows: + symbol = str(row[0]).upper().replace(".", "-") + if symbol not in symbols: + continue + result[symbol].append( + { + "period_end_date": date.fromisoformat(str(row[1])[:10]), + "eps_estimate": row[2], + "eps_actual": row[3], + } + ) + return dict(result) + + +def _pct(numerator: int, denominator: int) -> float: + return round(numerator / max(1, denominator) * 100.0, 1) + + +def _data_quality( + events: list[dict[str, Any]], + symbols: set[str], + *, + window_start: date, + window_end: date, + backfill_status: dict[str, Any], +) -> dict[str, Any]: + in_window = [ + event + for event in events + if window_start <= event["announce_date"] <= window_end ] - return events, meta + by_symbol: dict[str, list[dict[str, Any]]] = defaultdict(list) + for event in in_window: + by_symbol[event["symbol"]].append(event) + paired = [ + event + for event in in_window + if event.get("eps_estimate") is not None + and event.get("eps_actual") is not None + ] + paired_by_symbol: dict[str, int] = defaultdict(int) + for event in paired: + paired_by_symbol[event["symbol"]] += 1 + ge8 = sum(len(by_symbol.get(symbol, [])) >= 8 for symbol in symbols) + ge8_paired = sum(paired_by_symbol.get(symbol, 0) >= 8 for symbol in symbols) + recognised_sessions = {"bmo", "amc", "during"} + session_known = sum( + str(event.get("announce_time") or "").lower() in recognised_sessions + for event in in_window + ) + session_pct = _pct(session_known, len(in_window)) + session_reliable = session_pct >= 80.0 -def _percentile(xs: list[float], q: float) -> float | None: - if not xs: - return None - s = sorted(xs) - if len(s) == 1: - return s[0] - idx = q * (len(s) - 1) - lo = int(math.floor(idx)) - hi = int(math.ceil(idx)) - if lo == hi: - return s[lo] - w = idx - lo - return s[lo] * (1 - w) + s[hi] * w + far_off = [] + annual_rates: list[float] = [] + for symbol in sorted(symbols): + symbol_events = sorted( + by_symbol.get(symbol, []), key=lambda event: event["announce_date"] + ) + if not symbol_events: + far_off.append( + {"symbol": symbol, "events": 0, "events_per_year": 0.0} + ) + continue + first = symbol_events[0]["announce_date"] + last = symbol_events[-1]["announce_date"] + active_years = max(1.0, (last - first).days / 365.25) + rate = len(symbol_events) / active_years + annual_rates.append(rate) + if rate < 2.0 or rate > 6.0: + far_off.append( + { + "symbol": symbol, + "events": len(symbol_events), + "first": first.isoformat(), + "last": last.isoformat(), + "events_per_year": round(rate, 2), + } + ) - -def _r_dist(rs: list[float]) -> dict[str, Any]: - if not rs: - return {"n": 0} + keys = [(event["symbol"], event["announce_date"]) for event in in_window] + duplicate_rows_in_table = len(keys) - len(set(keys)) + paired_pct = _pct(len(paired), len(in_window)) + ge8_paired_pct = _pct(ge8_paired, len(symbols)) + fallback_needed = paired_pct < 50.0 or ge8_paired_pct < 50.0 return { - "n": len(rs), - "mean": round(sum(rs) / len(rs), 4), - "win_rate": round(sum(1 for r in rs if r > 0) / len(rs), 4), - "p05": round(_percentile(rs, 0.05), 4), - "p25": round(_percentile(rs, 0.25), 4), - "p50": round(_percentile(rs, 0.50), 4), - "p75": round(_percentile(rs, 0.75), 4), - "p95": round(_percentile(rs, 0.95), 4), - "min": round(min(rs), 4), - "max": round(max(rs), 4), + "window": {"from": window_start.isoformat(), "to": window_end.isoformat()}, + "prod_symbols": len(symbols), + "events": len(in_window), + "symbols_with_any_event": len(by_symbol), + "symbols_with_ge8_announcements": ge8, + "symbols_with_ge8_announcements_pct": _pct(ge8, len(symbols)), + "symbols_with_ge8_paired_announcements": ge8_paired, + "symbols_with_ge8_paired_announcements_pct": ge8_paired_pct, + "events_with_actual_and_estimate": len(paired), + "events_with_actual_and_estimate_pct": paired_pct, + "duplicate_rows_in_table": duplicate_rows_in_table, + "duplicate_rows_fetched": backfill_status.get( + "duplicate_rows_logged_total", 0 + ), + "restated_rows_fetched": backfill_status.get( + "restated_rows_logged_total", 0 + ), + "dedupe_policy": backfill_status.get("dedupe_policy"), + "events_per_symbol_year": { + "mean_active_span_rate": ( + round(sum(annual_rates) / len(annual_rates), 2) + if annual_rates + else None + ), + "expected": "approximately 4", + "far_off_rule": "active-span rate <2 or >6, plus zero-event symbols", + "far_off_count": len(far_off), + "far_off_symbols": far_off, + }, + "announcement_session": { + "recognised_bmo_amc_or_during": session_known, + "recognised_pct": session_pct, + "reliable": session_reliable, + "assessment": ( + "usable" + if session_reliable + else "missing/unreliable; do not use same-day availability" + ), + }, + "point_in_time_policy": "announce_date_plus_1_trading_day_for_all_events", + "sue_scaling": { + "primary": "eps_surprise_over_stdev_of_prior_8_surprises_min_4", + "fallback_trigger": ( + "paired event coverage <50% or symbols with >=8 paired events <50%" + ), + "fallback_needed": fallback_needed, + "fallback_name": ( + "eps_surprise_over_price" + if fallback_needed + else "not_used" + ), + }, + "backfill": backfill_status, } -def _trading_days_between( - entry: date, exit_: date, calendar: set[date] -) -> list[date]: - """Inclusive trading dates in [entry, exit_] present on the union calendar.""" - out = [] - d = entry - while d <= exit_: - if d in calendar: - out.append(d) - d += timedelta(days=1) - return out - - -def _nth_trading_day_after( - start: date, n: int, ordered_calendar: list[date] -) -> date | None: - """First calendar date strictly after ``start``, then + (n-1) more sessions. - - announce+1 trading day: n=1 → first session after announce date - (if announce is a trading day, still use the *next* session for PIT). - """ - # Sessions strictly after start. - after = [d for d in ordered_calendar if d > start] - if len(after) < n: +def _percentile(values: list[float], quantile: float) -> float | None: + if not values: return None - return after[n - 1] + ordered = sorted(values) + if len(ordered) == 1: + return ordered[0] + location = quantile * (len(ordered) - 1) + lower = int(math.floor(location)) + upper = int(math.ceil(location)) + if lower == upper: + return ordered[lower] + weight = location - lower + return ordered[lower] * (1.0 - weight) + ordered[upper] * weight -def _build_sue_series( - events_by_symbol: dict[str, list[dict]], - prices: dict[str, tuple], -) -> dict[str, dict[date, float]]: - """symbol → {asof_date: sue_value} for days when SUE is live (announce+1 .. +63).""" - out: dict[str, dict[date, float]] = {} - for sym, cols in prices.items(): - ords = cols[0] - closes = cols[4] - dates = [date.fromordinal(int(o)) for o in ords] - if not dates: +def _r_dist(values: list[float]) -> dict[str, Any]: + if not values: + return {"count": 0} + return { + "count": len(values), + "mean_r": round(sum(values) / len(values), 4), + "median_r": round(float(_percentile(values, 0.50)), 4), + "win_rate": round(sum(value > 0 for value in values) / len(values), 4), + "p05_r": round(float(_percentile(values, 0.05)), 4), + "p95_r": round(float(_percentile(values, 0.95)), 4), + "min_r": round(min(values), 4), + "max_r": round(max(values), 4), + } + + +def _first_session_after(announcement: date, calendar: list[date]) -> date | None: + index = bisect.bisect_right(calendar, announcement) + return calendar[index] if index < len(calendar) else None + + +def _entry_in_last_three_sessions( + entry: date, announcement: date, calendar: list[date] +) -> bool: + index = bisect.bisect_left(calendar, announcement) + return entry in calendar[max(0, index - 3) : index] + + +def _analyse_2a_trades( + details: list[dict[str, Any]], + events: list[dict[str, Any]], + calendar: list[date], + *, + cost_per_side: float, +) -> dict[str, Any]: + by_symbol: dict[str, list[date]] = defaultdict(list) + for event in events: + by_symbol[event["symbol"]].append(event["announce_date"]) + for dates in by_symbol.values(): + dates.sort() + + parsed = [] + for trade in details: + symbol = str(trade.get("symbol") or "").upper() + entry_raw = trade.get("entry_date") + exit_raw = trade.get("exit_date") + raw_r = trade.get("r") + if entry_raw is None or exit_raw is None or raw_r is None: continue - ordered = dates # already chronological - cal_set = set(ordered) - events = events_by_symbol.get(sym.upper(), []) - # Chronological surprises with actual+estimate. - surprises: list[tuple[date, float, float]] = [] # announce, surprise, close_for_scale - for ev in events: - act, est = ev.get("eps_actual"), ev.get("eps_estimate") - if act is None or est is None: - continue - ad = ev["announce_date"] - # Close on/before announce for price fallback scale. - close_px = None - for d, c in zip(reversed(dates), reversed(closes)): - if d <= ad and float(c) > 0: - close_px = float(c) - break - surprises.append((ad, float(act) - float(est), close_px or 1.0)) - surprises.sort(key=lambda x: x[0]) + entry_date = date.fromisoformat(str(entry_raw)[:10]) + exit_date = date.fromisoformat(str(exit_raw)[:10]) + entry_price = float(trade.get("entry") or 0.0) + initial_stop = float(trade.get("initial_stop") or 0.0) + exit_fill = float(trade.get("fill") or 0.0) + risk = entry_price - initial_stop + if risk <= 0: + continue + net_r = float(raw_r) - cost_per_side * (entry_price + exit_fill) / risk + announcements = by_symbol.get(symbol, []) + strict_hold = [ + event_date + for event_date in announcements + if entry_date < event_date < exit_date + ] + pre_entry = any( + _entry_in_last_three_sessions(entry_date, event_date, calendar) + for event_date in announcements + ) + is_stop = str(trade.get("reason") or "") in {"stop", "trailing_stop"} + stop_after = is_stop and any( + _first_session_after(event_date, calendar) == exit_date + for event_date in announcements + ) + parsed.append( + { + "symbol": symbol, + "entry": entry_date, + "exit": exit_date, + "net_r": net_r, + "earnings_strictly_in_hold": bool(strict_hold), + "pre_earnings_entry": pre_entry, + "is_stop": is_stop, + "stop_within_1d_after_earnings": stop_after, + } + ) - sue_on_day: dict[date, float] = {} - for i, (ad, surprise, px) in enumerate(surprises): - trail = [surprises[j][1] for j in range(max(0, i - SUE_TRAIL), i)] - # Need history of surprises; include current only for value, stdev from prior 8. - if len(trail) >= 3: - mean_t = sum(trail) / len(trail) - var = sum((x - mean_t) ** 2 for x in trail) / (len(trail) - 1) - sd = math.sqrt(var) if var > 0 else None - else: - sd = None - if sd is not None and sd > 1e-9: - sue = surprise / sd - else: - # Fallback: scale by price (EPS surprise / price). - sue = surprise / px if px > 0 else None - if sue is None or not math.isfinite(sue): - continue - usable_from = _nth_trading_day_after(ad, 1, ordered) - if usable_from is None: - continue - # Carry for SUE_CARRY_DAYS trading sessions starting at usable_from. - try: - start_idx = ordered.index(usable_from) - except ValueError: - # usable_from not in this symbol's calendar (halted etc.) - start_idx = next( - (k for k, d in enumerate(ordered) if d >= usable_from), None - ) - if start_idx is None: - continue - end_idx = min(len(ordered) - 1, start_idx + SUE_CARRY_DAYS - 1) - for k in range(start_idx, end_idx + 1): - # Later announcements overwrite earlier carry (latest SUE wins). - sue_on_day[ordered[k]] = sue - if sue_on_day: - out[sym.upper()] = sue_on_day - return out + losses = [trade for trade in parsed if trade["net_r"] <= -1.0] + losses_with = [trade for trade in losses if trade["earnings_strictly_in_hold"]] + all_with = [trade for trade in parsed if trade["earnings_strictly_in_hold"]] + pre = [trade["net_r"] for trade in parsed if trade["pre_earnings_entry"]] + other = [trade["net_r"] for trade in parsed if not trade["pre_earnings_entry"]] + stops = [trade for trade in parsed if trade["is_stop"]] + stops_after = [ + trade["net_r"] for trade in stops if trade["stop_within_1d_after_earnings"] + ] + other_stops = [ + trade["net_r"] for trade in stops if not trade["stop_within_1d_after_earnings"] + ] + all_other_exits = [ + trade["net_r"] + for trade in parsed + if not trade["stop_within_1d_after_earnings"] + ] + pre_dist = _r_dist(pre) + other_dist = _r_dist(other) + left_delta = None + right_delta = None + if pre and other: + left_delta = round(pre_dist["p05_r"] - other_dist["p05_r"], 4) + right_delta = round(pre_dist["p95_r"] - other_dist["p95_r"], 4) + tail_condition = ( + left_delta is not None + and left_delta < 0 + and right_delta is not None + and right_delta <= 0 + ) + return { + "verdict": "INFORMATIONAL", + "costs": {"per_side": cost_per_side, "r_is_net_of_round_trip_costs": True}, + "closed_trades": len(parsed), + "q1_loss_concentration": { + "loss_definition": "realized_net_R <= -1.0", + "holding_period_definition": "announcement strictly after entry and before exit", + "losses_count": len(losses), + "losses_with_announcement_count": len(losses_with), + "losses_with_announcement_fraction": ( + round(len(losses_with) / len(losses), 4) if losses else None + ), + "all_trades_with_announcement_count": len(all_with), + "all_trades_with_announcement_fraction": ( + round(len(all_with) / len(parsed), 4) if parsed else None + ), + }, + "q2_entries_within_3_trading_days_before_announcement": { + "pre_earnings": pre_dist, + "all_other_entries": other_dist, + "tail_deltas_pre_minus_other": { + "p05_r": left_delta, + "p95_r": right_delta, + }, + "directional_tail_condition_present": tail_condition, + "tail_read": ( + "Directional left-worse/right-not-better condition is present; " + "materiality and any filter design require separate human approval." + if tail_condition + else "Registered directional tail condition is not present." + ), + }, + "q3_stop_exits_within_1_trading_day_after_announcement": { + "stops_after_earnings": _r_dist(stops_after), + "all_other_stops": _r_dist(other_stops), + "all_other_exits": _r_dist(all_other_exits), + }, + "implementation": "REPORT_ONLY_NO_FILTER_ARM_NO_FILTER_CHANGE", + } async def _run_2a( snapshot: Path, - events: list[dict], + events: list[dict[str, Any]], + symbols: set[str], *, - quiet: bool, + analysis_start: date, + analysis_end: date, workers: int, + quiet: bool, ) -> dict[str, Any]: from app.config import settings + from app.models.ticker import Ticker from app.services import backtest_service as bt from app.services.admin_service import get_activation_config - from app.services.recommendation_service import get_recommendation_config - from app.services.paper_trade_service import get_exit_policy from app.services.benchmark_service import load_benchmark_closes - from app.models.ticker import Ticker - from sqlalchemy import select + from app.services.paper_trade_service import get_exit_policy + from app.services.recommendation_service import get_recommendation_config os.environ["BACKTEST_SNAPSHOT_OFFLINE"] = "1" settings.backtest_workers = workers - engine = create_async_engine(_sqlite_url(snapshot), pool_pre_ping=True) - Session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) - + session_factory = async_sessionmaker( + engine, class_=AsyncSession, expire_on_commit=False + ) try: - async with Session() as db: + async with session_factory() as db: config = await get_recommendation_config(db) activation = await get_activation_config(db) exit_config = await get_exit_policy(db) tickers = list( - (await db.execute(select(Ticker).order_by(Ticker.symbol))).scalars() + ( + await db.execute( + select(Ticker) + .where(Ticker.symbol.in_(sorted(symbols))) + .order_by(Ticker.symbol) + ) + ).scalars() ) spy = await load_benchmark_closes(db, "SPY") prices: dict[str, tuple] = {} - candidates: list[dict] = [] - for idx, t in enumerate(tickers): - if not quiet and idx % 50 == 0: - print(f" 2a fetch {idx}/{len(tickers)}", end="\r", flush=True) - cols = await bt._fetch_columns(db, t.symbol) - if cols is None: + replay_inputs: list[tuple[str, tuple]] = [] + for index, ticker in enumerate(tickers): + if not quiet and index % 50 == 0: + print(f" 2a load {index}/{len(tickers)}", flush=True) + columns = await bt._fetch_columns(db, ticker.symbol) + if columns is None: continue - prices[t.symbol] = cols - cands, _ = bt._replay_and_signals( - t.symbol, - cols, - config, - activation, - spy, - bt.PRODUCTION_GTL_TARGET_MODEL, - "weekly", - False, - ) - candidates.extend(cands) + prices[ticker.symbol] = columns + replay_inputs.append((ticker.symbol, columns)) finally: await engine.dispose() - if not quiet: - print() - # Production ranks + qualify. + candidates: list[dict] = [] + process_count = bt._backtest_worker_count() + context = bt._mp_context() if process_count > 1 else None + if context is not None: + loop = asyncio.get_running_loop() + chunk_size = process_count * 2 + with ProcessPoolExecutor( + max_workers=process_count, mp_context=context + ) as pool: + for start in range(0, len(replay_inputs), chunk_size): + batch = replay_inputs[start : start + chunk_size] + futures = [ + loop.run_in_executor( + pool, + bt._replay_candidates_for_period, + symbol, + columns, + config, + activation, + spy, + analysis_start, + "weekly", + True, + False, + ) + for symbol, columns in batch + ] + for ticker_candidates in await asyncio.gather(*futures): + candidates.extend(ticker_candidates) + if not quiet: + print( + f" 2a replay {min(start + len(batch), len(replay_inputs))}/" + f"{len(replay_inputs)} workers={process_count}", + flush=True, + ) + else: + for index, (symbol, columns) in enumerate(replay_inputs): + if not quiet and index % 25 == 0: + print(f" 2a replay {index}/{len(replay_inputs)}", flush=True) + ticker_candidates = bt._replay_candidates_for_period( + symbol, + columns, + config, + activation, + spy, + analysis_start, + "weekly", + True, + False, + ) + candidates.extend(ticker_candidates) + bt._assign_momentum_percentiles(candidates) bt._assign_residual_momentum_percentiles(candidates) bt._assign_low_volatility_percentiles(candidates) bt._assign_activation_momentum_percentiles(candidates) bt._assign_residual_high_vol_blend(candidates) - for c in candidates: - c["qualified"] = bt._momentum_qualifies(c, 80.0) + cutoff = float(activation.get("min_momentum_percentile", 80.0)) + for candidate in candidates: + candidate["qualified"] = bt._momentum_qualifies(candidate, cutoff) longs = [ - c for c in candidates if c.get("qualified") and c.get("direction") == "long" + candidate + for candidate in candidates + if candidate.get("qualified") and candidate.get("direction") == "long" ] - - strategy = next(s for s in bt.PORTFOLIO_MONITOR_STRATEGIES if s.get("is_production")) - entry_cfg = bt._entry_variant_config(str(strategy["entry_variant"])) - assert entry_cfg is not None - ranking_key = str(entry_cfg.get("ranking_key") or entry_cfg["percentile_key"]) + strategy = next( + row for row in bt.PORTFOLIO_MONITOR_STRATEGIES if row.get("is_production") + ) + entry_config = bt._entry_variant_config(str(strategy["entry_variant"])) + if entry_config is None: + raise RuntimeError("Production entry configuration missing") + ranking_key = str( + entry_config.get("ranking_key") or entry_config["percentile_key"] + ) exit_policy = bt.LIVE_EXIT_MODE_TO_SIM.get( str(exit_config.get("mode", "atr_trailing")), "atr_trail3" ) @@ -325,580 +691,858 @@ async def _run_2a( reentry = bt._make_gate_reset_reentry_fn( longs, prices, cadence="weekly", ranking_key=ranking_key ) - sim = bt._simulate_portfolio( + simulation = bt._simulate_portfolio( longs, prices, spy, exit_policy, hold_days, ranking_key=ranking_key, - max_positions=int(entry_cfg["max_positions"]), - risk_per_trade=float(entry_cfg["risk_per_trade"]), + max_positions=int(entry_config["max_positions"]), + risk_per_trade=float(entry_config["risk_per_trade"]), atr_trail_multiplier=trail, post_stop_reentry_fn=reentry, fill_mode=bt.FILL_MODE_CLOSE, + cost_per_side=COST_PER_SIDE, include_trades=True, ) - if sim is None: - return {"error": "no_trades"} - - details = sim.get("trade_details") or [] - # Build per-symbol earnings announce dates. - earns_by_sym: dict[str, list[date]] = defaultdict(list) - for ev in events: - earns_by_sym[ev["symbol"]].append(ev["announce_date"]) - for sym in earns_by_sym: - earns_by_sym[sym].sort() - - # Union trading calendar from prices. - cal: set[date] = set() - for cols in prices.values(): - for o in cols[0]: - cal.add(date.fromordinal(int(o))) - ordered_cal = sorted(cal) - - # Map entry date → list of announce dates for symbol (for pre-entry lookback). - trades_parsed: list[dict] = [] - for t in details: - sym = str(t.get("symbol") or "").upper() - # Field names from simulator. - entry_s = t.get("entry_date") or t.get("open_date") or t.get("date") - exit_s = t.get("exit_date") or t.get("close_date") - r = t.get("realized_r") - if r is None: - r = t.get("r") - if entry_s is None or exit_s is None or r is None: + if simulation is None: + raise RuntimeError("Production-config simulation returned no result") + calendar = sorted( + { + date.fromordinal(int(ordinal)) + for columns in prices.values() + for ordinal in columns[0] + } + ) + all_trade_details = simulation.get("trade_details") or [] + eligible_trade_details = [] + for trade in all_trade_details: + entry_raw = trade.get("entry_date") + exit_raw = trade.get("exit_date") + if entry_raw is None or exit_raw is None: continue - entry_d = date.fromisoformat(str(entry_s)[:10]) - exit_d = date.fromisoformat(str(exit_s)[:10]) - announces = earns_by_sym.get(sym, []) - # Earnings between entry and exit (exclusive of entry day? inclusive hold). - # "between entry and exit" — any announce with entry < announce <= exit - # (gap often overnight after entry). Also count announce on entry day. - in_hold = [ - a for a in announces if entry_d <= a <= exit_d - ] - # Entries within 3 trading days BEFORE an announcement: - # exists announce such that entry is in the 3 sessions immediately before announce. - pre_earn = False - for a in announces: - # trading sessions in (a-lookback, a) - sessions_before = [d for d in ordered_cal if d < a] - last3 = sessions_before[-3:] if len(sessions_before) >= 3 else sessions_before - if entry_d in last3: - pre_earn = True - break - trades_parsed.append({ - "symbol": sym, - "entry": entry_d.isoformat(), - "exit": exit_d.isoformat(), - "r": float(r), - "earnings_in_hold": len(in_hold) > 0, - "n_earnings_in_hold": len(in_hold), - "entry_within_3d_before_earn": pre_earn, - }) + entry_date = date.fromisoformat(str(entry_raw)[:10]) + exit_date = date.fromisoformat(str(exit_raw)[:10]) + if analysis_start <= entry_date and exit_date <= analysis_end: + eligible_trade_details.append(trade) + analysis = _analyse_2a_trades( + eligible_trade_details, + events, + calendar, + cost_per_side=COST_PER_SIDE, + ) + analysis["analysis_window"] = { + "from": analysis_start.isoformat(), + "to": analysis_end.isoformat(), + "rule": "entry_on_or_after_start_and_exit_on_or_before_end", + "simulation_trades_total": len(all_trade_details), + "trades_excluded_outside_earnings_coverage": ( + len(all_trade_details) - len(eligible_trade_details) + ), + } + analysis["run_config"] = { + "universe_symbols": len(tickers), + "fill_mode": "close", + "cost_per_side": COST_PER_SIDE, + "momentum_cutoff": cutoff, + "exit_policy": exit_policy, + "hold_days": hold_days, + "max_positions": int(entry_config["max_positions"]), + "risk_per_trade": float(entry_config["risk_per_trade"]), + } + analysis["sim_summary"] = { + key: simulation.get(key) + for key in ( + "start_date", + "end_date", + "trades", + "sharpe", + "cagr_pct", + "max_drawdown_pct", + "total_return_pct", + ) + } + return analysis - all_r = [t["r"] for t in trades_parsed] - loss_lt_1r = [t for t in trades_parsed if t["r"] < -1.0] - loss_with_earn = [t for t in loss_lt_1r if t["earnings_in_hold"]] - pre = [t["r"] for t in trades_parsed if t["entry_within_3d_before_earn"]] - other = [t["r"] for t in trades_parsed if not t["entry_within_3d_before_earn"]] - return { - "sim_summary": { - k: sim.get(k) - for k in ( - "sharpe", - "sharpe_se", - "cagr_pct", - "max_drawdown_pct", - "trades", - "total_return_pct", +def _build_sue_series( + events_by_symbol: dict[str, list[dict[str, Any]]], + prices: dict[str, tuple], + *, + use_price_fallback: bool, + surprise_history_by_symbol: dict[str, list[dict[str, Any]]] | None = None, +) -> tuple[dict[str, dict[date, float]], dict[str, int]]: + result: dict[str, dict[date, float]] = {} + standard_values = 0 + fallback_values = 0 + history_scaled_values = 0 + dropped_insufficient_history = 0 + dropped_missing_period_alignment = 0 + dropped_zero_stdev = 0 + for symbol, columns in prices.items(): + dates = [date.fromordinal(int(value)) for value in columns[0]] + closes = [float(value) for value in columns[4]] + if not dates: + continue + surprises = [] + for event in events_by_symbol.get(symbol.upper(), []): + actual = event.get("eps_actual") + estimate = event.get("eps_estimate") + if actual is None or estimate is None: + continue + surprises.append( + { + "announce_date": event["announce_date"], + "period_end_date": event.get("period_end_date"), + "surprise": float(actual) - float(estimate), + } ) - }, - "n_trades_parsed": len(trades_parsed), - "q1_losses_worse_than_minus_1r": { - "n_losses_lt_minus_1r": len(loss_lt_1r), - "n_with_earnings_in_hold": len(loss_with_earn), - "fraction_with_earnings": ( - round(len(loss_with_earn) / len(loss_lt_1r), 4) if loss_lt_1r else None - ), - "all_trades_with_earnings_in_hold": sum( - 1 for t in trades_parsed if t["earnings_in_hold"] - ), - "fraction_all_trades_with_earnings": ( - round( - sum(1 for t in trades_parsed if t["earnings_in_hold"]) - / len(trades_parsed), - 4, + surprises.sort(key=lambda item: item["announce_date"]) + history = [] + if surprise_history_by_symbol is not None: + for row in surprise_history_by_symbol.get(symbol.upper(), []): + actual = row.get("eps_actual") + estimate = row.get("eps_estimate") + if actual is None or estimate is None: + continue + history.append( + ( + row["period_end_date"], + float(actual) - float(estimate), + ) ) - if trades_parsed - else None - ), - }, - "q2_entry_within_3d_before_announce": { - "pre_earn_entries": _r_dist(pre), - "other_entries": _r_dist(other), - "all_entries": _r_dist(all_r), - "tail_trim_note": ( - "Compare p95/max and mean of pre_earn vs other. " - "Rising win_rate with falling mean/p95 = right-tail trim red flag." - ), - }, - "note": "REPORT-ONLY — no filter shipped.", + history.sort(key=lambda item: item[0]) + live: dict[date, float] = {} + for index, event in enumerate(surprises): + announcement = event["announce_date"] + surprise = float(event["surprise"]) + period_end = event.get("period_end_date") + if surprise_history_by_symbol is not None: + if period_end is None: + dropped_missing_period_alignment += 1 + continue + trailing = [ + value for history_period, value in history if history_period < period_end + ][-SUE_TRAIL:] + else: + trailing = [ + float(prior["surprise"]) + for prior in surprises[max(0, index - SUE_TRAIL) : index] + ] + sue = None + if len(trailing) >= SUE_MIN_TRAIL: + mean = sum(trailing) / len(trailing) + variance = sum((value - mean) ** 2 for value in trailing) / ( + len(trailing) - 1 + ) + stdev = math.sqrt(variance) if variance > 0 else 0.0 + if stdev > 1e-12: + sue = surprise / stdev + standard_values += 1 + if surprise_history_by_symbol is not None: + history_scaled_values += 1 + else: + dropped_zero_stdev += 1 + else: + dropped_insufficient_history += 1 + if sue is None and use_price_fallback: + price_index = bisect.bisect_right(dates, announcement) - 1 + if price_index >= 0 and closes[price_index] > 0: + sue = surprise / closes[price_index] + fallback_values += 1 + if sue is None or not math.isfinite(sue): + continue + start_index = bisect.bisect_right(dates, announcement) + if start_index >= len(dates): + continue + end_index = min(len(dates), start_index + SUE_CARRY_DAYS) + for trading_index in range(start_index, end_index): + live[dates[trading_index]] = float(sue) + if live: + result[symbol.upper()] = live + return result, { + "standard_scaled_events": standard_values, + "events_scaled_from_period_history": history_scaled_values, + "price_fallback_events": fallback_values, + "dropped_insufficient_trailing_history": dropped_insufficient_history, + "dropped_missing_period_alignment": dropped_missing_period_alignment, + "dropped_zero_stdev": dropped_zero_stdev, } -async def _run_2b_ic( +def _find_signal(rows: list[dict[str, Any]], signal: str) -> dict[str, Any] | None: + return next((row for row in rows if row.get("signal") == signal), None) + + +def _mechanical_sue_grade( + sue_row: dict[str, Any] | None, + pre_row: dict[str, Any] | None, + post_row: dict[str, Any] | None, +) -> tuple[bool, bool]: + sign_stable = bool( + pre_row + and post_row + and float(pre_row.get("mean_ic", 0.0)) > 0 + and float(post_row.get("mean_ic", 0.0)) > 0 + ) + passed = bool( + sue_row + and float(sue_row.get("mean_ic", -999.0)) >= IRON_IC_BAR + and bool(sue_row.get("reliable")) + and sign_stable + ) + return passed, sign_stable + + +def _conditional_ic(bt, sue_weeks: dict) -> dict[str, Any]: + usable = [ + week + for week, records in sue_weeks.items() + if len(records) >= bt.MIN_CROSS_SECTION + ] + kept = bt._nonoverlapping_weeks( + usable, max(1, round(bt.HORIZON / 5)) + ) + values = [] + sizes = [] + for week in kept: + records = [ + record for record in sue_weeks[week] if record.get("mom_12_1") is not None + ] + if len(records) < bt.MIN_CROSS_SECTION: + continue + ordered = sorted(records, key=lambda record: float(record["mom_12_1"])) + top = ordered[-max(1, len(ordered) // 5) :] + if len(top) < 5: + continue + ic = bt._spearman( + [float(record["val"]) for record in top], + [float(record["fwd"]) for record in top], + ) + if ic is not None: + values.append(float(ic)) + sizes.append(len(top)) + if not values: + return {"mean_ic": None, "weeks": 0, "avg_cross_section": None} + mean = sum(values) / len(values) + if len(values) > 1: + stdev = math.sqrt( + sum((value - mean) ** 2 for value in values) / (len(values) - 1) + ) + t_stat = mean / stdev * math.sqrt(len(values)) if stdev > 0 else None + else: + t_stat = None + return { + "mean_ic": round(mean, 4), + "ic_t_stat": round(t_stat, 2) if t_stat is not None else None, + "weeks": len(values), + "avg_cross_section": round(sum(sizes) / len(sizes), 1), + "population": "top_mom_12_1_quintile_only", + } + + +async def _run_2b( snapshot: Path, - events: list[dict], + events: list[dict[str, Any]], + surprise_history: dict[str, list[dict[str, Any]]], + symbols: set[str], *, - quiet: bool, + quality: dict[str, Any], workers: int, + quiet: bool, ) -> dict[str, Any]: - """SUE IC via harness on identical cross-sections as momentum baselines.""" from app.config import settings + from app.models.ticker import Ticker from app.services import backtest_service as bt from app.services.benchmark_service import load_benchmark_closes - from app.models.ticker import Ticker - from sqlalchemy import select - from collections import defaultdict as dd os.environ["BACKTEST_SNAPSHOT_OFFLINE"] = "1" os.environ["BACKTEST_SIGNAL_EVAL_ONLY"] = "1" settings.backtest_workers = workers - engine = create_async_engine(_sqlite_url(snapshot), pool_pre_ping=True) - Session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) - - # Collect base signals + attach SUE. - collected: dict = dd(lambda: dd(list)) + session_factory = async_sessionmaker( + engine, class_=AsyncSession, expire_on_commit=False + ) + prices: dict[str, tuple] = {} try: - async with Session() as db: + async with session_factory() as db: tickers = list( - (await db.execute(select(Ticker).order_by(Ticker.symbol))).scalars() + ( + await db.execute( + select(Ticker) + .where(Ticker.symbol.in_(sorted(symbols))) + .order_by(Ticker.symbol) + ) + ).scalars() ) spy = await load_benchmark_closes(db, "SPY") - - prices: dict[str, tuple] = {} - for idx, t in enumerate(tickers): - if not quiet and idx % 50 == 0: - print(f" 2b fetch {idx}/{len(tickers)}", end="\r", flush=True) - cols = await bt._fetch_columns(db, t.symbol) - if cols is None: - continue - prices[t.symbol] = cols - series = bt._signal_series( - [ - type( - "R", - (), - { - "date": date.fromordinal(int(cols[0][i])), - "close": cols[4][i], - "high": cols[2][i], - "volume": cols[5][i] if len(cols) > 5 else 0, - }, - )() - for i in range(len(cols[0])) - ], - spy, - symbol=t.symbol, - ) - for name, weeks in series.items(): - for wk, pairs in weeks.items(): - collected[name][wk].extend(pairs) + for index, ticker in enumerate(tickers): + if not quiet and index % 25 == 0: + print(f" 2b signals {index}/{len(tickers)}", flush=True) + columns = await bt._fetch_columns(db, ticker.symbol) + if columns is not None: + prices[ticker.symbol] = columns finally: await engine.dispose() - if not quiet: - print() - # SUE series. - events_by_sym: dict[str, list[dict]] = defaultdict(list) - for ev in events: - events_by_sym[ev["symbol"]].append(ev) - sue_map = _build_sue_series(events_by_sym, prices) + events_by_symbol: dict[str, list[dict[str, Any]]] = defaultdict(list) + for event in events: + events_by_symbol[event["symbol"]].append(event) + fallback = bool(quality["sue_scaling"]["fallback_needed"]) + sue_map, scaling_counts = _build_sue_series( + events_by_symbol, + prices, + use_price_fallback=fallback, + surprise_history_by_symbol=surprise_history, + ) - # Inject sue_latest into collected using mom_12_1 observations as the - # weekly as-of skeleton (same weeks / symbols). - sue_collected: dict = dd(list) - mom_weeks = collected.get("mom_12_1") or {} - for week_key, recs in mom_weeks.items(): - for rec in recs: - pair = bt._obs_val_fwd(rec) - if pair is None: - continue - _val, fwd = pair - sym = None - if isinstance(rec, dict): - sym = rec.get("symbol") - if not sym: - continue - # Need as-of date: recover from week — use Friday of ISO week as proxy - # is weak. Better: re-derive from prices weekly indices. - # Store asof on rich recs? Current rich rows lack asof date. - # Fall back: compute SUE observations directly from prices weekly as-ofs. - pass - - # Direct weekly as-of SUE + forward return (authoritative). - for sym, cols in prices.items(): - ords, _o, highs, _l, closes, _v = cols - dates = [date.fromordinal(int(o)) for o in ords] - sue_days = sue_map.get(sym.upper()) or {} - if not sue_days: - continue - n = len(dates) - # weekly as-of indices: reuse harness helper via fake records. + sue_all: dict = defaultdict(list) + identical: dict = defaultdict(lambda: defaultdict(list)) + for symbol, columns in prices.items(): + dates = [date.fromordinal(int(value)) for value in columns[0]] + highs = [float(value) for value in columns[2]] + closes = [float(value) for value in columns[4]] + volumes = [float(value) for value in columns[5]] records = [ - type("R", (), {"date": dates[i], "close": closes[i], "high": highs[i]})() - for i in range(n) - ] - for i in bt._weekly_asof_indices(records): - j = i + bt.HORIZON - if j >= n or closes[i] <= 0: - continue - asof = dates[i] - sue = sue_days.get(asof) - if sue is None: - continue - fwd = float(closes[j]) / float(closes[i]) - 1.0 - iso = asof.isocalendar() - week_key = (iso[0], iso[1]) - # Also grab mom for conditional. - mom = None - if i >= 252 and closes[i - 252] > 0: - mom = float(closes[i - 21]) / float(closes[i - 252]) - 1.0 - sue_collected[week_key].append({ - "val": float(sue), - "fwd": fwd, - "symbol": sym, - "mom_12_1": mom, - }) - collected["sue_latest"] = sue_collected - - signal_eval = bt._signal_evaluation(collected) - - # Fair side-by-side: re-evaluate mom baselines on the *same* (symbol, week) - # observations where SUE is present (incomplete backfill otherwise inflates - # mom N relative to SUE). - sue_pairs_by_week = sue_collected - restricted: dict = dd(lambda: dd(list)) - for week_key, recs in sue_pairs_by_week.items(): - syms = {str(r.get("symbol")).upper() for r in recs if r.get("symbol")} - for base_name in ("mom_12_1", "mom_12_1_resid"): - base_recs = (collected.get(base_name) or {}).get(week_key) or [] - for rec in base_recs: - pair = bt._obs_val_fwd(rec) - if pair is None: - continue - sym = None - if isinstance(rec, dict): - sym = rec.get("symbol") - if not sym or str(sym).upper() not in syms: - continue - restricted[base_name][week_key].append(rec) - restricted["sue_latest"][week_key].extend(recs) - restricted_eval = bt._signal_evaluation(restricted) - - # Momentum-conditional: IC of SUE within top mom quintile each week. - cond_ics: list[float] = [] - stride = max(1, round(bt.HORIZON / 5)) - usable = [wk for wk, recs in sue_collected.items() if len(recs) >= bt.MIN_CROSS_SECTION] - kept = bt._nonoverlapping_weeks(usable, stride) - for wk in kept: - recs = sue_collected[wk] - with_mom = [r for r in recs if r.get("mom_12_1") is not None] - if len(with_mom) < bt.MIN_CROSS_SECTION: - continue - ordered = sorted(with_mom, key=lambda r: float(r["mom_12_1"])) - k = max(1, len(ordered) // 5) - top = ordered[-k:] - if len(top) < 5: - continue - ic = bt._spearman( - [float(r["val"]) for r in top], - [float(r["fwd"]) for r in top], - ) - if ic is not None: - cond_ics.append(ic) - if cond_ics: - mean_c = sum(cond_ics) / len(cond_ics) - if len(cond_ics) > 1: - std = math.sqrt( - sum((x - mean_c) ** 2 for x in cond_ics) / (len(cond_ics) - 1) + SimpleNamespace( + date=dates[index], + close=closes[index], + high=highs[index], + volume=volumes[index], ) - t_c = mean_c / std * math.sqrt(len(cond_ics)) if std > 0 else None - else: - t_c = None - mom_cond = { - "mean_ic": round(mean_c, 4), - "ic_t_stat": round(t_c, 2) if t_c is not None else None, - "weeks": len(cond_ics), - "note": "IC of sue_latest within top mom_12_1 quintile (non-overlapping weeks)", - } - else: - mom_cond = {"mean_ic": None, "weeks": 0} + for index in range(len(dates)) + ] + live_sue = sue_map.get(symbol.upper(), {}) + if not live_sue: + continue + for index in bt._weekly_asof_indices(records): + forward_index = index + bt.HORIZON + if forward_index >= len(records) or closes[index] <= 0: + continue + as_of = dates[index] + sue_value = live_sue.get(as_of) + if sue_value is None: + continue + forward = closes[forward_index] / closes[index] - 1.0 + signal_values = bt._signal_values( + dates, closes, highs, index, spy + ) + momentum = signal_values.get("mom_12_1") + residual = signal_values.get("mom_12_1_resid") + iso = as_of.isocalendar() + week = (iso.year, iso.week) + sue_record = { + "val": float(sue_value), + "fwd": float(forward), + "symbol": symbol, + "mom_12_1": momentum, + } + sue_all[week].append(sue_record) + if momentum is not None and residual is not None: + identical["sue_latest"][week].append(sue_record) + identical["mom_12_1"][week].append( + {"val": float(momentum), "fwd": forward, "symbol": symbol} + ) + identical["mom_12_1_resid"][week].append( + {"val": float(residual), "fwd": forward, "symbol": symbol} + ) - def _find(name: str) -> dict | None: - for row in signal_eval: - if row.get("signal") == name: - return row - return None - - sue = _find("sue_latest") - grade = { - "green": False, - "reason": "sue_latest missing", - } - if sue: - mean_ic = sue.get("mean_ic") - t = sue.get("ic_t_stat") - reliable = bool(sue.get("reliable")) - sign_ok = mean_ic is not None and float(mean_ic) > 0 - mag_ok = mean_ic is not None and abs(float(mean_ic)) >= IRON_IC_BAR - grade = { - "green": bool(sign_ok and mag_ok and reliable), - "checks": { - "mean_ic": mean_ic, - "sign_positive": sign_ok, - "abs_ge_0_03": mag_ok, - "reliable": reliable, - "ic_t_stat": t, - "weeks": sue.get("weeks"), - }, - "reason": ( - "iron rule cleared — STOP; book-integration is a separate human step" - if (sign_ok and mag_ok and reliable) - else "iron rule not met" - ), - "row": sue, - } - - def _find_r(name: str) -> dict | None: - for row in restricted_eval: - if row.get("signal") == name: - return row - return None - - # Side-by-side baselines from same evaluation. - side = { - name: _find(name) - for name in ( - "mom_12_1", - "mom_12_1_resid", - "sue_latest", - "fip_id", - ) - } - side_restricted = { - name: _find_r(name) - for name in ("mom_12_1", "mom_12_1_resid", "sue_latest") + full_eval = bt._signal_evaluation({"sue_latest": sue_all}) + identical_eval = bt._signal_evaluation(identical) + pre_eval = bt._signal_evaluation( + {"sue_latest": {week: rows for week, rows in sue_all.items() if week[0] < 2021}} + ) + post_eval = bt._signal_evaluation( + {"sue_latest": {week: rows for week, rows in sue_all.items() if week[0] >= 2021}} + ) + sue_row = _find_signal(full_eval, "sue_latest") + pre_row = _find_signal(pre_eval, "sue_latest") + post_row = _find_signal(post_eval, "sue_latest") + pass_grade, sign_stable = _mechanical_sue_grade(sue_row, pre_row, post_row) + verdict = "PASS" if pass_grade else "FAIL" + verdict_detail = ( + "SUE candidate confirmed - book-integration design (tilt vs second gate) " + "is PENDING_HUMAN. Do not integrate anything yourself." + if pass_grade + else "SUE DEAD for this stack" + ) + weekly_sizes = [len(rows) for rows in sue_all.values()] + average_weekly_n = ( + round(sum(weekly_sizes) / len(weekly_sizes), 1) if weekly_sizes else 0.0 + ) + average_scored_n = sue_row.get("avg_cross_section") if sue_row else None + thin = average_scored_n is None or float(average_scored_n) < 100.0 + side_by_side = { + signal: _find_signal(identical_eval, signal) + for signal in ("sue_latest", "mom_12_1", "mom_12_1_resid") } return { - "signal_eval_side_by_side": side, - "signal_eval_identical_sue_subset": side_restricted, - "identical_subset_note": ( - "Mom baselines re-scored only on (week, symbol) cells where SUE exists. " - "Use this table when backfill is incomplete — full-universe mom N is not comparable." - ), - "full_signal_eval": signal_eval, - "sue_grade": grade, - "momentum_conditional_sue": mom_cond, - "sue_coverage": { - "symbols_with_sue": len(sue_map), - "avg_weeks_with_sue": ( - round( - sum(len(v) for v in sue_collected.values()) - / max(1, len(sue_collected)), - 1, - ) - if sue_collected - else 0 + "verdict": verdict, + "verdict_detail": verdict_detail, + "grade_rule": { + "mean_ic_ge_0_03_positive": ( + bool(sue_row and float(sue_row.get("mean_ic", -999.0)) >= IRON_IC_BAR) ), - "weeks_with_min_cross_section": len(usable), + "reliable_ge_12_windows": bool(sue_row and sue_row.get("reliable")), + "positive_sign_pre_and_post_2021": sign_stable, + "pass": pass_grade, }, + "sue_unconditional": sue_row, + "era_split": {"pre_2021": pre_row, "post_2021": post_row}, + "signal_eval_identical_cross_sections": side_by_side, + "identical_cross_section_definition": ( + "same week-symbol-forward-return cells where sue_latest, mom_12_1, " + "and mom_12_1_resid are all non-null" + ), + "momentum_conditional_top_quintile": _conditional_ic(bt, sue_all), + "coverage": { + "symbols_with_live_sue": len(sue_map), + "avg_weekly_live_n_all_weeks": average_weekly_n, + "avg_cross_section_n_scored_nonoverlap": average_scored_n, + "thin_cross_section_lt_100": thin, + "warning": ( + "THIN CROSS-SECTION: fewer than 100 live SUE names per scored week." + if thin + else None + ), + }, + "scaling": { + "method": quality["sue_scaling"]["primary"], + "fallback": quality["sue_scaling"]["fallback_name"], + "counts": scaling_counts, + "pre_coverage_history_policy": ( + "period-end EPS surprises may scale later events but are never " + "treated as live signals without an announcement date" + ), + "availability": "announce_date_plus_1_trading_day", + "carry_trading_days": SUE_CARRY_DAYS, + }, + "universe_symbols": len(prices), } -def _write_md(path: Path, payload: dict) -> None: - pre = path.read_text(encoding="utf-8") if path.exists() else "" - marker = "## Results" - idx = pre.find(marker) - header = pre[:idx] if idx >= 0 else pre.split("## Verdict")[0] +def _format(value: Any) -> str: + if value is None: + return "-" + if isinstance(value, bool): + return "true" if value else "false" + return str(value) + +def _quality_markdown(quality: dict[str, Any], depth: dict[str, Any]) -> str: + annual = quality["events_per_symbol_year"] + session = quality["announcement_session"] + backfill = quality["backfill"] + source = backfill.get("source") or {} lines = [ - header.rstrip(), + "### Data quality gate", + "", + ( + f"Approved earnings window: {quality['window']['from']} to " + f"{quality['window']['to']}. Source mode: {backfill.get('mode')}." + ), + "", + "| check | result |", + "|---|---:|", + f"| Prod symbols requested / tradable | {depth['requested_symbols']} / {depth['tradable_symbols']} |", + f"| Manifest complete + live counts match | {depth['manifest'].get('complete')} |", + f"| Prod symbols with pre-2021 bars | {depth['symbols_with_pre2021_bars']} ({depth['symbols_with_pre2021_bars_pct']}%) |", + f"| SPY benchmark depth | {depth['benchmark_spy']['rows']} rows, {depth['benchmark_spy']['min']} to {depth['benchmark_spy']['max']} |", + f"| Snapshot depth gate | {depth['gate_pass']} |", + f"| Bulk source windows / requests logged | {backfill.get('bulk_windows_done')}/{backfill.get('bulk_windows_total')} / {backfill.get('bulk_requests_logged_total')} |", + f"| Source repository / pinned commit | {source.get('repository')} @ {source.get('commit')} |", + f"| Source license / upstream provider documented | {source.get('license')} / {source.get('upstream_provider_documented')} |", + f"| Existing-source conflicts preserved | {backfill.get('conflicting_existing_rows')} rows / {backfill.get('conflicting_existing_fields')} fields |", + f"| Symbols with >=8 announcements | {quality['symbols_with_ge8_announcements']} ({quality['symbols_with_ge8_announcements_pct']}%) |", + f"| Symbols with >=8 paired announcements | {quality['symbols_with_ge8_paired_announcements']} ({quality['symbols_with_ge8_paired_announcements_pct']}%) |", + f"| Events with estimate + actual | {quality['events_with_actual_and_estimate']}/{quality['events']} ({quality['events_with_actual_and_estimate_pct']}%) |", + f"| Duplicate rows in keyed table | {quality['duplicate_rows_in_table']} |", + f"| Duplicate / restated payload rows fetched | {quality['duplicate_rows_fetched']} / {quality['restated_rows_fetched']} |", + f"| Mean announcements per active symbol-year | {annual['mean_active_span_rate']} (expected about 4) |", + f"| Symbols far off (<2 or >6/year, incl. zero) | {annual['far_off_count']} |", + f"| Recognised BMO/AMC/during | {session['recognised_pct']}% (reliable={session['reliable']}) |", + f"| Point-in-time policy | {quality['point_in_time_policy']} |", + f"| SUE price fallback | {quality['sue_scaling']['fallback_name']} |", + "", + f"Deduplication: {quality['dedupe_policy']}", + "", + "Far-off announcement-rate symbols: " + + (", ".join(row["symbol"] for row in annual["far_off_symbols"]) or "none"), + ] + return "\n".join(lines) + + +def _dist_markdown(label: str, row: dict[str, Any]) -> str: + return ( + f"| {label} | {_format(row.get('count'))} | {_format(row.get('mean_r'))} | " + f"{_format(row.get('median_r'))} | {_format(row.get('win_rate'))} | " + f"{_format(row.get('p05_r'))} | {_format(row.get('p95_r'))} |" + ) + + +def _two_a_markdown(result: dict[str, Any]) -> str: + q1 = result["q1_loss_concentration"] + q2 = result["q2_entries_within_3_trading_days_before_announcement"] + q3 = result["q3_stop_exits_within_1_trading_day_after_announcement"] + window = result["analysis_window"] + lines = [ + "### Experiment 2a - earnings-gap risk diagnostic", + "", + "Verdict: **INFORMATIONAL**. Report-only; no filter arm or implementation.", + "", + ( + f"Trade cohort is restricted to the approved earnings-coverage window " + f"{window['from']} to {window['to']}; " + f"{window['trades_excluded_outside_earnings_coverage']} simulated trades " + "outside that window were excluded." + ), + "", + "| cohort | count | fraction |", + "|---|---:|---:|", + f"| Realized net R <= -1.0 | {q1['losses_count']} | - |", + f"| Losses with announcement strictly inside hold | {q1['losses_with_announcement_count']} | {q1['losses_with_announcement_fraction']} |", + f"| All trades with announcement strictly inside hold | {q1['all_trades_with_announcement_count']} | {q1['all_trades_with_announcement_fraction']} |", + "", + "| Entry cohort | count | mean R | median R | win rate | p05 R | p95 R |", + "|---|---:|---:|---:|---:|---:|---:|", + _dist_markdown("Within 3 sessions before earnings", q2["pre_earnings"]), + _dist_markdown("All other entries", q2["all_other_entries"]), + "", + f"Tail deltas (pre minus other): p05={q2['tail_deltas_pre_minus_other']['p05_r']}, p95={q2['tail_deltas_pre_minus_other']['p95_r']}.", + "", + q2["tail_read"], + "", + "| Exit cohort | count | mean R | median R | win rate | p05 R | p95 R |", + "|---|---:|---:|---:|---:|---:|---:|", + _dist_markdown("Stops within 1 session after earnings", q3["stops_after_earnings"]), + _dist_markdown("All other stops", q3["all_other_stops"]), + _dist_markdown("All other exits", q3["all_other_exits"]), + ] + return "\n".join(lines) + + +def _ic_row(label: str, row: dict[str, Any] | None) -> str: + row = row or {} + return ( + f"| {label} | {_format(row.get('mean_ic'))} | {_format(row.get('ic_t_stat'))} | " + f"{_format(row.get('weeks'))} | {_format(row.get('avg_cross_section'))} | " + f"{_format(row.get('ic_positive_pct'))} | {_format(row.get('reliable'))} |" + ) + + +def _two_b_markdown(result: dict[str, Any]) -> str: + side = result["signal_eval_identical_cross_sections"] + era = result["era_split"] + coverage = result["coverage"] + conditional = result["momentum_conditional_top_quintile"] + lines = [ + "### Experiment 2b - SUE / post-earnings drift", + "", + f"Mechanical verdict: **{result['verdict']}** - {result['verdict_detail']}", + "", + "Identical cross-sections:", + "", + "| signal | mean IC | t | windows | avg N | IC positive % | reliable |", + "|---|---:|---:|---:|---:|---:|---|", + _ic_row("sue_latest", side.get("sue_latest")), + _ic_row("mom_12_1", side.get("mom_12_1")), + _ic_row("mom_12_1_resid", side.get("mom_12_1_resid")), + "", + "Unconditional SUE grade row:", + "", + "| signal | mean IC | t | windows | avg N | IC positive % | reliable |", + "|---|---:|---:|---:|---:|---:|---|", + _ic_row("sue_latest", result.get("sue_unconditional")), + "", + "Era stability:", + "", + "| era | mean IC | t | windows | avg N | IC positive % | reliable |", + "|---|---:|---:|---:|---:|---:|---|", + _ic_row("pre-2021", era.get("pre_2021")), + _ic_row("post-2021", era.get("post_2021")), + "", + f"Coverage: {coverage['symbols_with_live_sue']} symbols with live SUE; avg weekly N={coverage['avg_weekly_live_n_all_weeks']}; scored non-overlap avg N={coverage['avg_cross_section_n_scored_nonoverlap']}.", + "", + coverage.get("warning") or "Cross-section is not flagged thin at the registered <100-name read.", + "", + f"Momentum-conditional top-quintile SUE: mean IC={conditional.get('mean_ic')}, t={conditional.get('ic_t_stat')}, windows={conditional.get('weeks')}, avg N={conditional.get('avg_cross_section')}.", + ] + return "\n".join(lines) + + +def _write_reports( + *, + stamp: str, + generated_at: str, + snapshot: Path, + depth: dict[str, Any], + quality: dict[str, Any], + result_2a: dict[str, Any], + result_2b: dict[str, Any], +) -> tuple[Path, Path]: + reports = Path("reports") + reports.mkdir(parents=True, exist_ok=True) + path_2a = reports / f"earnings-2a-gap-{stamp}.json" + path_2b = reports / f"earnings-2b-sue-{stamp}.json" + common = { + "generated_at": generated_at, + "snapshot": str(snapshot.resolve()), + "snapshot_depth": depth, + "data_quality": quality, + "production_impact": "none", + } + payload_2a = {**common, "experiment": "2a", "result": result_2a} + payload_2b = {**common, "experiment": "2b", "result": result_2b} + path_2a.write_text( + json.dumps(payload_2a, indent=2, default=str) + "\n", encoding="utf-8" + ) + path_2b.write_text( + json.dumps(payload_2b, indent=2, default=str) + "\n", encoding="utf-8" + ) + path_2a.with_suffix(".md").write_text( + "# Earnings Task 2a - gap diagnostic\n\n" + + _quality_markdown(quality, depth) + + "\n\n" + + _two_a_markdown(result_2a) + + "\n", + encoding="utf-8", + ) + path_2b.with_suffix(".md").write_text( + "# Earnings Task 2b - SUE / PEAD\n\n" + + _quality_markdown(quality, depth) + + "\n\n" + + _two_b_markdown(result_2b) + + "\n", + encoding="utf-8", + ) + return path_2a, path_2b + + +def _update_research_doc( + *, + depth: dict[str, Any], + quality: dict[str, Any], + result_2a: dict[str, Any], + result_2b: dict[str, Any], + path_2a: Path, + path_2b: Path, +) -> None: + path = Path("docs/research/earnings-gap-and-sue.md") + existing = path.read_text(encoding="utf-8") if path.exists() else "# Earnings gap and SUE" + marker = "## Results" + index = existing.find(marker) + preregistration = existing[:index].rstrip() if index >= 0 else existing.rstrip() + final_status = ( + "Task 2 CLOSED (SUE PASS→PENDING_HUMAN)" + if result_2b["verdict"] == "PASS" + else "Task 2 CLOSED (SUE DEAD)" + ) + body = [ + preregistration, "", "## Results", "", - f"Generated: `{payload.get('generated_at')}`", + _quality_markdown(quality, depth), "", - "### Data provenance", + _two_a_markdown(result_2a), "", - f"```json\n{json.dumps(payload.get('data_provenance') or {}, indent=2, default=str)}\n```", + _two_b_markdown(result_2b), "", - "### 2a — Earnings-gap risk (report-only)", + "## Artifacts", + "", + f"- `{path_2a.as_posix()}` and companion Markdown", + f"- `{path_2b.as_posix()}` and companion Markdown", + "- `reports/earnings-backfill-status.json`", + "", + "Production changes: **none**. No earnings filter or SUE integration was implemented.", + "", + f"## Final status: **{final_status}**", "", ] - a = payload.get("experiment_2a") - if not a: - lines.append("_Skipped or unavailable._") - else: - lines.append(f"```json\n{json.dumps(a, indent=2, default=str)}\n```") - lines.extend(["", "### 2b — SUE / PEAD IC", ""]) - b = payload.get("experiment_2b") - if not b: - lines.append("_Skipped or unavailable._") - else: - side = b.get("signal_eval_side_by_side") or {} - lines.extend([ - "| signal | mean_ic | ic_t_stat | weeks | avg_N | reliable |", - "|---|---:|---:|---:|---:|---|", - ]) - for name in ( - "mom_12_1", - "mom_12_1_resid", - "sue_latest", - "fip_id", - ): - r = side.get(name) or {} - lines.append( - f"| {name} | {r.get('mean_ic', '')} | {r.get('ic_t_stat', '')} | " - f"{r.get('weeks', '')} | {r.get('avg_cross_section', '')} | " - f"{r.get('reliable', '')} |" - ) - lines.extend([ - "", - f"**SUE grade:** `{json.dumps(b.get('sue_grade') or {}, default=str)}`", - "", - f"**Momentum-conditional SUE:** `{json.dumps(b.get('momentum_conditional_sue') or {}, default=str)}`", - "", - ]) - - lines.extend([ - "", - "## Verdict", - "", - f"**{payload.get('verdict')}**", - "", - payload.get("verdict_detail") or "", - "", - "## What a human must decide next", - "", - payload.get("human_next") or "- Review; no auto-ship.", - "", - f"Artifacts: `{payload.get('report_path')}`", - "", - ]) - path.write_text("\n".join(lines) + "\n", encoding="utf-8") + path.write_text("\n".join(body), encoding="utf-8") async def _main() -> None: args = _parse_args() snapshot = Path(args.snapshot) - if not snapshot.exists(): - raise SystemExit(f"Missing snapshot {snapshot}") + universe_snapshot = Path(args.universe_snapshot) + earnings_snapshot = Path(args.earnings_snapshot) + for path in (snapshot, universe_snapshot, earnings_snapshot): + if not path.exists(): + raise SystemExit(f"Missing snapshot: {path}") if args.allow_spawn: os.environ["BACKTEST_ALLOW_SPAWN"] = "1" - events, meta = _load_earnings(snapshot) - # Race guard lite on earnings completeness. - provenance = { - "snapshot": str(snapshot.resolve()), - "n_earnings_events": len(events), - "backfill_meta": meta, - "announce_range": { - "min": min((e["announce_date"] for e in events), default=None), - "max": max((e["announce_date"] for e in events), default=None), - }, - "with_actual_and_estimate": sum( - 1 - for e in events - if e.get("eps_actual") is not None and e.get("eps_estimate") is not None - ), - } + requested_symbols = set(_read_symbols(universe_snapshot)) + depth = _snapshot_depth(snapshot, requested_symbols) print( - f"Earnings events: {provenance['n_earnings_events']} " - f"(with act+est={provenance['with_actual_and_estimate']}) meta={meta}" - ) - if meta and meta.get("done", 0) < 0.9 * (meta.get("universe_tickers") or 1): - print( - "WARNING: earnings backfill incomplete " - f"({meta.get('done')}/{meta.get('universe_tickers')}). " - "Results may be biased; resume backfill." - ) - - exp_2a = None - exp_2b = None - if not args.skip_2a: - print("Running 2a earnings-gap diagnostic…") - exp_2a = await _run_2a( - snapshot, events, quiet=args.quiet, workers=args.workers - ) - print( - " 2a losses<-1R with earnings:", - (exp_2a.get("q1_losses_worse_than_minus_1r") or {}), - ) - if not args.skip_2b: - print("Running 2b SUE IC harness…") - exp_2b = await _run_2b_ic( - snapshot, events, quiet=args.quiet, workers=args.workers - ) - g = exp_2b.get("sue_grade") or {} - print(f" 2b SUE green={g.get('green')} {g.get('reason')}") - - # Verdict - if exp_2b and (exp_2b.get("sue_grade") or {}).get("green"): - verdict = "PROMOTE (2b SUE) — STOP for human wire design" - detail = ( - "SUE cleared iron rule. No book integration without human approval. " - "2a remains report-only." - ) - human = ( - "- Design tilt vs second gate if desired.\n" - "- Do not auto-filter from 2a without separate approval + tail review." - ) - else: - sue_ic = None - if exp_2b: - sue_ic = ((exp_2b.get("sue_grade") or {}).get("row") or {}).get("mean_ic") - if sue_ic is not None and abs(float(sue_ic)) >= 0.015: - verdict = "PARK" - detail = f"SUE IC={sue_ic} below iron bar or unreliable; keep data, no wire." - else: - verdict = "DEAD (2b) / REPORT-ONLY (2a)" - detail = ( - "SUE does not clear iron rule on this window. " - "2a distributions for human risk review only — no filter." - ) - human = ( - "- No SUE book change.\n" - "- Read 2a tails before considering any earnings-avoid filter." - ) - - stamp = datetime.now().strftime("%Y%m%d-%H%M%S") - out = Path(args.out) if args.out else Path("reports") / f"earnings-gap-sue-{stamp}.json" - payload = { - "generated_at": datetime.now().isoformat(), - "data_provenance": provenance, - "experiment_2a": exp_2a, - "experiment_2b": exp_2b, - "verdict": verdict, - "verdict_detail": detail, - "human_next": human, - "report_path": str(out.as_posix()), - "fmp_note": ( - "Bulk earnings-calendar is paid (402 on free tier). " - "Backfill used per-symbol /stable/earnings; see earnings-backfill-status.json." + "Snapshot guard:", + json.dumps( + { + "manifest_complete": depth["manifest"].get("complete"), + "tradable": depth["tradable_symbols"], + "pre2021_pct": depth["symbols_with_pre2021_bars_pct"], + "benchmark": depth["benchmark_spy"], + "gate_pass": depth["gate_pass"], + }, + default=str, ), - } - out.parent.mkdir(parents=True, exist_ok=True) - out.write_text(json.dumps(payload, indent=2, default=str) + "\n", encoding="utf-8") - md = Path("docs/research/earnings-gap-and-sue.md") - _write_md(md, payload) - out.with_suffix(".md").write_text(md.read_text(encoding="utf-8"), encoding="utf-8") - print(f"Verdict: {verdict}") - print(f"Wrote {out}") + ) + if not depth["gate_pass"]: + raise SystemExit( + "Snapshot depth gate failed. Repair the production-universe depth and " + "SPY benchmark before running either experiment." + ) + tradable_symbols = requested_symbols - set(depth["missing_symbols"]) - set( + depth["zero_bar_symbols"] + ) + + backfill_status = _load_backfill_status() + price_start = date.fromisoformat(depth["price_window"]["min"]) + price_end = date.fromisoformat(depth["price_window"]["max"]) + backfill_window = backfill_status.get("window") or {} + coverage_start = ( + date.fromisoformat(backfill_window["from"]) + if backfill_window.get("from") + else None + ) + coverage_end = ( + date.fromisoformat(backfill_window["to"]) + if backfill_window.get("to") + else None + ) + approved_shorter_window = bool( + backfill_status.get("mode") == "dolthub_public_bulk_clone" + and (backfill_status.get("coverage_amendment") or {}).get( + "approved_by_user" + ) + ) + backfill_covers_approved_window = bool( + coverage_start + and coverage_end + and coverage_end >= price_end + and (coverage_start <= price_start or approved_shorter_window) + ) + allowed_modes = {"fmp_bulk_date_range_only", "dolthub_public_bulk_clone"} + if ( + backfill_status.get("mode") not in allowed_modes + or not backfill_status.get("complete") + or not backfill_covers_approved_window + ): + raise SystemExit( + "Bulk earnings backfill is incomplete or does not cover its approved " + "research window through the price snapshot end." + ) + if coverage_start is None or coverage_end is None: + raise SystemExit("Backfill status is missing its approved coverage window.") + analysis_start = max(price_start, coverage_start) + analysis_end = min(price_end, coverage_end) + + events = [ + event + for event in _load_earnings(earnings_snapshot, tradable_symbols) + if analysis_start <= event["announce_date"] <= analysis_end + ] + surprise_history = _load_surprise_history( + earnings_snapshot, tradable_symbols + ) + quality = _data_quality( + events, + tradable_symbols, + window_start=analysis_start, + window_end=analysis_end, + backfill_status=backfill_status, + ) + print( + "Earnings quality:", + json.dumps( + { + "events": quality["events"], + "symbols_ge8_pct": quality["symbols_with_ge8_announcements_pct"], + "paired_pct": quality["events_with_actual_and_estimate_pct"], + "session": quality["announcement_session"], + "fallback": quality["sue_scaling"]["fallback_name"], + } + ), + ) + + stamp = args.stamp or datetime.now().strftime("%Y%m%d-%H%M%S") + generated_at = datetime.now(timezone.utc).isoformat() + reports = Path("reports") + reports.mkdir(parents=True, exist_ok=True) + print("Running Experiment 2a...") + result_2a = await _run_2a( + snapshot, + events, + tradable_symbols, + analysis_start=analysis_start, + analysis_end=analysis_end, + workers=args.workers, + quiet=args.quiet, + ) + checkpoint_2a = reports / f"earnings-2a-gap-{stamp}.checkpoint.json" + checkpoint_2a.write_text( + json.dumps( + { + "generated_at": generated_at, + "snapshot_depth": depth, + "data_quality": quality, + "result": result_2a, + }, + indent=2, + default=str, + ) + + "\n", + encoding="utf-8", + ) + print(f"Wrote 2a checkpoint: {checkpoint_2a}", flush=True) + print("Running Experiment 2b...") + result_2b = await _run_2b( + snapshot, + events, + surprise_history, + tradable_symbols, + quality=quality, + workers=args.workers, + quiet=args.quiet, + ) + checkpoint_2b = reports / f"earnings-2b-sue-{stamp}.checkpoint.json" + checkpoint_2b.write_text( + json.dumps( + { + "generated_at": generated_at, + "snapshot_depth": depth, + "data_quality": quality, + "result": result_2b, + }, + indent=2, + default=str, + ) + + "\n", + encoding="utf-8", + ) + print(f"Wrote 2b checkpoint: {checkpoint_2b}", flush=True) + path_2a, path_2b = _write_reports( + stamp=stamp, + generated_at=generated_at, + snapshot=snapshot, + depth=depth, + quality=quality, + result_2a=result_2a, + result_2b=result_2b, + ) + _update_research_doc( + depth=depth, + quality=quality, + result_2a=result_2a, + result_2b=result_2b, + path_2a=path_2a, + path_2b=path_2b, + ) + print(f"2a verdict: {result_2a['verdict']}") + print(f"2b verdict: {result_2b['verdict']} - {result_2b['verdict_detail']}") + print(f"Wrote {path_2a} and {path_2b} (+ Markdown companions)") if __name__ == "__main__": diff --git a/scripts/run_tier1_macbook.sh b/scripts/run_tier1_macbook.sh index 3254e4e..26c3b18 100755 --- a/scripts/run_tier1_macbook.sh +++ b/scripts/run_tier1_macbook.sh @@ -120,12 +120,14 @@ case "$PHASE" in ssl) ssl_check ;; earnings) need_file "$PROD_SNAP" - log "Earnings backfill + research (parked experiment)" + need_file "$RESEARCH_SNAP" + log "Earnings Task 2 bulk backfill + registered 2a/2b closeout" "$PYTHON" scripts/backfill_earnings_events.py \ - --snapshot "$PROD_SNAP" --provider fmp --force-symbol \ + --snapshot "$PROD_SNAP" --from-date 2016-01-04 --window-days 30 \ --limit "$FMP_LIMIT" --sleep "$FMP_SLEEP" "$PYTHON" scripts/run_earnings_research.py \ - --snapshot "$PROD_SNAP" --workers "$WORKERS" --allow-spawn + --snapshot "$RESEARCH_SNAP" --universe-snapshot "$PROD_SNAP" \ + --earnings-snapshot "$PROD_SNAP" --workers "$WORKERS" --allow-spawn ;; prod_book) need_file "$RESEARCH_SNAP" diff --git a/tests/unit/test_earnings_research.py b/tests/unit/test_earnings_research.py new file mode 100644 index 0000000..cfff0b3 --- /dev/null +++ b/tests/unit/test_earnings_research.py @@ -0,0 +1,225 @@ +from datetime import date, timedelta + +from scripts.backfill_earnings_events import _dedupe_bulk_rows, _windows +from scripts.import_dolthub_earnings import _align_symbol +from scripts.run_earnings_research import ( + _analyse_2a_trades, + _build_sue_series, + _mechanical_sue_grade, +) + + +def test_dolthub_alignment_is_monotonic_across_close_calendar_events() -> None: + events = [ + {"announce_date": date(2020, 3, 17), "announce_time": "bmo"}, + {"announce_date": date(2020, 4, 30), "announce_time": "bmo"}, + ] + periods = [ + {"period_end_date": date(2019, 12, 31)}, + {"period_end_date": date(2020, 3, 31)}, + ] + matches, unmatched_events, unmatched_periods = _align_symbol( + events, periods, max_lag_days=90, max_lead_days=14 + ) + assert matches == [(0, 0), (1, 1)] + assert unmatched_events == [] + assert unmatched_periods == [] + + +def test_dolthub_alignment_allows_fiscal_period_label_after_announcement() -> None: + events = [ + {"announce_date": date(2023, 2, 28), "announce_time": "bmo"}, + {"announce_date": date(2023, 5, 23), "announce_time": "bmo"}, + ] + periods = [ + {"period_end_date": date(2023, 2, 28)}, + {"period_end_date": date(2023, 5, 31)}, + ] + matches, _, _ = _align_symbol( + events, periods, max_lag_days=90, max_lead_days=14 + ) + assert matches == [(0, 0), (1, 1)] + + +def test_bulk_windows_cover_range_without_overlap() -> None: + result = _windows(date(2020, 1, 1), date(2020, 1, 10), 4) + assert result == [ + (date(2020, 1, 1), date(2020, 1, 4)), + (date(2020, 1, 5), date(2020, 1, 8)), + (date(2020, 1, 9), date(2020, 1, 10)), + ] + + +def test_bulk_dedupe_prefers_more_complete_and_counts_restatement() -> None: + rows = [ + { + "symbol": "AAPL", + "announce_date": "2024-01-01", + "announce_time": None, + "eps_estimate": 1.0, + "eps_actual": 1.1, + "revenue_estimate": None, + "revenue_actual": None, + }, + { + "symbol": "AAPL", + "announce_date": "2024-01-01", + "announce_time": "amc", + "eps_estimate": 1.0, + "eps_actual": 1.2, + "revenue_estimate": 10.0, + "revenue_actual": 11.0, + }, + ] + deduped, duplicates, restated = _dedupe_bulk_rows(rows) + assert duplicates == 1 + assert restated == 1 + assert deduped == [rows[1]] + + +def test_2a_uses_net_r_strict_hold_and_next_session_stop() -> None: + calendar = [ + date(2024, 1, 2), + date(2024, 1, 3), + date(2024, 1, 4), + date(2024, 1, 5), + date(2024, 1, 8), + date(2024, 1, 9), + ] + events = [ + { + "symbol": "AAPL", + "announce_date": date(2024, 1, 5), + } + ] + trades = [ + { + "symbol": "AAPL", + "entry_date": "2024-01-03", + "exit_date": "2024-01-08", + "entry": 100.0, + "initial_stop": 90.0, + "fill": 90.0, + "r": -1.0, + "reason": "stop", + }, + { + "symbol": "MSFT", + "entry_date": "2024-01-02", + "exit_date": "2024-01-09", + "entry": 100.0, + "initial_stop": 90.0, + "fill": 110.0, + "r": 1.0, + "reason": "time", + }, + ] + result = _analyse_2a_trades( + trades, events, calendar, cost_per_side=0.001 + ) + assert result["q1_loss_concentration"]["losses_count"] == 1 + assert result["q1_loss_concentration"]["losses_with_announcement_count"] == 1 + assert ( + result["q2_entries_within_3_trading_days_before_announcement"][ + "pre_earnings" + ]["count"] + == 1 + ) + assert ( + result["q3_stop_exits_within_1_trading_day_after_announcement"][ + "stops_after_earnings" + ]["count"] + == 1 + ) + assert result["q1_loss_concentration"]["loss_definition"] == ( + "realized_net_R <= -1.0" + ) + + +def test_sue_needs_four_prior_surprises_and_starts_next_trading_day() -> None: + dates = [date(2024, 1, 1) + timedelta(days=index) for index in range(100)] + columns = ( + [value.toordinal() for value in dates], + [100.0] * len(dates), + [101.0] * len(dates), + [99.0] * len(dates), + [100.0] * len(dates), + [1_000_000] * len(dates), + ) + event_dates = [date(2024, 1, 2) + timedelta(days=10 * index) for index in range(5)] + surprises = [0.1, -0.2, 0.3, -0.1, 0.4] + events = { + "AAPL": [ + { + "announce_date": event_date, + "eps_actual": 1.0 + surprise, + "eps_estimate": 1.0, + } + for event_date, surprise in zip(event_dates, surprises) + ] + } + series, counts = _build_sue_series( + events, {"AAPL": columns}, use_price_fallback=False + ) + first_live = event_dates[-1] + timedelta(days=1) + assert first_live in series["AAPL"] + assert event_dates[-1] not in series["AAPL"] + assert counts["standard_scaled_events"] == 1 + assert counts["price_fallback_events"] == 0 + + +def test_sue_uses_period_history_only_for_scaling() -> None: + dates = [date(2020, 1, 1) + timedelta(days=index) for index in range(100)] + columns = ( + [value.toordinal() for value in dates], + [100.0] * len(dates), + [101.0] * len(dates), + [99.0] * len(dates), + [100.0] * len(dates), + [1_000_000] * len(dates), + ) + event_date = date(2020, 2, 3) + events = { + "AAPL": [ + { + "announce_date": event_date, + "period_end_date": date(2019, 12, 31), + "eps_actual": 1.4, + "eps_estimate": 1.0, + } + ] + } + history = { + "AAPL": [ + { + "period_end_date": date(2018, 12, 31) + + timedelta(days=90 * index), + "eps_actual": 1.0 + surprise, + "eps_estimate": 1.0, + } + for index, surprise in enumerate([0.1, -0.2, 0.3, -0.1]) + ] + } + series, counts = _build_sue_series( + events, + {"AAPL": columns}, + use_price_fallback=False, + surprise_history_by_symbol=history, + ) + assert event_date + timedelta(days=1) in series["AAPL"] + assert event_date not in series["AAPL"] + assert counts["events_scaled_from_period_history"] == 1 + + +def test_sue_grade_requires_positive_both_eras() -> None: + full = {"mean_ic": 0.03, "reliable": True} + passed, stable = _mechanical_sue_grade( + full, {"mean_ic": 0.01}, {"mean_ic": 0.02} + ) + assert passed is True + assert stable is True + failed, stable = _mechanical_sue_grade( + full, {"mean_ic": -0.01}, {"mean_ic": 0.02} + ) + assert failed is False + assert stable is False