From e09ec4ab1db2d2838fa7e01eb22bc8a5fc625ee9 Mon Sep 17 00:00:00 2001 From: Dennis Thiessen Date: Tue, 21 Jul 2026 21:48:05 +0200 Subject: [PATCH] docs: add Dolt integration plan + review clarifications MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the Dolt bulk-data integration hand-off plan (authored in prior session) and applies clarifications found in a pre-handoff review: - step (c): fields refresh from three distinct sources, not "snapshots + close" — earnings_surprise/next_earnings_date come from earnings_events, not SEC facts. pe_ratio = close / TTM diluted EPS and market_cap = issuer-wide diluted shares × close stated as separate formulas. - fundamental_snapshots made implementable: add period_start, fiscal_year, fiscal_period; duration facts store the filing's cumulative YTD/FY values, balance-sheet facts store period-end values. Discrete quarters, Q4, TTM and YoY are derived at read time (correct for non-calendar fiscal years; amendments never freeze a stale derived quarter). - flag Q4 derivation / fiscal-period alignment as the primary A3 risk. - anchor "tracked universe" to ticker_universe_service + per-run CIK resolution. All code anchors in the doc verified accurate against the current tree. Co-Authored-By: Claude Opus 4.8 --- docs/dolt-integration-plan.md | 464 ++++++++++++++++++++++++++++++++++ 1 file changed, 464 insertions(+) create mode 100644 docs/dolt-integration-plan.md diff --git a/docs/dolt-integration-plan.md b/docs/dolt-integration-plan.md new file mode 100644 index 0000000..abf477e --- /dev/null +++ b/docs/dolt-integration-plan.md @@ -0,0 +1,464 @@ +# Dolt bulk-data integration — implementation plan + +Status: approved 2026-07-21, revised through four review rounds; direction: KISS +backend, UI value first. Hand-off document for the implementing agent; +self-contained. + +## Objective + +Replace the free-tier fundamentals APIs (FMP, Finnhub, Alpha Vantage) with bulk +data: SEC Company Facts for fundamentals, the DoltHub earnings repo for the +earnings calendar/history, and — later, independently — the DoltHub stocks repo for +historical OHLCV. PostgreSQL stays the production system of record. + +**Delivery order: two independent workstreams.** + +- **Workstream A (build first):** SEC fundamentals + Dolt earnings + API v1 + + FundamentalsPanel + decommission FMP/Finnhub/Alpha Vantage. Valuation uses the + existing Alpaca closes already in `ohlcv_records`. This alone achieves the goal + (killing the quota-limited APIs) and delivers all the UI value. +- **Workstream B (later, optional until needed):** replace historical OHLCV with + the Dolt stocks repo. The most complex machinery (4.7 GB clone, split + adjustment, source-bar table, reconciliation) lives here and blocks nothing in A. + +**Guiding principle: KISS.** Plain daily importers with staging and atomic +promotion — no forensic replay, no permanent archive store, no conflict tables, no +aggregate tables. Engineering budget goes into the UI (quarter trends, peer +comparison). Deferred until a concrete need: exact source replay, point-in-time +backtest enforcement, fundamental metrics in scoring. + +**Non-negotiables** + +- The application never queries Dolt/DoltHub or SEC at request time. All access is + batch import → PostgreSQL. If a sync fails or the source is unchanged, production + continues on the last successfully imported data. +- Do not replace PostgreSQL with Dolt/Doltgres. Never commit to the upstream clones. +- No owned SEC Dolt repo: SEC JSON is normalized straight into PostgreSQL. +- Scoring **code** is unchanged, but swapping the data source changes production + behavior: `app/services/scoring_service.py` (~line 450) scores pe_ratio / + revenue_growth / earnings_surprise from `fundamental_data`, so new definitions + change rankings even with identical code. Cutover of `fundamental_data` + population requires the **score-parity gate** (phase A5) — never silently. All + *new* metrics are display-only. +- Intraday (10:00–15:00), near-close (15:30) and after-close (16:45) pipelines stay + on Alpaca unchanged. + +## Data sources + +1. **SEC Company Facts + submissions bulk files** (free, no key; costs bandwidth, + CPU and disk — optimize accordingly) — XBRL facts per **issuer (CIK), not per + ticker**. Tickers resolve to a CIK via SEC `company_tickers.json` (multi-class + issuers like GOOGL/GOOG share one CIK and one set of fundamentals). Submissions + also supply the SIC code (peer grouping) and `acceptanceDateTime`. Handle unit + variants and fiscal-period alignment (derive Q4 = FY − Q1..Q3 where needed). + **Amendments:** retain every accession immutably; readers select the newest + valid `accepted_at` snapshot per reporting period at read time. No flags, no + mutation. +2. **`post-no-preference/earnings`** (DoltHub) — announcement date, BMO/AMC session + (partial), period end, EPS estimate/actual, surprise history. Small clone. + `scripts/import_dolthub_earnings.py` is a research/SQLite importer — reuse its + normalization/alignment logic (calendar↔EPS-history monotonic alignment, SUE + scaling) but write a production PostgreSQL importer; do not extend the script. +3. **`post-no-preference/stocks`** (DoltHub, **workstream B**) — daily raw OHLCV + (unadjusted), symbol metadata, splits, dividends. Publishes ~01:30 ET the + following calendar day. Clone is ~4.7 GB. + +**Licensing (phase A0):** verify the DoltHub repos' CC-BY-SA 4.0 terms are +compatible before building. Internal, non-redistributed use is expected to be fine; +record the conclusion in this doc. + +## Schema + +**Migration 026 (workstream A)** — current head: `025_trade_setup_scan_run_id`: + +- `data_import_runs` — lean: id, source (`sec_facts` | `dolt_earnings` | + `dolt_stocks`), revision (Dolt commit hash, or SEC archive SHA-256), status + (`running`/`validated`/`promoted`/`no_op`/`failed`), source_max_date, row_counts + JSON, validation JSON (includes reconciliation/discrepancy summaries — no + separate conflicts table; details go to structured logs), started_at, + completed_at, error_details. One run per source at a time (Postgres advisory + lock keyed by source). +- `earnings_events` — ticker_id, announce_date, session (`bmo`/`amc`/`unknown`), + period_end, eps_estimate, eps_actual, source, import_run_id. Unique + (ticker_id, announce_date). **Rescheduling:** within each promotion transaction, + delete this source's future-dated rows (announce_date > today) and re-insert + from the new snapshot, so moved or cancelled dates never linger. Past rows + (results) are never deleted. +- `tickers` — add nullable `cik`, `sic`, `sic_description` (from SEC submissions / + `company_tickers.json`; refreshed by the SEC import; multi-class tickers share + values). The only ticker↔issuer join point. +- `fundamental_snapshots` — **CIK-keyed, one immutable row per accession**: cik, + accession (unique), form, filed_date, **accepted_at** (kept although PIT + enforcement is deferred — one timestamp now vs painful retrofit later), + **period_start, period_end, fiscal_year, fiscal_period** (the filing's own + `dei`/`us-gaap` period identity — required to align non-calendar fiscal years and + to derive discrete quarters from cumulative facts), and the **price-independent + raw facts** so metrics are recomputable. **Store facts as the filing reports + them, not as derived quarters:** duration facts (revenue, net income, diluted EPS, + CFO, capex, EBITDA inputs) retain the filing's normalized **cumulative YTD/FY** + value for the (period_start → period_end) span; balance-sheet facts (cash+ST + investments, total debt, diluted shares) are **period-end** values. **Nothing + derived is frozen into a row:** discrete quarters (10-Q YTD deltas, Q4 = FY − + Q1..Q3), TTM, YoY and the quarter-tape series are all computed **at read time** by + picking the newest valid accepted_at snapshot for *each* required period — so + non-calendar fiscal years resolve correctly and a later amendment to a prior + quarter is reflected automatically without ever storing a stale derived quarter. + Readers pick the newest valid accepted_at per period; history powers the UI + quarter tape. +- Keep `fundamental_data` (`app/models/fundamental.py`) as the latest-value compat + cache, repopulated by the daily SEC job — but only after the phase-A5 parity + gate. + +**Migration 027 (workstream B, written when B starts):** + +- `ohlcv_source_bars` — source-truth bar table, required because `ohlcv_records` + allows one row per (ticker_id, date) (`app/models/ohlcv.py:12`) and Alpaca + ingestion upserts it in place (`app/services/price_service.py:82`) — Dolt and + Alpaca bars cannot coexist there. Holds **Dolt raw (unadjusted) bars only** — + Alpaca bars are already split-adjusted at the provider (`app/providers/alpaca.py:77` + requests `Adjustment.SPLIT`) and live exclusively in `ohlcv_records`. Columns: + source (`dolt`), adjustment (`raw` — explicit), ticker_id, date, OHLCV, + import_run_id; unique (source, ticker_id, date). Changed bars are counted in the + run's validation JSON and logged before overwrite. +- `corporate_actions` — ticker_id, type (`split`/`dividend`), ex_date, + ratio/amount, source, import_run_id. Unique (ticker_id, type, ex_date). +- `ohlcv_records` — add nullable `import_run_id` FK and `source` text (default + `'alpaca'`). + +## Import framework + +Every importer: idempotent per revision (same Dolt commit / archive checksum → +`no_op`, zero row changes); staging tables first; promotion in one transaction; +safe to retry; a failed or unchanged run leaves the current dataset untouched. +Record every attempt in `data_import_runs`. + +**SEC access requirements (operational safeguards, per SEC fair-access policy):** +send an identifying `User-Agent` with a contact email on every request; stay far +below the 10 req/s limit (the bulk endpoints need only a handful of requests per +run); exponential backoff on 429; a 403 means the User-Agent or request pattern is +wrong — alert and stop, never retry-loop. See SEC developer resources +(https://www.sec.gov/about/developer-resources). + +**Reproducibility scope (deliberately limited):** the normalized snapshots in +PostgreSQL *are* the durable record. Keep only the last ~2 SEC archives on disk for +debugging. Byte-level replay of old runs is out of scope until a concrete need. + +Dolt access: `dolt pull` on the persistent clone, record the resulting commit hash, +read via `dolt sql -r csv` (no long-running sql-server). **The scheduler shares one +event loop with the API** (`app/scheduler.py:73`) — run dolt/unzip/download +subprocesses via `asyncio.create_subprocess_exec` (or an executor), never blocking +calls. Check free disk space before pulling; alert and skip if below threshold. + +**Deployment constraints:** deploy is `rsync --delete` of the repo tree +(`.gitea/workflows/deploy.yml:127`), so clones and archives must live **outside the +deployment path** — an env-configured persistent directory (e.g. +`DOLT_DATA_DIR=/var/lib/signal-platform/dolt`), backed up. The dolt binary is a new +prod runtime dependency: install with a pinned version in the deploy workflow. + +**Validation gates (block promotion, raise an alert via the existing system-events +path):** source freshness as expected; tracked-universe coverage; no duplicate +business keys; fundamental units/periods consistent; row-count deltas within +reason; upstream schema change stops promotion. Workstream B adds: OHLC sanity +(high ≥ open/close/low, low ≤ open/close/high, volume ≥ 0); no unexplained split +discontinuities. + +**Split adjustment (workstream B):** `ohlcv_source_bars` + `corporate_actions` are +the source of truth; canonical `ohlcv_records` is *generated* from them to match +Alpaca `Adjustment.SPLIT`, selecting only adjustment = `raw` rows as input so +adjustment is applied exactly once. A newly published split rewrites the symbol's +entire adjusted history — treat whole-symbol rewrites as a normal import event +(exempt that symbol from the row-count-delta gate for that run) and stamp rows with +the import_run_id (a backtest↔prod parity guard exists; changed history changes +backtests). + +## Scheduling (`app/scheduler.py` — `SCHEDULE_DEFAULTS` / `_CRON_JOBS`, ~line 1451) + +Follow the existing pattern: cron strings in SystemSettings via +`app/services/settings_store.py`, day-of-week as names never numbers, logging via +`_log_event`. + +Workstream A: + +- Dolt earnings import: daily ~02:30 ET (with the future-row replacement above). +- **SEC fundamentals job: daily ~04:00 ET.** One job, three steps: + (a) freshness check **using conditional HTTP metadata (ETag/Last-Modified / + If-None-Match) before downloading** — an unchanged archive is a `no_op` without + pulling the multi-GB file; verify by SHA-256 after any actual download; + (b) when changed, **parse and write only tracked-universe CIKs** through + staging→promotion (the tracked universe is `ticker_universe_service`'s set; + CIKs are resolved from `company_tickers.json` each run, so a newly added ticker + self-resolves on its next SEC run — until then its metrics are simply null); + (c) **always, locally, and only after production activation** (the phase-A5 + parity approval): refresh the legacy `fundamental_data` fields and mark affected + cached fundamental scores stale. **Sources differ per field** — do not assume all + five come from SEC: `pe_ratio` and `market_cap` from the newest valid snapshots × + latest PostgreSQL close, each with its own formula — `pe_ratio` = latest close / + TTM diluted EPS; `market_cap` = issuer-wide diluted shares × latest close; + `revenue_growth` from the snapshots alone; `earnings_surprise` and + `next_earnings_date` from `earnings_events` (the Dolt earnings feed — these two do + not exist in SEC facts). Before activation the job imports snapshots only + (shadow). Step (c) must run identically when SEC is unreachable — prices move + daily even when filings don't, and the earnings-derived fields already live in + PostgreSQL. + **The new API valuation object is not stored anywhere** — it is computed at + request time (below). No valuation cache or table exists. + +Workstream B: + +- Dolt OHLCV+splits pull/import: `0 2 * * tue-sat` ET. If source_max_date is not + fresh, retry hourly until ~06:00, then give up quietly. After a successful + import, reconcile the previous session's Dolt-derived bars against the Alpaca + bars; summary into the run's validation JSON, details to logs. +- Move `schedule_daily_pipeline_cron` (morning refresh) from `0 2 * * *` to + `0 3 * * *` (only needed once the 02:00 slot is taken by the OHLCV pull). + **Late Dolt publication is a non-event:** the canonical scan runs at 15:30 on + Alpaca, so the morning pipeline runs normally even when the import hasn't + landed — no gating, no defensive coupling. + +## Metrics catalog (curated — TTM basis) + +Snapshots store **price-independent per-period facts** (the "snapshot" column below +means *derived from stored snapshots, assembled across periods at read time* — see +Schema — not frozen at import); price-dependent ratios are never frozen into +snapshots and have **no storage location at all**: the API computes +them at request time from the stored snapshots + the latest `ohlcv_records` close +(both already in PostgreSQL, so this works identically when SEC is unreachable). +The only stored price-dependent values are the legacy `fundamental_data` fields +that scoring already reads, refreshed daily by step (c) after activation. + +| Metric | Definition | Where computed | +|---|---|---| +| Revenue growth YoY | TTM revenue vs prior TTM | snapshot | +| EPS growth YoY | TTM diluted EPS vs prior TTM | snapshot | +| Operating margin + 4q trend | TTM operating income / revenue | snapshot | +| FCF margin | (TTM CFO − capex) / revenue | snapshot | +| Net cash / net debt | cash + ST investments − total debt | snapshot | +| Net debt / EBITDA | net debt / TTM EBITDA | snapshot | +| Share count Δ YoY | diluted shares vs year ago | snapshot | +| Trailing P/E | price / TTM diluted EPS | request time | +| FCF yield | TTM FCF / est. market cap | request time | +| Est. market cap | issuer-wide diluted shares × ticker price | request time | +| Earnings surprise history | last 4+ from `earnings_events` | query | + +**Market cap is an estimate** (issuer-wide diluted shares × one ticker's price — +approximate for multi-class issuers). Label it "est." in the UI and round +aggressively rather than withholding it; false precision is the failure mode, not +the approximation. + +**Units follow existing app conventions:** percentages are percentage points +(21.0 = 21%), P/E and net-debt/EBITDA are multiples, market cap and net debt are +dollars. + +Deliberately **excluded**: ROIC (invested-capital/NOPAT normalization too noisy), +gross margin (COGS tagging too inconsistent), any new composite score. + +## Peer comparison (read-time only) + +- Peer group = tracked-universe issuers sharing the **first two SIC digits**, + **deduplicated by CIK** — GOOG and GOOGL are one issuer, one observation, in + medians, percentiles and peer_count. +- Computed at read time from current snapshots — no aggregate tables until + performance demonstrates a need. +- Medians exclude null/invalid values. **Fewer than 5 valid peer issuers → omit + the peer result entirely** rather than showing a misleading universe comparison. +- Percentile direction respects metric polarity (higher-is-better for FCF yield, + lower-is-better for P/E and leverage). + +## API contract (additive v1) + +Every existing top-level field is preserved unchanged (name, type, position) — +backend and frontend ship independently, no breaking interval. New objects, exact +names and types: + +```jsonc +{ + // ...all existing legacy fields, unchanged... + "earnings": { + "next": {"date": "YYYY-MM-DD", "session": "bmo|amc|unknown", "days_until": 12} | null, + "recent": [ // newest first, max 4, may be empty + {"announce_date": "YYYY-MM-DD", "period_end": "YYYY-MM-DD|null", + "eps_estimate": 1.02|null, "eps_actual": 1.10|null, "surprise_pct": 7.8|null} + ] + }, + "metrics": [ // fixed row set — every key always present, value null when unavailable + { + "key": "revenue_growth_yoy", // revenue_growth_yoy | eps_growth_yoy | operating_margin | + // fcf_margin | net_debt | net_debt_to_ebitda | share_count_change_yoy + "value": 18.0, // number | null — pp / multiples / dollars per units above + "history": [ // oldest→newest, max 4 points, [] when unavailable + {"period_end": "YYYY-MM-DD", "value": 8.0} + ], + "industry": { // object | null — null when < 5 valid peer issuers (CIK-deduped) + "label": "SIC 73 peers", // truthful 2-digit group label — grouping IS 2-digit, + // so no 4-digit description like "Prepackaged Software" + "median": 11.0, + "favorable_percentile": 82, // 0-100, polarity-aware (higher = more favorable) + "peer_count": 12 // issuers, not tickers + }, + "period_end": "YYYY-MM-DD|null", + "filed_date": "YYYY-MM-DD|null", + "source": "sec|dolt|legacy_api" + } + ], + "valuation": { // object | null (null until SEC snapshots exist, phase A3); same industry sub-object rules + // computed at REQUEST TIME from stored snapshots + latest PostgreSQL close — + // no valuation cache or table; unaffected by SEC availability + "pe": 29.2|null, "fcf_yield": 3.8|null, + "market_cap_est": 1.2e9|null, // estimated — UI labels "est." + "pe_industry": {...}|null, "fcf_yield_industry": {...}|null, + "price_date": "YYYY-MM-DD" // close used for the ratios + } +} +``` + +Null/freshness semantics: absent data is `null` with the row still present (the UI +shows "n/a", never hides rows); every metric carries its own source, period and +filing date — no panel-wide source label. The objects may serve partial data during +rollout (e.g. `earnings` live, `metrics` still `legacy_api`); the shape never +changes. + +## UI — `frontend/src/components/ticker/FundamentalsPanel.tsx` + +One distinctive visual device — the **quarter tape** — in an otherwise restrained +panel. Preserve the app's dark glass styling and numeric typography. + +``` +Fundamentals Quality improving · valuation rich +Next earnings Aug 3 · AMC Last 4: beat beat miss beat + +Quarter tape Q−3 Q−2 Q−1 Latest Read +Revenue growth 8% 11% 15% 18% accelerating +Operating margin 19% 20% 20% 22% improving +FCF margin 12% 10% 14% 16% above own average +Share count change 1.8% dilution + +Balance & valuation +Net debt / EBITDA 1.4× Industry median 2.1× healthy leverage +P/E 29.2× Industry median 23.5× priced above peers +FCF yield 3.8% Industry median 3.1% above peers +``` + +- Growth, margins, share count: latest four periods as a compact four-cell tape + (or sparkline) plus a deterministic text read (rules below). +- P/E, FCF yield, leverage: horizontal industry-percentile strip with a median + marker. Hidden entirely when `industry` is null (< 5 peer issuers). +- Earnings: four bars around a zero baseline — green beats, red misses, gray + unavailable — plus next date and BMO/AMC session countdown. +- Accessibility: color always paired with text or arrows; neutral/ambiguous stays + gray; green/red only when a read is genuinely favorable/adverse. +- Remove the hard-coded "FMP" source label — provenance is per metric. + +**Deterministic reads — one shared rule set.** Implement as a single function with +named constants; the tape reads and the header sentence use identical outputs. No +LLM, no new composite score. Defaults (tunable constants, not scattered literals): + +- A series read requires ≥ 3 periods; otherwise show "—" and no read. +- Growth metrics (pp): latest − prior ≥ +2.0 → "accelerating"; + ≤ −2.0 → "decelerating"; else "steady". +- Margins (latest vs mean of prior periods, pp): ≥ +1.0 → "improving"; + ≤ −1.0 → "deteriorating"; else "stable" (phrased "above/below own average" + where the layout calls for it). +- Share count YoY: > +1.0% → "N% dilution"; < −1.0% → "buying back"; else "flat". +- Peer-relative: favorable_percentile ≥ 60 → favorable ("above peers"); + ≤ 40 → adverse ("priced above peers" for P/E, "elevated leverage" for + net-debt/EBITDA); else "in line". +- Header sentence: join the growth read, margin read and peer-relative valuation + read with " · ", omitting segments that have no read (e.g. "Growth accelerating + · margins stable · valuation above industry median"). **Segment sources are + fixed:** growth = revenue growth read; margins = operating margin read; + valuation = P/E peer-relative read, falling back to FCF yield when P/E is null. + This keeps the header unambiguous when sibling metrics (EPS vs revenue growth, + P/E vs FCF yield) point in different directions. + +## Decommissioning (end of workstream A) + +Remove completely: FMP (`app/providers/fmp.py`), Finnhub + Alpha Vantage +(`app/providers/fundamentals_chain.py`), their config keys (`app/config.py`), and +their wiring in `app/scheduler.py`, `app/routers/ingestion.py`, +`app/services/ticker_universe_service.py`. Retain: Alpaca (prices), FRED, +sentiment provider, Telegram. Note: decommissioning does **not** depend on +workstream B — Alpaca remains the price source throughout. + +## Rollout + +**Workstream A:** + +- A0. License review; dolt binary pinned in deploy; `DOLT_DATA_DIR` outside the + rsync tree (earnings clone only — small), provisioned and backed up. +- A1. Migration 026, import-run framework. +- A2. Earnings ingestion in shadow (writes `earnings_events`, prod untouched); + verify forward-calendar coverage and rescheduling behavior. +- A3. SEC daily job in shadow (writes `fundamental_snapshots`). **Primary technical + risk here: Q4 derivation and fiscal-period alignment** — non-calendar fiscal years, + restatements/amendments, and XBRL unit/dimension variants; budget accordingly. +- A4. API v1 + FundamentalsPanel + peer comparison — served from snapshots and + earnings_events, independent of the scoring cutover (the additive API supports + partial data). UI value ships before anything touches scoring inputs. +- A5. **Score-parity gate** → `fundamental_data` cutover: compute candidate + pe_ratio/revenue_growth/earnings_surprise from SEC/Dolt side by side with the + API values across the tracked universe, report per-field deltas and resulting + fundamental-score/ranking changes, require explicit approval. Definition + changes (e.g. TTM vs provider convention) called out, not averaged away. +- A6. Remove FMP/Finnhub/Alpha Vantage; keep monitoring + manual fallback. + +**Workstream B (independent, start when wanted):** + +- B0. Stocks clone (~4.7 GB) provisioned; migration 027. +- B1. OHLCV + split adjustment in shadow (writes `ohlcv_source_bars` only; Alpaca + keeps owning `ohlcv_records`); historical backfill. +- B2. Reconciliation window (≥ 2 weeks) vs Alpaca; review validation summaries. +- B3. Promote Dolt as historical OHLCV source (canonical rebuilt from raw source + bars + splits); morning pipeline → 03:00. + +## Test plan + +- Daily SEC job: changed revision imports; unchanged conditional-HTTP check is a + `no_op` with zero downloads and zero row changes; validation failure leaves + production untouched; source unavailable still runs the local + `fundamental_data` refresh (step c, post-activation); before activation the job + never writes `fundamental_data`. +- Valuation endpoint returns identical values with SEC reachable and unreachable + (pure PostgreSQL computation); no valuation rows exist in any table. +- Multi-class tickers resolve to the same CIK snapshots; peer medians and + peer_count are CIK-deduplicated (GOOG+GOOGL = one observation). +- Earnings rescheduling: a moved future date replaces the old row atomically; a + cancelled date disappears; historical results are never touched. +- Amendment selection: for a period with multiple accessions, the newest valid + accepted_at wins at read time; older rows remain unchanged. +- History arrays are chronological, ≤ 4 points. +- Percentage-point units stay compatible with existing formatters and scoring + inputs. +- Deterministic reads: threshold boundary cases (exactly +2.0pp, exactly 60th + percentile) resolve per the stated rules; header uses identical outputs and + falls back from P/E to FCF yield for the valuation segment when P/E is null. +- Peer comparison disappears below 5 peer issuers; favorable-percentile direction + correct for both polarities. +- Workstream B: split-adjusted OHLCV matches Alpaca on representative normal / + split / reverse-split symbols. +- UI states: positive, adverse, neutral, insufficient history, insufficient + peers; mobile layout; non-color accessibility. +- Unit, integration, scheduler and frontend suites pass. + +## Acceptance criteria + +- App works normally with Dolt/DoltHub/SEC unreachable. +- Re-running the same revision: zero duplicate or changed rows. +- **Upcoming earnings dates present and timely for the tracked universe** — the + forward calendar is the hardest thing to replace and gates decommissioning. +- Coverage meets the tracked-universe target; scheduler runs cleanly with + FMP/Finnhub/AV keys removed from the environment. +- Score-parity diff reviewed and approved before `fundamental_data` cutover. +- Scheduled imports never block the API event loop. + +## Deferred (explicitly, until a concrete need appears) + +- Workstream B itself is deferred relative to A and blocks nothing in A. +- Exact byte-level source replay of historical imports; permanent archive store. +- Point-in-time backtest enforcement (`accepted_at` is stored now; derivation and + backtest visibility rules are built only when fundamentals enter + scoring/backtesting). +- Fundamental metrics in the score; sector-relative scoring. +- Aggregate/rollup tables for peer statistics. +- Any valuation cache or table (request-time computation from snapshots + latest + close suffices). +- A dedicated conflicts table (validation JSON + logs suffice).