2026-07-23 13:27:08 +02:00
1 changed files with 464 additions and 0 deletions
@@ -0,0 +1,464 @@
+# Dolt bulk-data integration — implementation plan
+
+Status: approved 2026-07-21, revised through four review rounds; direction: KISS
+backend, UI value first. Hand-off document for the implementing agent;
+self-contained.
+
+## Objective
+
+Replace the free-tier fundamentals APIs (FMP, Finnhub, Alpha Vantage) with bulk
+data: SEC Company Facts for fundamentals, the DoltHub earnings repo for the
+earnings calendar/history, and — later, independently — the DoltHub stocks repo for
+historical OHLCV. PostgreSQL stays the production system of record.
+
+**Delivery order: two independent workstreams.**
+
+- **Workstream A (build first):** SEC fundamentals + Dolt earnings + API v1 +
+  FundamentalsPanel + decommission FMP/Finnhub/Alpha Vantage. Valuation uses the
+  existing Alpaca closes already in `ohlcv_records`. This alone achieves the goal
+  (killing the quota-limited APIs) and delivers all the UI value.
+- **Workstream B (later, optional until needed):** replace historical OHLCV with
+  the Dolt stocks repo. The most complex machinery (4.7 GB clone, split
+  adjustment, source-bar table, reconciliation) lives here and blocks nothing in A.
+
+**Guiding principle: KISS.** Plain daily importers with staging and atomic
+promotion — no forensic replay, no permanent archive store, no conflict tables, no
+aggregate tables. Engineering budget goes into the UI (quarter trends, peer
+comparison). Deferred until a concrete need: exact source replay, point-in-time
+backtest enforcement, fundamental metrics in scoring.
+
+**Non-negotiables**
+
+- The application never queries Dolt/DoltHub or SEC at request time. All access is
+  batch import → PostgreSQL. If a sync fails or the source is unchanged, production
+  continues on the last successfully imported data.
+- Do not replace PostgreSQL with Dolt/Doltgres. Never commit to the upstream clones.
+- No owned SEC Dolt repo: SEC JSON is normalized straight into PostgreSQL.
+- Scoring **code** is unchanged, but swapping the data source changes production
+  behavior: `app/services/scoring_service.py` (~line 450) scores pe_ratio /
+  revenue_growth / earnings_surprise from `fundamental_data`, so new definitions
+  change rankings even with identical code. Cutover of `fundamental_data`
+  population requires the **score-parity gate** (phase A5) — never silently. All
+  *new* metrics are display-only.
+- Intraday (10:00–15:00), near-close (15:30) and after-close (16:45) pipelines stay
+  on Alpaca unchanged.
+
+## Data sources
+
+1. **SEC Company Facts + submissions bulk files** (free, no key; costs bandwidth,
+   CPU and disk — optimize accordingly) — XBRL facts per **issuer (CIK), not per
+   ticker**. Tickers resolve to a CIK via SEC `company_tickers.json` (multi-class
+   issuers like GOOGL/GOOG share one CIK and one set of fundamentals). Submissions
+   also supply the SIC code (peer grouping) and `acceptanceDateTime`. Handle unit
+   variants and fiscal-period alignment (derive Q4 = FY − Q1..Q3 where needed).
+   **Amendments:** retain every accession immutably; readers select the newest
+   valid `accepted_at` snapshot per reporting period at read time. No flags, no
+   mutation.
+2. **`post-no-preference/earnings`** (DoltHub) — announcement date, BMO/AMC session
+   (partial), period end, EPS estimate/actual, surprise history. Small clone.
+   `scripts/import_dolthub_earnings.py` is a research/SQLite importer — reuse its
+   normalization/alignment logic (calendar↔EPS-history monotonic alignment, SUE
+   scaling) but write a production PostgreSQL importer; do not extend the script.
+3. **`post-no-preference/stocks`** (DoltHub, **workstream B**) — daily raw OHLCV
+   (unadjusted), symbol metadata, splits, dividends. Publishes ~01:30 ET the
+   following calendar day. Clone is ~4.7 GB.
+
+**Licensing (phase A0):** verify the DoltHub repos' CC-BY-SA 4.0 terms are
+compatible before building. Internal, non-redistributed use is expected to be fine;
+record the conclusion in this doc.
+
+## Schema
+
+**Migration 026 (workstream A)** — current head: `025_trade_setup_scan_run_id`:
+
+- `data_import_runs` — lean: id, source (`sec_facts` | `dolt_earnings` |
+  `dolt_stocks`), revision (Dolt commit hash, or SEC archive SHA-256), status
+  (`running`/`validated`/`promoted`/`no_op`/`failed`), source_max_date, row_counts
+  JSON, validation JSON (includes reconciliation/discrepancy summaries — no
+  separate conflicts table; details go to structured logs), started_at,
+  completed_at, error_details. One run per source at a time (Postgres advisory
+  lock keyed by source).
+- `earnings_events` — ticker_id, announce_date, session (`bmo`/`amc`/`unknown`),
+  period_end, eps_estimate, eps_actual, source, import_run_id. Unique
+  (ticker_id, announce_date). **Rescheduling:** within each promotion transaction,
+  delete this source's future-dated rows (announce_date > today) and re-insert
+  from the new snapshot, so moved or cancelled dates never linger. Past rows
+  (results) are never deleted.
+- `tickers` — add nullable `cik`, `sic`, `sic_description` (from SEC submissions /
+  `company_tickers.json`; refreshed by the SEC import; multi-class tickers share
+  values). The only ticker↔issuer join point.
+- `fundamental_snapshots` — **CIK-keyed, one immutable row per accession**: cik,
+  accession (unique), form, filed_date, **accepted_at** (kept although PIT
+  enforcement is deferred — one timestamp now vs painful retrofit later),
+  **period_start, period_end, fiscal_year, fiscal_period** (the filing's own
+  `dei`/`us-gaap` period identity — required to align non-calendar fiscal years and
+  to derive discrete quarters from cumulative facts), and the **price-independent
+  raw facts** so metrics are recomputable. **Store facts as the filing reports
+  them, not as derived quarters:** duration facts (revenue, net income, diluted EPS,
+  CFO, capex, EBITDA inputs) retain the filing's normalized **cumulative YTD/FY**
+  value for the (period_start → period_end) span; balance-sheet facts (cash+ST
+  investments, total debt, diluted shares) are **period-end** values. **Nothing
+  derived is frozen into a row:** discrete quarters (10-Q YTD deltas, Q4 = FY −
+  Q1..Q3), TTM, YoY and the quarter-tape series are all computed **at read time** by
+  picking the newest valid accepted_at snapshot for *each* required period — so
+  non-calendar fiscal years resolve correctly and a later amendment to a prior
+  quarter is reflected automatically without ever storing a stale derived quarter.
+  Readers pick the newest valid accepted_at per period; history powers the UI
+  quarter tape.
+- Keep `fundamental_data` (`app/models/fundamental.py`) as the latest-value compat
+  cache, repopulated by the daily SEC job — but only after the phase-A5 parity
+  gate.
+
+**Migration 027 (workstream B, written when B starts):**
+
+- `ohlcv_source_bars` — source-truth bar table, required because `ohlcv_records`
+  allows one row per (ticker_id, date) (`app/models/ohlcv.py:12`) and Alpaca
+  ingestion upserts it in place (`app/services/price_service.py:82`) — Dolt and
+  Alpaca bars cannot coexist there. Holds **Dolt raw (unadjusted) bars only** —
+  Alpaca bars are already split-adjusted at the provider (`app/providers/alpaca.py:77`
+  requests `Adjustment.SPLIT`) and live exclusively in `ohlcv_records`. Columns:
+  source (`dolt`), adjustment (`raw` — explicit), ticker_id, date, OHLCV,
+  import_run_id; unique (source, ticker_id, date). Changed bars are counted in the
+  run's validation JSON and logged before overwrite.
+- `corporate_actions` — ticker_id, type (`split`/`dividend`), ex_date,
+  ratio/amount, source, import_run_id. Unique (ticker_id, type, ex_date).
+- `ohlcv_records` — add nullable `import_run_id` FK and `source` text (default
+  `'alpaca'`).
+
+## Import framework
+
+Every importer: idempotent per revision (same Dolt commit / archive checksum →
+`no_op`, zero row changes); staging tables first; promotion in one transaction;
+safe to retry; a failed or unchanged run leaves the current dataset untouched.
+Record every attempt in `data_import_runs`.
+
+**SEC access requirements (operational safeguards, per SEC fair-access policy):**
+send an identifying `User-Agent` with a contact email on every request; stay far
+below the 10 req/s limit (the bulk endpoints need only a handful of requests per
+run); exponential backoff on 429; a 403 means the User-Agent or request pattern is
+wrong — alert and stop, never retry-loop. See SEC developer resources
+(https://www.sec.gov/about/developer-resources).
+
+**Reproducibility scope (deliberately limited):** the normalized snapshots in
+PostgreSQL *are* the durable record. Keep only the last ~2 SEC archives on disk for
+debugging. Byte-level replay of old runs is out of scope until a concrete need.
+
+Dolt access: `dolt pull` on the persistent clone, record the resulting commit hash,
+read via `dolt sql -r csv` (no long-running sql-server). **The scheduler shares one
+event loop with the API** (`app/scheduler.py:73`) — run dolt/unzip/download
+subprocesses via `asyncio.create_subprocess_exec` (or an executor), never blocking
+calls. Check free disk space before pulling; alert and skip if below threshold.
+
+**Deployment constraints:** deploy is `rsync --delete` of the repo tree
+(`.gitea/workflows/deploy.yml:127`), so clones and archives must live **outside the
+deployment path** — an env-configured persistent directory (e.g.
+`DOLT_DATA_DIR=/var/lib/signal-platform/dolt`), backed up. The dolt binary is a new
+prod runtime dependency: install with a pinned version in the deploy workflow.
+
+**Validation gates (block promotion, raise an alert via the existing system-events
+path):** source freshness as expected; tracked-universe coverage; no duplicate
+business keys; fundamental units/periods consistent; row-count deltas within
+reason; upstream schema change stops promotion. Workstream B adds: OHLC sanity
+(high ≥ open/close/low, low ≤ open/close/high, volume ≥ 0); no unexplained split
+discontinuities.
+
+**Split adjustment (workstream B):** `ohlcv_source_bars` + `corporate_actions` are
+the source of truth; canonical `ohlcv_records` is *generated* from them to match
+Alpaca `Adjustment.SPLIT`, selecting only adjustment = `raw` rows as input so
+adjustment is applied exactly once. A newly published split rewrites the symbol's
+entire adjusted history — treat whole-symbol rewrites as a normal import event
+(exempt that symbol from the row-count-delta gate for that run) and stamp rows with
+the import_run_id (a backtest↔prod parity guard exists; changed history changes
+backtests).
+
+## Scheduling (`app/scheduler.py` — `SCHEDULE_DEFAULTS` / `_CRON_JOBS`, ~line 1451)
+
+Follow the existing pattern: cron strings in SystemSettings via
+`app/services/settings_store.py`, day-of-week as names never numbers, logging via
+`_log_event`.
+
+Workstream A:
+
+- Dolt earnings import: daily ~02:30 ET (with the future-row replacement above).
+- **SEC fundamentals job: daily ~04:00 ET.** One job, three steps:
+  (a) freshness check **using conditional HTTP metadata (ETag/Last-Modified /
+  If-None-Match) before downloading** — an unchanged archive is a `no_op` without
+  pulling the multi-GB file; verify by SHA-256 after any actual download;
+  (b) when changed, **parse and write only tracked-universe CIKs** through
+  staging→promotion (the tracked universe is `ticker_universe_service`'s set;
+  CIKs are resolved from `company_tickers.json` each run, so a newly added ticker
+  self-resolves on its next SEC run — until then its metrics are simply null);
+  (c) **always, locally, and only after production activation** (the phase-A5
+  parity approval): refresh the legacy `fundamental_data` fields and mark affected
+  cached fundamental scores stale. **Sources differ per field** — do not assume all
+  five come from SEC: `pe_ratio` and `market_cap` from the newest valid snapshots ×
+  latest PostgreSQL close, each with its own formula — `pe_ratio` = latest close /
+  TTM diluted EPS; `market_cap` = issuer-wide diluted shares × latest close;
+  `revenue_growth` from the snapshots alone; `earnings_surprise` and
+  `next_earnings_date` from `earnings_events` (the Dolt earnings feed — these two do
+  not exist in SEC facts). Before activation the job imports snapshots only
+  (shadow). Step (c) must run identically when SEC is unreachable — prices move
+  daily even when filings don't, and the earnings-derived fields already live in
+  PostgreSQL.
+  **The new API valuation object is not stored anywhere** — it is computed at
+  request time (below). No valuation cache or table exists.
+
+Workstream B:
+
+- Dolt OHLCV+splits pull/import: `0 2 * * tue-sat` ET. If source_max_date is not
+  fresh, retry hourly until ~06:00, then give up quietly. After a successful
+  import, reconcile the previous session's Dolt-derived bars against the Alpaca
+  bars; summary into the run's validation JSON, details to logs.
+- Move `schedule_daily_pipeline_cron` (morning refresh) from `0 2 * * *` to
+  `0 3 * * *` (only needed once the 02:00 slot is taken by the OHLCV pull).
+  **Late Dolt publication is a non-event:** the canonical scan runs at 15:30 on
+  Alpaca, so the morning pipeline runs normally even when the import hasn't
+  landed — no gating, no defensive coupling.
+
+## Metrics catalog (curated — TTM basis)
+
+Snapshots store **price-independent per-period facts** (the "snapshot" column below
+means *derived from stored snapshots, assembled across periods at read time* — see
+Schema — not frozen at import); price-dependent ratios are never frozen into
+snapshots and have **no storage location at all**: the API computes
+them at request time from the stored snapshots + the latest `ohlcv_records` close
+(both already in PostgreSQL, so this works identically when SEC is unreachable).
+The only stored price-dependent values are the legacy `fundamental_data` fields
+that scoring already reads, refreshed daily by step (c) after activation.
+
+| Metric | Definition | Where computed |
+|---|---|---|
+| Revenue growth YoY | TTM revenue vs prior TTM | snapshot |
+| EPS growth YoY | TTM diluted EPS vs prior TTM | snapshot |
+| Operating margin + 4q trend | TTM operating income / revenue | snapshot |
+| FCF margin | (TTM CFO − capex) / revenue | snapshot |
+| Net cash / net debt | cash + ST investments − total debt | snapshot |
+| Net debt / EBITDA | net debt / TTM EBITDA | snapshot |
+| Share count Δ YoY | diluted shares vs year ago | snapshot |
+| Trailing P/E | price / TTM diluted EPS | request time |
+| FCF yield | TTM FCF / est. market cap | request time |
+| Est. market cap | issuer-wide diluted shares × ticker price | request time |
+| Earnings surprise history | last 4+ from `earnings_events` | query |
+
+**Market cap is an estimate** (issuer-wide diluted shares × one ticker's price —
+approximate for multi-class issuers). Label it "est." in the UI and round
+aggressively rather than withholding it; false precision is the failure mode, not
+the approximation.
+
+**Units follow existing app conventions:** percentages are percentage points
+(21.0 = 21%), P/E and net-debt/EBITDA are multiples, market cap and net debt are
+dollars.
+
+Deliberately **excluded**: ROIC (invested-capital/NOPAT normalization too noisy),
+gross margin (COGS tagging too inconsistent), any new composite score.
+
+## Peer comparison (read-time only)
+
+- Peer group = tracked-universe issuers sharing the **first two SIC digits**,
+  **deduplicated by CIK** — GOOG and GOOGL are one issuer, one observation, in
+  medians, percentiles and peer_count.
+- Computed at read time from current snapshots — no aggregate tables until
+  performance demonstrates a need.
+- Medians exclude null/invalid values. **Fewer than 5 valid peer issuers → omit
+  the peer result entirely** rather than showing a misleading universe comparison.
+- Percentile direction respects metric polarity (higher-is-better for FCF yield,
+  lower-is-better for P/E and leverage).
+
+## API contract (additive v1)
+
+Every existing top-level field is preserved unchanged (name, type, position) —
+backend and frontend ship independently, no breaking interval. New objects, exact
+names and types:
+
+```jsonc
+{
+  // ...all existing legacy fields, unchanged...
+  "earnings": {
+    "next": {"date": "YYYY-MM-DD", "session": "bmo|amc|unknown", "days_until": 12} | null,
+    "recent": [   // newest first, max 4, may be empty
+      {"announce_date": "YYYY-MM-DD", "period_end": "YYYY-MM-DD|null",
+       "eps_estimate": 1.02|null, "eps_actual": 1.10|null, "surprise_pct": 7.8|null}
+    ]
+  },
+  "metrics": [    // fixed row set — every key always present, value null when unavailable
+    {
+      "key": "revenue_growth_yoy",   // revenue_growth_yoy | eps_growth_yoy | operating_margin |
+                                     // fcf_margin | net_debt | net_debt_to_ebitda | share_count_change_yoy
+      "value": 18.0,                 // number | null — pp / multiples / dollars per units above
+      "history": [                   // oldest→newest, max 4 points, [] when unavailable
+        {"period_end": "YYYY-MM-DD", "value": 8.0}
+      ],
+      "industry": {                  // object | null — null when < 5 valid peer issuers (CIK-deduped)
+        "label": "SIC 73 peers",     // truthful 2-digit group label — grouping IS 2-digit,
+                                     // so no 4-digit description like "Prepackaged Software"
+        "median": 11.0,
+        "favorable_percentile": 82,  // 0-100, polarity-aware (higher = more favorable)
+        "peer_count": 12             // issuers, not tickers
+      },
+      "period_end": "YYYY-MM-DD|null",
+      "filed_date": "YYYY-MM-DD|null",
+      "source": "sec|dolt|legacy_api"
+    }
+  ],
+  "valuation": {  // object | null (null until SEC snapshots exist, phase A3); same industry sub-object rules
+    // computed at REQUEST TIME from stored snapshots + latest PostgreSQL close —
+    // no valuation cache or table; unaffected by SEC availability
+    "pe": 29.2|null, "fcf_yield": 3.8|null,
+    "market_cap_est": 1.2e9|null,    // estimated — UI labels "est."
+    "pe_industry": {...}|null, "fcf_yield_industry": {...}|null,
+    "price_date": "YYYY-MM-DD"       // close used for the ratios
+  }
+}
+```
+
+Null/freshness semantics: absent data is `null` with the row still present (the UI
+shows "n/a", never hides rows); every metric carries its own source, period and
+filing date — no panel-wide source label. The objects may serve partial data during
+rollout (e.g. `earnings` live, `metrics` still `legacy_api`); the shape never
+changes.
+
+## UI — `frontend/src/components/ticker/FundamentalsPanel.tsx`
+
+One distinctive visual device — the **quarter tape** — in an otherwise restrained
+panel. Preserve the app's dark glass styling and numeric typography.
+
+```
+Fundamentals                     Quality improving · valuation rich
+Next earnings  Aug 3 · AMC       Last 4:  beat  beat  miss  beat
+
+Quarter tape          Q−3    Q−2    Q−1    Latest      Read
+Revenue growth         8%     11%    15%     18%       accelerating
+Operating margin      19%     20%    20%     22%       improving
+FCF margin            12%     10%    14%     16%       above own average
+Share count change                                     1.8% dilution
+
+Balance & valuation
+Net debt / EBITDA      1.4×      Industry median 2.1×   healthy leverage
+P/E                   29.2×      Industry median 23.5×  priced above peers
+FCF yield              3.8%      Industry median 3.1%   above peers
+```
+
+- Growth, margins, share count: latest four periods as a compact four-cell tape
+  (or sparkline) plus a deterministic text read (rules below).
+- P/E, FCF yield, leverage: horizontal industry-percentile strip with a median
+  marker. Hidden entirely when `industry` is null (< 5 peer issuers).
+- Earnings: four bars around a zero baseline — green beats, red misses, gray
+  unavailable — plus next date and BMO/AMC session countdown.
+- Accessibility: color always paired with text or arrows; neutral/ambiguous stays
+  gray; green/red only when a read is genuinely favorable/adverse.
+- Remove the hard-coded "FMP" source label — provenance is per metric.
+
+**Deterministic reads — one shared rule set.** Implement as a single function with
+named constants; the tape reads and the header sentence use identical outputs. No
+LLM, no new composite score. Defaults (tunable constants, not scattered literals):
+
+- A series read requires ≥ 3 periods; otherwise show "—" and no read.
+- Growth metrics (pp): latest − prior ≥ +2.0 → "accelerating";
+  ≤ −2.0 → "decelerating"; else "steady".
+- Margins (latest vs mean of prior periods, pp): ≥ +1.0 → "improving";
+  ≤ −1.0 → "deteriorating"; else "stable" (phrased "above/below own average"
+  where the layout calls for it).
+- Share count YoY: > +1.0% → "N% dilution"; < −1.0% → "buying back"; else "flat".
+- Peer-relative: favorable_percentile ≥ 60 → favorable ("above peers");
+  ≤ 40 → adverse ("priced above peers" for P/E, "elevated leverage" for
+  net-debt/EBITDA); else "in line".
+- Header sentence: join the growth read, margin read and peer-relative valuation
+  read with " · ", omitting segments that have no read (e.g. "Growth accelerating
+  · margins stable · valuation above industry median"). **Segment sources are
+  fixed:** growth = revenue growth read; margins = operating margin read;
+  valuation = P/E peer-relative read, falling back to FCF yield when P/E is null.
+  This keeps the header unambiguous when sibling metrics (EPS vs revenue growth,
+  P/E vs FCF yield) point in different directions.
+
+## Decommissioning (end of workstream A)
+
+Remove completely: FMP (`app/providers/fmp.py`), Finnhub + Alpha Vantage
+(`app/providers/fundamentals_chain.py`), their config keys (`app/config.py`), and
+their wiring in `app/scheduler.py`, `app/routers/ingestion.py`,
+`app/services/ticker_universe_service.py`. Retain: Alpaca (prices), FRED,
+sentiment provider, Telegram. Note: decommissioning does **not** depend on
+workstream B — Alpaca remains the price source throughout.
+
+## Rollout
+
+**Workstream A:**
+
+- A0. License review; dolt binary pinned in deploy; `DOLT_DATA_DIR` outside the
+  rsync tree (earnings clone only — small), provisioned and backed up.
+- A1. Migration 026, import-run framework.
+- A2. Earnings ingestion in shadow (writes `earnings_events`, prod untouched);
+  verify forward-calendar coverage and rescheduling behavior.
+- A3. SEC daily job in shadow (writes `fundamental_snapshots`). **Primary technical
+  risk here: Q4 derivation and fiscal-period alignment** — non-calendar fiscal years,
+  restatements/amendments, and XBRL unit/dimension variants; budget accordingly.
+- A4. API v1 + FundamentalsPanel + peer comparison — served from snapshots and
+  earnings_events, independent of the scoring cutover (the additive API supports
+  partial data). UI value ships before anything touches scoring inputs.
+- A5. **Score-parity gate** → `fundamental_data` cutover: compute candidate
+  pe_ratio/revenue_growth/earnings_surprise from SEC/Dolt side by side with the
+  API values across the tracked universe, report per-field deltas and resulting
+  fundamental-score/ranking changes, require explicit approval. Definition
+  changes (e.g. TTM vs provider convention) called out, not averaged away.
+- A6. Remove FMP/Finnhub/Alpha Vantage; keep monitoring + manual fallback.
+
+**Workstream B (independent, start when wanted):**
+
+- B0. Stocks clone (~4.7 GB) provisioned; migration 027.
+- B1. OHLCV + split adjustment in shadow (writes `ohlcv_source_bars` only; Alpaca
+  keeps owning `ohlcv_records`); historical backfill.
+- B2. Reconciliation window (≥ 2 weeks) vs Alpaca; review validation summaries.
+- B3. Promote Dolt as historical OHLCV source (canonical rebuilt from raw source
+  bars + splits); morning pipeline → 03:00.
+
+## Test plan
+
+- Daily SEC job: changed revision imports; unchanged conditional-HTTP check is a
+  `no_op` with zero downloads and zero row changes; validation failure leaves
+  production untouched; source unavailable still runs the local
+  `fundamental_data` refresh (step c, post-activation); before activation the job
+  never writes `fundamental_data`.
+- Valuation endpoint returns identical values with SEC reachable and unreachable
+  (pure PostgreSQL computation); no valuation rows exist in any table.
+- Multi-class tickers resolve to the same CIK snapshots; peer medians and
+  peer_count are CIK-deduplicated (GOOG+GOOGL = one observation).
+- Earnings rescheduling: a moved future date replaces the old row atomically; a
+  cancelled date disappears; historical results are never touched.
+- Amendment selection: for a period with multiple accessions, the newest valid
+  accepted_at wins at read time; older rows remain unchanged.
+- History arrays are chronological, ≤ 4 points.
+- Percentage-point units stay compatible with existing formatters and scoring
+  inputs.
+- Deterministic reads: threshold boundary cases (exactly +2.0pp, exactly 60th
+  percentile) resolve per the stated rules; header uses identical outputs and
+  falls back from P/E to FCF yield for the valuation segment when P/E is null.
+- Peer comparison disappears below 5 peer issuers; favorable-percentile direction
+  correct for both polarities.
+- Workstream B: split-adjusted OHLCV matches Alpaca on representative normal /
+  split / reverse-split symbols.
+- UI states: positive, adverse, neutral, insufficient history, insufficient
+  peers; mobile layout; non-color accessibility.
+- Unit, integration, scheduler and frontend suites pass.
+
+## Acceptance criteria
+
+- App works normally with Dolt/DoltHub/SEC unreachable.
+- Re-running the same revision: zero duplicate or changed rows.
+- **Upcoming earnings dates present and timely for the tracked universe** — the
+  forward calendar is the hardest thing to replace and gates decommissioning.
+- Coverage meets the tracked-universe target; scheduler runs cleanly with
+  FMP/Finnhub/AV keys removed from the environment.
+- Score-parity diff reviewed and approved before `fundamental_data` cutover.
+- Scheduled imports never block the API event loop.
+
+## Deferred (explicitly, until a concrete need appears)
+
+- Workstream B itself is deferred relative to A and blocks nothing in A.
+- Exact byte-level source replay of historical imports; permanent archive store.
+- Point-in-time backtest enforcement (`accepted_at` is stored now; derivation and
+  backtest visibility rules are built only when fundamentals enter
+  scoring/backtesting).
+- Fundamental metrics in the score; sector-relative scoring.
+- Aggregate/rollup tables for peer statistics.
+- Any valuation cache or table (request-time computation from snapshots + latest
+  close suffices).
+- A dedicated conflicts table (validation JSON + logs suffice).