Docs/dolt plan clarifications #1

2026-07-23T13:26:46+02:00

dennisthiessen commented

2026-07-23 13:26:46 +02:00

No description provided.

dennisthiessen added 34 commits 2026-07-23 13:26:47 +02:00

docs: add Dolt integration plan + review clarifications e09ec4ab1d

Adds the Dolt bulk-data integration hand-off plan (authored in prior
session) and applies clarifications found in a pre-handoff review:

- step (c): fields refresh from three distinct sources, not "snapshots +
  close" — earnings_surprise/next_earnings_date come from earnings_events,
  not SEC facts. pe_ratio = close / TTM diluted EPS and market_cap =
  issuer-wide diluted shares × close stated as separate formulas.
- fundamental_snapshots made implementable: add period_start, fiscal_year,
  fiscal_period; duration facts store the filing's cumulative YTD/FY values,
  balance-sheet facts store period-end values. Discrete quarters, Q4, TTM
  and YoY are derived at read time (correct for non-calendar fiscal years;
  amendments never freeze a stale derived quarter).
- flag Q4 derivation / fiscal-period alignment as the primary A3 risk.
- anchor "tracked universe" to ticker_universe_service + per-run CIK
  resolution.

All code anchors in the doc verified accurate against the current tree.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(dolt): migration 026 + models — fundamentals/earnings schema (A1 schema) 949cbbe7c0

First reviewable slice of workstream A: schema only, no importers, no data.

- data_import_runs: lean batch-import audit (source/revision/status,
  row_counts_json + validation_json as Text-holding-JSON per repo convention).
- fundamental_snapshots: CIK-keyed, one immutable row per accession; stores
  per-period raw facts (duration = cumulative YTD/FY, balance-sheet =
  period-end) plus period_start/period_end/fiscal_year/fiscal_period so
  discrete quarters, Q4, TTM and YoY are derived at read time.
- earnings_events: Dolt-sourced calendar + surprise history, unique
  (ticker_id, announce_date).
- tickers: nullable cik/sic/sic_description — the ticker<->issuer join point.

fundamental_data is left untouched (cutover gated separately at A5). Models
registered in app/models/__init__.py; Ticker gains an earnings_events
relationship. Verified: create_all builds the tables, mappers configure, and
migration 026 renders valid Postgres DDL up and down.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(dolt): source-agnostic import-run framework (A1) febd741671

run_import + SourceImporter Protocol (detect_revision/stage/validate/promote)
giving every bulk importer the plan's non-negotiables, KISS:

- one run per source at a time — Postgres session-level advisory lock held on a
  single pinned engine.connect() so it survives the running-row and promotion
  commits; no-op on SQLite.
- idempotent per revision — cheap detect_revision compared to the last promoted
  run; unchanged revision records a no_op with zero writes (no fetch).
- staging (in-memory, no physical staging tables) → validate (read-only) →
  atomic promote + run-row flip in one transaction.
- failed validation or mid-run exception marks the run failed, alerts via
  system_event_service, and leaves live tables untouched.

Every attempt recorded in data_import_runs; conflicts summary in validation_json
(no conflicts table). Concrete SEC/earnings importers land in later phases.

Tests: 6 orchestration tests (no_op / promote / new-revision / failed-untouched
/ promote-exception-rollback) + deterministic advisory-key derivation. Full
suite 680 passed. Advisory-lock mutual exclusion is PG-verify-pending (SQLite
no-ops it — flagged, not covered).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(dolt): record/alert revision-detection failures + handle cancellation 8b45bdb361

Review fixes to the import-run framework (A1):

1. detect_revision ran outside the failure handler, so a failed revision probe
   (the most likely external failure) escaped unrecorded — violating "every
   attempt is recorded". Now the running row is created FIRST, then
   detect_revision + last-revision lookup + stage + validate + promote all run
   inside the same handler; the row converts to no_op when the revision is
   unchanged. New test covers a detection exception → recorded failed + alert.

2. asyncio.CancelledError (BaseException, not caught by except Exception) left a
   permanent running row on deploy/scheduler shutdown. Now caught explicitly:
   best-effort mark failed, then re-raise the cancellation (never swallowed).
   New test asserts the run is failed and the error re-propagates.

Full suite 682 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: shares_outstanding wording + staged-representation terminology fc192c9f74

Review items 3-4 on the plan doc:

- Replace remaining "diluted shares" (the share *count*) with point-in-time
  shares_outstanding (dei:EntityCommonStockSharesOutstanding) across schema,
  metrics catalog and market-cap note. "diluted EPS" is left as-is (correctly a
  duration fact). Adds the multi-class rule: derive the issuer-wide count from
  the consolidated cover-page figure OR by summing class-specific facts (GOOG +
  GOOGL) — never both, to avoid double counting.
- The framework stages into a representation *outside the live tables* (in-memory
  for workstream A; a file/table handle is fine if B needs it), not physical
  "staging tables" — wording now matches the implementation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: record A0 license decision — earnings approved (CC BY-SA 4.0, internal) 35e44f681b

post-no-preference/earnings approved for private/internal ingestion under
CC BY-SA 4.0. Conditions the A2 importer must honor: preserve upstream license /
attribution / transformation notes; no public API, bulk export, or
redistribution; re-review before any public or commercial access. The stocks
repo (workstream B) is not covered and will be reviewed separately if B begins.
A0 rollout item marked done (dolt binary pin + DOLT_DATA_DIR still pending at
deploy time).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

chore: gitignore local Dolt dev clones (dolt-data/) 1b821e1a1e

Local dev uses a Dolt clone of post-no-preference/earnings under dolt-data/
(git-ignored). Production keeps clones in DOLT_DATA_DIR outside the repo tree —
the deploy is rsync --delete of the tree, so a clone inside it would be unsafe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

refactor(dolt): pass run_id to SourceImporter.promote e5a62ca648

Both real target tables (fundamental_snapshots, earnings_events) carry an
import_run_id; stamping requires the current run's id. A2 (the earnings
importer) is the first real consumer, so promote gains a run_id argument rather
than having importers hack the running row out of the framework. Protocol +
call site updated; the A1 fake importer now stamps and asserts import_run_id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(dolt): A2 — DoltHub earnings importer (shadow ingestion) 54ae8ba153

A SourceImporter that ingests post-no-preference/earnings into earnings_events
for the tracked universe. Shadow by construction (nothing reads earnings_events
until A4).

- earnings_alignment.py: pure calendar<->EPS-history min-cost monotonic DP,
  reused from scripts/import_dolthub_earnings.py with identical constants (not
  extending that one-off script); symbol/session normalization; unit-tested
  against the pinned constants.
- dolt_client.py: async dolt CLI wrapper (pull / current_commit / query_csv via
  asyncio.create_subprocess_exec — never blocks the shared event loop) + disk
  guard before pull.
- dolt_earnings_importer.py: detect_revision = pull + HEAD hash; stage = query
  earnings_calendar + eps_history, dedup, align, map act_symbol->ticker_id
  (normalize both sides so dotted BRK.B joins); promote is destructive
  (delete future dolt_earnings rows + upsert; past never deleted) so validate is
  FAIL-CLOSED — blocks when the staged forward calendar is empty or has collapsed
  below 50% of what's loaded (the forward calendar is the acceptance gate).
- NOTICE: CC BY-SA 4.0 attribution; config: DOLT_BINARY / DOLT_DATA_DIR / etc.

Verified end-to-end against the real 1.68 GB clone (5 tickers: 133 events, 128
paired, forward calendar to 2026-08-26, BRK.B joined). Tests: 9 alignment + 7
importer + 1 skip-guarded real-clone smoke. Full suite 699 passed.

Remaining for A2: wire the daily ~02:30 ET shadow cron — deferred to pair with
the deploy-time dolt install + DOLT_DATA_DIR provisioning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(dolt): add shares_outstanding_date to fundamental_snapshots (026) a4e33d7a39

A1 carry-forward. The SEC cover-page share count
(dei:EntityCommonStockSharesOutstanding) is reported "as of" its own date, which
can differ from the fiscal period_end — store that date so market cap uses the
right point-in-time count. Migration 026 edited in place (never run with data).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(dolt): A2 review — subprocess timeouts, stronger initial gate, HASHOF 5d275c8df7

Addresses the A2 review:

1. Every dolt subprocess is now bounded by a hard timeout
   (dolt_command_timeout_seconds, default 600s); on expiry the process is killed
   and DoltError raised — a hung pull/sql can no longer pin the import
   connection and advisory lock indefinitely. Tested (timeout + non-zero exit).
2. Initial-load validate is stronger: besides zero-future, an initial load now
   requires a real forward horizon (>= 21d, under the ~35d observed on the
   clone) AND universe coverage >= 50% (a broken symbol join can't seed a hollow
   calendar). Subsequent runs keep the 50% collapse gate.
3. Revision uses DOLT_HASHOF('HEAD') — formally HEAD, not dolt_log-by-timestamp.
4. Free-disk floor raised 2 GB -> 5 GB (safe headroom over the ~1.7 GB clone).

Full suite 702 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: A3 design pass — SEC fundamentals importer 18fca28b7c

Design (not implementation) for phase A3, grounded in live SEC data probes.
Key findings: fp has no Q4 (derive it); fy/fp are the filing's context not each
fact's period (select by end==reportDate); SEC provides both discrete and YTD
facts (confirms stored-YTD schema); companyfacts endpoint has no ETag/
Last-Modified (conditional GET impossible); tickers are dash-form and GOOG/GOOGL
share one CIK.

Two plan deviations flagged for sign-off:
1. Fetch via the EDGAR daily-index (fetch companyfacts only for tracked issuers
   that filed) rather than the multi-GB bulk zip — lighter and restores the
   revision/no_op model.
2. One snapshot row per accession for its primary period (YTD-cumulative);
   comparative-only restatements out of scope (only real 10-K/A updates a period).

Plus a metric tag catalog, read-time derivation rules (missing period -> null),
CIK resolution, validation gates, and SEC fair-access handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: A3 design signed off — daily-index, primary-period, full backfill d923f41f85

All three decisions approved: fetch via EDGAR daily-index (not bulk zip),
one snapshot per accession for its primary period (comparative-only
restatements out of scope), full-history backfill on first run. Doc status
flipped to approved / ready to implement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: A3 design — four review correctness fixes b1397fa82e

Fold in the A3 design review:

1. Composite revision = latest-index-date + index-content-hash + tracked
   symbol->CIK fingerprint, so a newly added ticker forces a run instead of
   being no_op'd/starved. No backfill sentinel — absence of a prior promoted
   run triggers backfill; source_max_date records the processed index date.
2. Full history needs the paginated submissions shards: filings.recent caps at
   1000; older accessions (reportDate/acceptanceDateTime/isXBRL) live in
   filings.files[] shards (verified on Apple: recent=1000, one 1994-2015 shard).
3. Index<->Company-Facts consistency gate: they are separate SEC products that
   can lag; for every tracked isXBRL index accession, confirm it exists in
   Company Facts before promotion, else fail+retry (never record a null/partial
   snapshot). Non-XBRL amendments skipped with a recorded reason.
4. Immutable = insert-only (ON CONFLICT DO NOTHING); a differing re-fetch is a
   reported discrepancy, never a silent mutation / import_run_id replacement.

Plus deterministic, mutually-exclusive cash/debt composition (aggregate-first;
each source tag counted at most once).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(sec): A3 slice 1 — SEC client + CIK/SIC resolution + composite revision cc67aebe61

First A3 implementation checkpoint (design: docs/dolt-sec-a3-design.md).

- sec_client.py: async SEC EDGAR client honoring fair-access — identifying
  User-Agent (config), request spacing < 10 req/s, exponential backoff on 429,
  and 403 -> SecForbiddenError (alert and stop, never retry-loop). Fetchers:
  company_tickers (normalised, multi-class share CIK), submissions (merges the
  paginated filings.files shards so full history is visible), companyfacts,
  daily_index (fixed-width form.idx parse), latest_index_date.
- sec_universe.py: resolve_ciks (tickers.cik backfill), refresh_sic
  (sic/sic_description), and the composite-revision pieces — universe_fingerprint
  (a new ticker changes the revision, so it's never no_op'd/starved),
  index_content_hash, compose_revision.
- config + .env.example: SEC_USER_AGENT (must be a real contact email) + spacing
  / retries / timeout.

Verified live against real SEC: AAPL->320193, GOOG==GOOGL, BRK-B resolved;
submissions shard-merge proven (131 filings back to 1993); daily index parsed.
Tests: 11 (mocked-transport parsing + 403/429 handling + resolution/fingerprint).
Full suite 713 passed. Next slice: companyfacts -> snapshot parser + importer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(sec): A3 slice-1 review — read-only resolution, error propagation, fair-access 5939be7b7f

Addresses the slice-1 review:

1. Resolution is now read-only (A1 transaction contract). resolve_ciks /
   fetch_sic_updates compute proposals and mutate nothing; a new
   apply_ticker_updates issues the writes, called only in promote — so a failed
   validation can't leak ticker changes on the framework's failure commit.
2. Only 404 means "missing". Added SecNotFoundError; daily_index /
   latest_index_date catch only that. 403, exhausted 429, 5xx, timeouts, and
   transport/parse errors now propagate instead of looking like "no index".
3. Fair-access enforced when opening a REAL client (transport=None): reject
   blank/placeholder/non-email User-Agent and sub-0.11s spacing. Mock transports
   skip it (tests use 0 spacing).
4. submissions(include_history=False) by default — only the one-time full
   backfill fetches the history shards; SIC/incremental work makes no extra
   requests.

Plus: retry transient 5xx/network errors and honor Retry-After during the 1 GB
backfill; compose_revision rejects a missing index date (no "None:..." revision).

Re-verified live vs real SEC (fair-access validation passes, shard merge intact).
Tests: 18 (added error propagation, read-only resolution, fair-access, recent-only
submissions, reject-None revision).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(sec): A3 slice 2a — companyfacts -> snapshot parser 7413de9301

Pure parser (no I/O/DB) turning one issuer's companyfacts + submissions filing
metadata into per-accession snapshot rows for the filing's primary period.

- Period identity from end == reportDate, never fy/fp (fy/fp is the filing's
  context; comparatives repeat it).
- Duration facts stored as cumulative YTD: pick the fact whose span matches the
  fiscal-period-to-date length (Q1~3mo..FY~12mo) within tolerance; no YTD-length
  fact -> null (never a discrete masquerading as YTD).
- Balance-sheet instants at end == reportDate; shares_outstanding is the dei
  cover-page fact whose own end (cover date) is stored in shares_outstanding_date.
- Cash and debt composites are aggregate-first and mutually exclusive (each
  source tag counted at most once).
- Carries filing_date through submissions rows (snapshot.filed_date).

Verified on REAL Apple companyfacts: 44 snapshots, 0 skipped, YTD revenue
124.3B->219.7B->313.7B->416.2B across FY2025 (Q4 derives at read time), every
shares_date is the cover date != period_end. Tests: 6 fixture + 1 skip-guarded
live-invariants (monotonic YTD, cover-date shares).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(sec): A3 slice-2a review — shares fallback, robust context, hardening 4754dbc17b

1. Multi-class shares: prefer the single dei:EntityCommonStockSharesOutstanding
   cover-page fact; else fall back to us-gaap:CommonStockSharesOutstanding at
   period end (Alphabet has no dei fact). Never sum class facts (companyfacts is
   non-dimensional) and never use weighted-average/diluted; conflicting values ->
   null, counted as an "ambiguous shares outstanding" note in validation. Plan's
   "sum class-specific" wording corrected. Verified live: Alphabet shares now
   populate (12.1B), Apple still uses its dei cover date.
2. Fiscal context is the majority (fy, fp) among facts ending at reportDate, with
   ties rejected — no longer the arbitrary first fact.
3. Hardening: catalog selectors require taxonomy == "us-gaap"; indexing drops
   malformed facts (missing accession/end, non-finite value) so a custom concept
   or bad date can't be selected.

Tests: +8 (dei precedence, us-gaap fallback, conflict->null, no weighted-average,
tie-context skip, foreign-taxonomy/malformed ignored, ambiguous-shares note).
14 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(sec): A3 slice 2b — SEC fundamentals importer (shadow ingestion) f7ce85a33e

SecFundamentalsImporter (SourceImporter, source=sec_facts): populates immutable
fundamental_snapshots from Company Facts and back-fills tickers.cik/sic, driven
by the EDGAR daily index. Shadow only. Guardrails per review:

- detect_revision caches the resolved universe + exact tracked index rows and
  composes the revision from them; stage consumes those same cached inputs
  (no index/universe refetch) so promoted data matches the computed revision.
- Resolution is read-only in stage (proposals only); ticker writes happen in
  promote via apply_ticker_updates.
- validate runs the index<->Company-Facts consistency gate before any write:
  a tracked XBRL index accession missing from Company Facts fails the run
  (they lag independently) so we retry, not record null. Non-XBRL amendments
  are skipped with a recorded reason. Backfill has a coverage floor.
- promote inserts ON CONFLICT (accession) DO NOTHING (immutable), reports
  differing existing accessions without mutating, and applies ticker updates in
  the same transaction.
- Full-history backfill on first run / for newly-added issuers (include_history);
  incremental fetch only for issuers that filed.

Parser: split parse result into skipped_filings vs field_issues (coverage must
not count field warnings); header notes the us-gaap shares fallback; added
companyfacts_accessions() for the gate.

Verified live end-to-end (AAPL + GOOGL backfill): 112 snapshots, cik/sic set,
GOOGL shares via us-gaap fallback, AAPL via dei. Tests: 6 importer (backfill,
incremental, consistency-gate fail, non-XBRL skip, read-only-on-failure,
conflict-discrepancy) + parser ParseResult updates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(sec): A3 slice-2b review — no index cap, full discrepancy + malformed gate 8dcdcac2a6

1. Removed the 45-day index-walk cap: it discarded the older part of a long
   outage while still advancing source_max_date, permanently losing filings.
   The walk now covers every unprocessed date (a large gap is one-time cost).
2. Discrepancy detection meets the immutability contract: it compares ALL source
   snapshot fields (not five), read-only during stage/validate, reports the
   differing accessions + fields in validation_json, and promote emits a warning
   system event (in-transaction) — never mutating the stored row.
3. Malformed companyfacts (missing facts/units structure) are recorded separately
   and FAIL validation, instead of silently degrading to skipped rows that the
   50% backfill coverage floor could still pass.

Also corrected the stale "sum share classes" / DEI-only wording in the snapshot
model docstring and the A3 design doc to describe the us-gaap fallback.

Tests: +4 regressions (>45-day gap loses nothing, newly-added issuer backfills
without filing, malformed payload fails, shares discrepancy detected + evented).
23 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(sec): A3 sign-off hardening — validate per-concept units structure 96d3b40560

- Extend the companyfacts structural check to reject a concept with a
  missing/non-dict `units` mapping (not just the top-level `facts`), so a
  partially-malformed payload fails promotion instead of silently dropping that
  concept's facts. New fixture proves it fails.
- Strengthen the newly-added-issuer test: keep latest_index equal to the prior
  run so ONLY the universe fingerprint changes the revision — proving the
  fingerprint alone prevents a new ticker from being starved/no_op'd.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(fundamentals): A4a — pure read-time metric derivation a549942afe

Derives the display metrics from the stored YTD snapshots at read time (no I/O,
no DB), per the A3 schema decision. Given an issuer's snapshot rows it produces:

- amendment selection (newest accepted_at per fiscal period);
- discrete quarters = YTD(Qn) - YTD(Qn-1), Q4 = YTD(FY) - YTD(Q3);
- TTM = trailing four discrete quarters; missing period -> null, never partial;
- metric series (value + 4-quarter tape, each point dated): revenue_growth_yoy,
  eps_growth_yoy, operating_margin, fcf_margin, net_debt, net_debt_to_ebitda,
  share_count_change_yoy;
- request-time valuation inputs (ttm_diluted_eps, ttm_fcf, shares_outstanding)
  for the API to combine with price.

Units per app convention (percentages = pp, leverage = multiple, dollars).
Tests: 6 (growth+Q4, margins, net-debt/EBITDA+dilution, valuation inputs,
missing-period-null, amendment selection). Verified on real Apple snapshots:
op margin 32.6%, net-debt/EBITDA 0.10, buyback -1.7%/yr, TTM EPS $8.26.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(fundamentals): A4a review — stricter null semantics in derivation 256880899b

1. net_debt requires BOTH cash and total_debt; a missing side is null, not
   treated as zero (which would be a partial, misleading value).
2. net_debt_to_ebitda is null when TTM EBITDA <= 0 — a negative denominator
   would otherwise rank a distressed issuer as favorably low-leverage.
3. The quarter tape is the CONSECUTIVE run ending at the latest period (stops at
   a gap), so trend text never compares non-adjacent quarters as if consecutive.
4. YoY growth is null when the prior-year TTM is <= 0 (e.g. loss->profit), which
   is not a meaningful percentage.

Also corrected the plan's net-debt formula to total debt − (cash + ST) matching
the positive-means-net-debt implementation. +4 tests. 10 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(fundamentals): A4 — pure peer comparison + deterministic reads 2038b84b72

Completes the pure read-time core.

fundamentals_peers.py: median + polarity-aware favorable percentile + peer_count
for a subject within its SIC group (CIK-deduped by the caller); returns None
below MIN_PEERS=5 so the caller omits the industry object. Absolute net_debt is
intentionally NOT peer-eligible (size-dependent) — leverage compares via
net_debt_to_ebitda. HIGHER_IS_BETTER polarity map + two_digit_sic() grouping key.

fundamentals_reads.py: one shared deterministic rule set (no LLM): growth_read
(+-2pp), margin_read (latest vs mean-of-prior, +-1pp), share_count_read (+-1%),
peer_read (60/40 bands, polarity-aware phrasing per metric), header_sentence
(growth · margins · valuation, omitting empty). Tunable named constants; >=3
periods required for a series read.

Tests: 8 peer + 5 reads, anchored on the boundary cases (exactly +2.0pp, exactly
60th percentile, exactly +1.0pp margin). 13 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(fundamentals): pure-core review — tie-aware percentile, null-safe reads 979b4047dc

1. Peer percentile is now a tie-aware rank against the OTHER issuers
   ((worse + 0.5*tied)/(peers-1)): an all-equal group maps to 50 (not 100), the
   median maps to 50, a unique best to 100, a unique worst to 0.
2. Deterministic reads use the consecutive non-null suffix ending at the latest
   point (>=3 values): a null latest or an internal gap yields no read, so a read
   never reflects a period displayed as n/a.
3. Peer filtering excludes non-finite (NaN/±inf) as well as null, including an
   invalid subject.

Tests updated + added (all-equal, median rank, non-finite, latest-null history,
internal gap). 15 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(fundamentals): A4 — additive API v1 (earnings, metrics, valuation, reads) 459a925e36

GET /fundamentals/{symbol} now returns the additive v1 objects alongside the
unchanged legacy fields (no legacy growth mapped onto the SEC TTM metric).

- earnings: next (date/session/days_until) + recent (<=4, with surprise_pct)
  from earnings_events.
- metrics: fixed key set (value + dated history + per-metric SIC-peer industry
  object + source=sec); net_debt has no industry (size-dependent).
- valuation: P/E, FCF yield, market_cap_est computed at REQUEST TIME from the
  derived TTM inputs x the latest ohlcv close (no stored valuation); guarded to
  null on missing/invalid inputs; pe_industry / fcf_yield_industry peer stats.
- reads: deterministic outputs in a SEPARATE object (header + per-metric reads).

Peer queries are batched and CIK-deduplicated by 2-digit SIC; industry omitted
below 5 valid peers. Schema extended with optional typed sub-models; the router
merges legacy + v1 so every existing field is preserved.

Tests: 4 (full assembly incl. peer industry + valuation + additive-merge, no-cik
null metrics, <5-peers omitted, price-guarded valuation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(fundamentals): API v1 review — multi-class pricing, reads contract, guards b3dcf356a6

1. Multi-class subject is priced by the REQUESTED ticker: the peer group's
   representative for the subject CIK is overridden to the requested ticker_id
   (other issuers pick a deterministic-by-symbol rep), so GOOGL's P/E uses
   GOOGL's price, not GOOG's. Differing-price GOOG/GOOGL test added.
2. reads matches the selected contract: header is null when there is no read;
   by_key is a fixed map over every metric key plus pe and fcf_yield, null when
   unavailable (was a sparse dict).
3. Earnings use the New York calendar date; same-day is UPCOMING (days_until 0),
   recent is strictly earlier.
4. Valuation is null when there is no usable price (> 0 required for P/E and
   market cap); when present, price_date is non-null.

Added a real router/API-envelope test with a seeded legacy record (the endpoint,
not just the schema merge). 6 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(fundamentals): compute eps_growth_yoy read; cover same-day + price guards fb6d39c68b

- The eps_growth_yoy read was never computed, leaving that fixed by_key entry
  null even with sufficient EPS history; now growth_read() is applied to EPS
  history just like revenue.
- Tests: same-day earnings returns as next with days_until 0 (and not in
  recent); zero close guards valuation to null; eps read populated. Fixture
  seeds three fiscal years so YoY growth reads have a >=3 run. 9 API tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(frontend): A4 — FundamentalsPanel v1 (quarter tape + earnings + peers) 9172c1a699

Reshapes FundamentalsPanel to consume the additive API v1, within the app's
existing dark-glass language.

- types.ts updated to the exact v1 shape (metrics/earnings/valuation/reads +
  legacy fields preserved).
- The quarter tape is the single distinctive device: per-metric 4-cell tape
  (revenue/EPS growth, operating + FCF margin, share count) with the latest cell
  toned by the deterministic read; color is always paired with the read text.
- Restrained peer strips for Net debt/EBITDA, P/E, FCF yield: value + a
  polarity-aware percentile bar with a median marker + the read; hidden ("peers
  n/a") when industry is null (< 5 peers).
- Earnings: next date/session/countdown + last-N beat/miss arrows (▲/▼/·) with
  text aria-labels; explicit "no date" state.
- Explicit n/a, insufficient-peer, and no-earnings states; header shows the
  deterministic sentence. Removed the hard-coded "FMP" source label.

Frontend tsc -b passes; backend suite 778 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(frontend): FundamentalsPanel review — mobile, local dates, peer a11y bd23f41a1d

Addresses the static review + adds a dev-only visual harness:

1. Tape no longer overflows narrow mobile: each row stacks (label + read on one
   line, cells below) under sm, keeping the single-line grid on desktop.
2. Date-only strings (earnings, price_date) are parsed as LOCAL calendar dates,
   so a viewer west of UTC no longer sees the previous day.
3. Peer context is visible ("med X · Np") on every width and the percentile
   strip carries a full aria-label — no longer hover-only / desktop-only.
4. Per-metric provenance + freshness surfaced (SEC filings · latest quarter,
   filed date) replacing the removed panel-wide FMP label.
5. Same-day earnings render "today", not "in 0d".

Harness: frontend/harness.html + src/dev/harness.tsx (dev-only, served at
/harness.html by vite, not in the production build) render full /
partial-insufficient-peer / empty fixtures for desktop + ~390px review.

tsc -b passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(frontend): FundamentalsPanel — Reference Rails redesign 103b182598

Replace the four-cell quarter tape (too many equally-weighted numbers) with one
comparison rail per metric, so the panel answers "improving? sound? fairly
valued?" instead of asking the reader to decode it.

- Operating trend (revenue/EPS growth, operating/FCF margin, share count): a rail
  centered on a truthful reference — prior quarter (growth), prior-period average
  (margins), or zero (share count) — with a dot at the current delta and a bar
  back to the reference, plus a shaded neutral band (backend's +-2pp / +-1pp /
  +-1% rules). Value, read, reference label, and signed delta stay visible;
  per-quarter history drops out of the default view.
- Valuation & balance (net debt/EBITDA, P/E, FCF yield): a 0-100 favorable-
  percentile rail with the peer median fixed at 50; right is always more
  favorable (percentile is polarity-aware). median + peer_count shown.
- Not a progress bar: reference line, not a 100% target.
- Horizon tokens: cyan #6EC9DB favorable / coral #EF9182 adverse / #5D6373 track,
  replacing emerald/rose. Two columns on desktop, single column (rows stack) on
  mobile. Null -> n/a with no rail; insufficient peers -> "peers n/a", no track.
- Kept: compact earnings line, provenance footer, local-date parsing, aria-labels
  on every rail. tsc -b passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

chore(frontend): harness at desktop width (768px) for two-column preview 480baf762f

max-w-md (448px) under-sized the panel vs its real full-width tab placement and
never triggered the desktop two-column layout. Widen to max-w-3xl; note to
resize to ~390px for the mobile (single-column) check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(frontend): finish fundamentals reference rails 71d21a45b6

feat(deploy): schedule fundamentals shadow imports 88fc90cc6f

dennisthiessen merged commit b68aff80a6 into main

2026-07-23 13:27:08 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dennisthiessen/signal-platform#1