feat(dolt): migration 026 + models — fundamentals/earnings schema (A1 schema)
First reviewable slice of workstream A: schema only, no importers, no data. - data_import_runs: lean batch-import audit (source/revision/status, row_counts_json + validation_json as Text-holding-JSON per repo convention). - fundamental_snapshots: CIK-keyed, one immutable row per accession; stores per-period raw facts (duration = cumulative YTD/FY, balance-sheet = period-end) plus period_start/period_end/fiscal_year/fiscal_period so discrete quarters, Q4, TTM and YoY are derived at read time. - earnings_events: Dolt-sourced calendar + surprise history, unique (ticker_id, announce_date). - tickers: nullable cik/sic/sic_description — the ticker<->issuer join point. fundamental_data is left untouched (cutover gated separately at A5). Models registered in app/models/__init__.py; Ticker gains an earnings_events relationship. Verified: create_all builds the tables, mappers configure, and migration 026 renders valid Postgres DDL up and down. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
from datetime import date, datetime
|
||||
|
||||
from sqlalchemy import Date, DateTime, Index, String, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class DataImportRun(Base):
|
||||
"""One row per bulk-import attempt (SEC facts / Dolt earnings / Dolt stocks).
|
||||
|
||||
Lean audit record for the batch import framework: every attempt is logged,
|
||||
whether it promoted, was a ``no_op`` (unchanged revision), or ``failed``.
|
||||
``row_counts`` and ``validation`` hold JSON strings (repo convention — see
|
||||
``fundamental_data.unavailable_fields_json``), not JSONB; the validation
|
||||
blob carries reconciliation/discrepancy summaries so no separate conflicts
|
||||
table is needed. One run per source at a time is enforced at write time by a
|
||||
Postgres advisory lock keyed by ``source``.
|
||||
"""
|
||||
|
||||
__tablename__ = "data_import_runs"
|
||||
__table_args__ = (
|
||||
Index("ix_data_import_runs_source_started", "source", "started_at"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
# sec_facts | dolt_earnings | dolt_stocks
|
||||
source: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
# Dolt commit hash, or SEC archive SHA-256. Null until known.
|
||||
revision: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
# running | validated | promoted | no_op | failed
|
||||
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
source_max_date: Mapped[date | None] = mapped_column(Date, nullable=True)
|
||||
row_counts_json: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
validation_json: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
started_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), default=datetime.utcnow, nullable=False
|
||||
)
|
||||
completed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
error_details: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
Reference in New Issue
Block a user