dennisthiessenandClaude Opus 4.8
949cbbe7c0
First reviewable slice of workstream A: schema only, no importers, no data. - data_import_runs: lean batch-import audit (source/revision/status, row_counts_json + validation_json as Text-holding-JSON per repo convention). - fundamental_snapshots: CIK-keyed, one immutable row per accession; stores per-period raw facts (duration = cumulative YTD/FY, balance-sheet = period-end) plus period_start/period_end/fiscal_year/fiscal_period so discrete quarters, Q4, TTM and YoY are derived at read time. - earnings_events: Dolt-sourced calendar + surprise history, unique (ticker_id, announce_date). - tickers: nullable cik/sic/sic_description — the ticker<->issuer join point. fundamental_data is left untouched (cutover gated separately at A5). Models registered in app/models/__init__.py; Ticker gains an earnings_events relationship. Verified: create_all builds the tables, mappers configure, and migration 026 renders valid Postgres DDL up and down. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
43 lines
1.9 KiB
Python
43 lines
1.9 KiB
Python
from datetime import date, datetime
|
|
|
|
from sqlalchemy import Date, DateTime, Index, String, Text
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from app.database import Base
|
|
|
|
|
|
class DataImportRun(Base):
|
|
"""One row per bulk-import attempt (SEC facts / Dolt earnings / Dolt stocks).
|
|
|
|
Lean audit record for the batch import framework: every attempt is logged,
|
|
whether it promoted, was a ``no_op`` (unchanged revision), or ``failed``.
|
|
``row_counts`` and ``validation`` hold JSON strings (repo convention — see
|
|
``fundamental_data.unavailable_fields_json``), not JSONB; the validation
|
|
blob carries reconciliation/discrepancy summaries so no separate conflicts
|
|
table is needed. One run per source at a time is enforced at write time by a
|
|
Postgres advisory lock keyed by ``source``.
|
|
"""
|
|
|
|
__tablename__ = "data_import_runs"
|
|
__table_args__ = (
|
|
Index("ix_data_import_runs_source_started", "source", "started_at"),
|
|
)
|
|
|
|
id: Mapped[int] = mapped_column(primary_key=True)
|
|
# sec_facts | dolt_earnings | dolt_stocks
|
|
source: Mapped[str] = mapped_column(String(32), nullable=False)
|
|
# Dolt commit hash, or SEC archive SHA-256. Null until known.
|
|
revision: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
# running | validated | promoted | no_op | failed
|
|
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
|
source_max_date: Mapped[date | None] = mapped_column(Date, nullable=True)
|
|
row_counts_json: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
validation_json: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
started_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), default=datetime.utcnow, nullable=False
|
|
)
|
|
completed_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True
|
|
)
|
|
error_details: Mapped[str | None] = mapped_column(Text, nullable=True)
|