dennisthiessenandClaude Opus 5
3e83d63b05
The A5 cutover has been on and observed in production, so SEC Company Facts + DoltHub earnings are already the live source for `fundamental_data`. This removes everything the legacy path still occupied. Gone: the three providers and their config/env keys; the weekly `fundamental_collector` job; the cutover toggle (SEC + Dolt is now the unconditional path, so `off` can no longer silently freeze scoring inputs); the A5 parity report, whose deltas became structurally zero once the candidate builder started writing the table it compared against; and the FMP tier of universe bootstrap. Two behavioral notes: - Disabling **SEC Fundamentals Import** now stops the SEC network fetch only. The local cache refresh moved outside the job-enable check, because candidates also derive from daily closes and earnings events — freezing those on an ingestion pause would stale scoring with no fallback left to recover from. - `/ingestion/fetch?sources=fundamentals` still accepts the key and reports `skipped`; there is no per-ticker fetch any more. Migration 029 does not blanket-delete the leftover settings rows. Migrations run before the service restart, and pre-A6 code reads an absent `job_*_enabled` row as *enabled* — so the two behavior-bearing keys become tombstones pinned to safe values (hidden in Admin) and only the inert three are deleted. Removing the provider keys from the production `.env` is the matching rollout step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
97 lines
3.3 KiB
Python
97 lines
3.3 KiB
Python
"""Retire the legacy fundamentals settings (A6)
|
|
|
|
Revision ID: 029
|
|
Revises: 028
|
|
Create Date: 2026-08-07 00:00:00.000000
|
|
|
|
A6 removed the FMP/Finnhub/Alpha Vantage providers, the weekly
|
|
``fundamental_collector`` job and the A5 parity report. Five SystemSetting rows
|
|
are left over. They are NOT all deleted, because the deploy runs migrations
|
|
before restarting the service: for a short window — and for the whole of any
|
|
rollback — pre-A6 code is still live, and it reads absent rows permissively
|
|
(cutover absent -> disabled; ``job_<name>_enabled`` absent -> enabled). Deleting
|
|
both would hand a rolled-back process a re-armed legacy collector writing over
|
|
the SEC/Dolt cache.
|
|
|
|
So the two rows that carry behavior become tombstones pinned to the safe value,
|
|
and only the inert ones are deleted. The tombstones are dropped in a later
|
|
release once the rollback window has closed; ``SettingsForm`` hides them
|
|
meanwhile.
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
from alembic import op
|
|
import sqlalchemy as sa
|
|
|
|
|
|
revision: str = "029"
|
|
down_revision: Union[str, None] = "028"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
# Behavior-bearing under pre-A6 code -> pin to the safe value, keep the row.
|
|
_TOMBSTONES: dict[str, str] = {
|
|
"fundamental_data_sec_dolt_cutover_enabled": "true",
|
|
"job_fundamental_collector_enabled": "false",
|
|
}
|
|
|
|
# Inert either way: an absent cron falls back to a default for a job that no
|
|
# longer registers, and the parity report never wrote anything.
|
|
_OBSOLETE: tuple[str, ...] = (
|
|
"schedule_fundamentals_cron",
|
|
"schedule_fundamentals_parity_cron",
|
|
"job_fundamentals_parity_report_enabled",
|
|
)
|
|
|
|
_settings = sa.table(
|
|
"system_settings",
|
|
sa.column("id", sa.Integer),
|
|
sa.column("key", sa.String),
|
|
sa.column("value", sa.Text),
|
|
sa.column("updated_at", sa.DateTime(timezone=True)),
|
|
)
|
|
|
|
|
|
def upgrade() -> None:
|
|
conn = op.get_bind()
|
|
now = sa.func.now()
|
|
|
|
for key, pinned in _TOMBSTONES.items():
|
|
row = conn.execute(
|
|
sa.select(_settings.c.value).where(_settings.c.key == key)
|
|
).fetchone()
|
|
old_value = row[0] if row is not None else None
|
|
print(f"a6_tombstone {key}: {old_value!r} -> {pinned!r}", flush=True)
|
|
if row is None:
|
|
conn.execute(
|
|
sa.insert(_settings).values(key=key, value=pinned, updated_at=now)
|
|
)
|
|
elif old_value != pinned:
|
|
conn.execute(
|
|
sa.update(_settings)
|
|
.where(_settings.c.key == key)
|
|
.values(value=pinned, updated_at=now)
|
|
)
|
|
|
|
# Print the value before deleting — a bare DELETE cannot be undone from the
|
|
# migration output.
|
|
for key in _OBSOLETE:
|
|
row = conn.execute(
|
|
sa.select(_settings.c.value).where(_settings.c.key == key)
|
|
).fetchone()
|
|
if row is None:
|
|
print(f"a6_delete {key}: absent", flush=True)
|
|
continue
|
|
print(f"a6_delete {key}: {row[0]!r}", flush=True)
|
|
conn.execute(sa.delete(_settings).where(_settings.c.key == key))
|
|
|
|
|
|
def downgrade() -> None:
|
|
"""No-op.
|
|
|
|
The deleted rows configured jobs this revision's code no longer registers,
|
|
and the tombstones already hold the values pre-A6 code needs. Recreating
|
|
them would restore nothing useful; the printed values above cover recovery.
|
|
"""
|