chore: decommission FMP, Finnhub and Alpha Vantage (A6)
The A5 cutover has been on and observed in production, so SEC Company Facts + DoltHub earnings are already the live source for `fundamental_data`. This removes everything the legacy path still occupied. Gone: the three providers and their config/env keys; the weekly `fundamental_collector` job; the cutover toggle (SEC + Dolt is now the unconditional path, so `off` can no longer silently freeze scoring inputs); the A5 parity report, whose deltas became structurally zero once the candidate builder started writing the table it compared against; and the FMP tier of universe bootstrap. Two behavioral notes: - Disabling **SEC Fundamentals Import** now stops the SEC network fetch only. The local cache refresh moved outside the job-enable check, because candidates also derive from daily closes and earnings events — freezing those on an ingestion pause would stale scoring with no fallback left to recover from. - `/ingestion/fetch?sources=fundamentals` still accepts the key and reports `skipped`; there is no per-ticker fetch any more. Migration 029 does not blanket-delete the leftover settings rows. Migrations run before the service restart, and pre-A6 code reads an absent `job_*_enabled` row as *enabled* — so the two behavior-bearing keys become tombstones pinned to safe values (hidden in Admin) and only the inert three are deleted. Removing the provider keys from the production `.env` is the matching rollout step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,96 @@
|
||||
"""Retire the legacy fundamentals settings (A6)
|
||||
|
||||
Revision ID: 029
|
||||
Revises: 028
|
||||
Create Date: 2026-08-07 00:00:00.000000
|
||||
|
||||
A6 removed the FMP/Finnhub/Alpha Vantage providers, the weekly
|
||||
``fundamental_collector`` job and the A5 parity report. Five SystemSetting rows
|
||||
are left over. They are NOT all deleted, because the deploy runs migrations
|
||||
before restarting the service: for a short window — and for the whole of any
|
||||
rollback — pre-A6 code is still live, and it reads absent rows permissively
|
||||
(cutover absent -> disabled; ``job_<name>_enabled`` absent -> enabled). Deleting
|
||||
both would hand a rolled-back process a re-armed legacy collector writing over
|
||||
the SEC/Dolt cache.
|
||||
|
||||
So the two rows that carry behavior become tombstones pinned to the safe value,
|
||||
and only the inert ones are deleted. The tombstones are dropped in a later
|
||||
release once the rollback window has closed; ``SettingsForm`` hides them
|
||||
meanwhile.
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = "029"
|
||||
down_revision: Union[str, None] = "028"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
# Behavior-bearing under pre-A6 code -> pin to the safe value, keep the row.
|
||||
_TOMBSTONES: dict[str, str] = {
|
||||
"fundamental_data_sec_dolt_cutover_enabled": "true",
|
||||
"job_fundamental_collector_enabled": "false",
|
||||
}
|
||||
|
||||
# Inert either way: an absent cron falls back to a default for a job that no
|
||||
# longer registers, and the parity report never wrote anything.
|
||||
_OBSOLETE: tuple[str, ...] = (
|
||||
"schedule_fundamentals_cron",
|
||||
"schedule_fundamentals_parity_cron",
|
||||
"job_fundamentals_parity_report_enabled",
|
||||
)
|
||||
|
||||
_settings = sa.table(
|
||||
"system_settings",
|
||||
sa.column("id", sa.Integer),
|
||||
sa.column("key", sa.String),
|
||||
sa.column("value", sa.Text),
|
||||
sa.column("updated_at", sa.DateTime(timezone=True)),
|
||||
)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
now = sa.func.now()
|
||||
|
||||
for key, pinned in _TOMBSTONES.items():
|
||||
row = conn.execute(
|
||||
sa.select(_settings.c.value).where(_settings.c.key == key)
|
||||
).fetchone()
|
||||
old_value = row[0] if row is not None else None
|
||||
print(f"a6_tombstone {key}: {old_value!r} -> {pinned!r}", flush=True)
|
||||
if row is None:
|
||||
conn.execute(
|
||||
sa.insert(_settings).values(key=key, value=pinned, updated_at=now)
|
||||
)
|
||||
elif old_value != pinned:
|
||||
conn.execute(
|
||||
sa.update(_settings)
|
||||
.where(_settings.c.key == key)
|
||||
.values(value=pinned, updated_at=now)
|
||||
)
|
||||
|
||||
# Print the value before deleting — a bare DELETE cannot be undone from the
|
||||
# migration output.
|
||||
for key in _OBSOLETE:
|
||||
row = conn.execute(
|
||||
sa.select(_settings.c.value).where(_settings.c.key == key)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
print(f"a6_delete {key}: absent", flush=True)
|
||||
continue
|
||||
print(f"a6_delete {key}: {row[0]!r}", flush=True)
|
||||
conn.execute(sa.delete(_settings).where(_settings.c.key == key))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""No-op.
|
||||
|
||||
The deleted rows configured jobs this revision's code no longer registers,
|
||||
and the tombstones already hold the values pre-A6 code needs. Recreating
|
||||
them would restore nothing useful; the printed values above cover recovery.
|
||||
"""
|
||||
Reference in New Issue
Block a user