fix: don't double-report a failed SEC run, and correct the rollback doc
Deploy / lint (push) Successful in 11s
Deploy / test (push) Successful in 1m24s
Deploy / deploy (push) Successful in 41s

Three follow-ups from review of the A6 commits.

The cache-summary re-finalize raised a second durable system event on a failed
run: _runtime_finish emits for `error`/`rate_limited`, and the dedup key
includes the message, so "SEC unavailable" and "SEC unavailable · cache 511 · 2
score inputs changed" landed as two unacknowledged Admin events. Adds
`emit_event` so a re-finalize that only rewords an outcome stays silent, with a
regression test asserting exactly one event.

The rollback section still claimed disabling the SEC job freezes the cache — the
opposite of what the same page says two lines earlier, and of what the code now
does. Rewritten: there is no Admin cache-off switch, restoring `fundamental_data`
alone is temporary because the next run rebuilds it from the same snapshots and
code, and a real freeze means stopping the service.

Remaining "shadow" wording: the two import jobs have never been shadow since
activation, so `_run_shadow_import` -> `_run_source_import`, its section heading,
the deployment doc's job label, and the plan doc's "production switch remains"
handoff paragraph are all brought up to date.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-07 11:46:34 +02:00
co-authored by Claude Opus 5
parent 13f3636b6a
commit e1607ddbff
4 changed files with 63 additions and 36 deletions
+11 -13
View File
@@ -496,20 +496,18 @@ Post-fix: candidate scores 504 of 511 vs legacy's 507 (gap = PSKY/Q new registra
FITB, all explained); revenue-growth agreement 0.0038 median abs delta where both exist.
Dennis reviewed the evidence 2026-07-24 and directed proceeding to cutover.
**Task 1 — A5 activation (IMPLEMENTED 2026-07-24; production switch remains).** The
post-activation local refresh of `fundamental_data` derives `pe_ratio` and
`market_cap` from newest valid snapshots × latest PostgreSQL close, `revenue_growth`
from snapshots, `earnings_surprise`/`next_earnings_date` from `earnings_events`; mark
affected cached fundamental scores stale; must run identically when SEC is unreachable.
**Task 1 — A5 activation: DONE.** Implemented 2026-07-24, switched on and observed
in production, and made unconditional by A6 (2026-08-07) — there is no longer a
switch, an Admin card, or a weekly legacy collector to skip. The local refresh of
`fundamental_data` derives `pe_ratio` and `market_cap` from newest valid snapshots ×
latest PostgreSQL close, `revenue_growth` from snapshots, and
`earnings_surprise`/`next_earnings_date` from `earnings_events`; it marks affected
cached fundamental scores stale and runs identically when SEC is unreachable.
It consumes `fundamentals_derivation.derive()` outputs, NOT raw snapshot fields —
that path carries the split guard (`ttm_diluted_eps`
nulls when contaminated, with `ttm_diluted_eps_caveat`) and the multi-class share
fallback (`shares_outstanding` + `shares_outstanding_estimated`). Parity and activation
share the same candidate builder. Activation is the explicit
`fundamental_data_sec_dolt_cutover_enabled` SystemSetting and defaults off. It is
managed by the **Fundamentals data source** card in Admin → Settings; while active,
the weekly legacy collector skips itself so it cannot overwrite the SEC/Dolt cache.
See `docs/fundamentals-deployment.md` for the production flip and rollback procedure.
that path carries the split guard (`ttm_diluted_eps` nulls when contaminated, with
`ttm_diluted_eps_caveat`) and the multi-class share fallback (`shares_outstanding` +
`shares_outstanding_estimated`). See `docs/fundamentals-deployment.md` for current
operations and rollback.
**Task 2 — A6 decommissioning: DONE 2026-08-07.** The cutover ran on and was
observed in production, so the legacy providers, their config/env keys, the weekly
+17 -9
View File
@@ -86,7 +86,7 @@ a reviewed change to `DOLT_VERSION`, followed by the same provision/check flow.
In Admin → Jobs, wait until no other job is running, then:
1. Trigger **Dolt Earnings Import (shadow)**. Expect `completed` with import
1. Trigger **Dolt Earnings Import**. Expect `completed` with import
status `promoted`; a repeat without an upstream change should report `no_op`.
2. Trigger **SEC Fundamentals Import**. The first run performs the
tracked-universe history backfill and can take materially longer than a daily
@@ -197,14 +197,22 @@ rollback window has closed, delete the rows and the `MANAGED_SETTINGS` filter in
## Failure and rollback
- **There is no provider fallback any more.** To recover bad `fundamental_data`
values, restore the table from the PostgreSQL backup; the next scheduled run
will rebuild it from the current snapshots. Confirm a recent backup exists
before any change that could corrupt the snapshots.
- Disable a failing source-import job in Admin → Jobs only when SEC network
access itself must stop — the local cache refresh keeps running, and existing
promoted snapshots/events remain available. To freeze the cache as well,
disable the job *and* accept that P/E, market cap and earnings dates go stale.
- **There is no provider fallback any more, and no Admin switch that freezes the
cache.** Disabling **SEC Fundamentals Import** stops SEC network access only;
the 04:00 job still rebuilds `fundamental_data` from the stored snapshots,
earnings events and closes.
- Restoring `fundamental_data` from the PostgreSQL backup is therefore a
*temporary* fix on its own: if the bad values come from the snapshots or from
the derivation code, the next scheduled run reproduces them. Fix the cause —
restore or repair `fundamental_snapshots` / `earnings_events`, or revert the
parser change and re-run `scripts/reparse_fundamentals.py --apply`.
- To genuinely freeze the cache while you work, stop the service
(`sudo systemctl stop signalplatform.service`) — that stops the scheduler with
it. There is no finer-grained control, by design: a silently frozen scoring
input is worse than an obvious outage.
- Disable a failing source-import job in Admin → Jobs when SEC network access
itself must stop. Existing promoted snapshots and events remain available, and
the job's runtime message still reports the cache result.
- Inspect the job runtime, latest `data_import_runs.validation_json`, service
logs, and Admin → System Events before retrying.
- `unresolved_filing` is emitted once when a filing enters automatic retry. It