fix: don't double-report a failed SEC run, and correct the rollback doc
Three follow-ups from review of the A6 commits. The cache-summary re-finalize raised a second durable system event on a failed run: _runtime_finish emits for `error`/`rate_limited`, and the dedup key includes the message, so "SEC unavailable" and "SEC unavailable · cache 511 · 2 score inputs changed" landed as two unacknowledged Admin events. Adds `emit_event` so a re-finalize that only rewords an outcome stays silent, with a regression test asserting exactly one event. The rollback section still claimed disabling the SEC job freezes the cache — the opposite of what the same page says two lines earlier, and of what the code now does. Rewritten: there is no Admin cache-off switch, restoring `fundamental_data` alone is temporary because the next run rebuilds it from the same snapshots and code, and a real freeze means stopping the service. Remaining "shadow" wording: the two import jobs have never been shadow since activation, so `_run_shadow_import` -> `_run_source_import`, its section heading, the deployment doc's job label, and the plan doc's "production switch remains" handoff paragraph are all brought up to date. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -496,20 +496,18 @@ Post-fix: candidate scores 504 of 511 vs legacy's 507 (gap = PSKY/Q new registra
|
||||
FITB, all explained); revenue-growth agreement 0.0038 median abs delta where both exist.
|
||||
Dennis reviewed the evidence 2026-07-24 and directed proceeding to cutover.
|
||||
|
||||
**Task 1 — A5 activation (IMPLEMENTED 2026-07-24; production switch remains).** The
|
||||
post-activation local refresh of `fundamental_data` derives `pe_ratio` and
|
||||
`market_cap` from newest valid snapshots × latest PostgreSQL close, `revenue_growth`
|
||||
from snapshots, `earnings_surprise`/`next_earnings_date` from `earnings_events`; mark
|
||||
affected cached fundamental scores stale; must run identically when SEC is unreachable.
|
||||
**Task 1 — A5 activation: DONE.** Implemented 2026-07-24, switched on and observed
|
||||
in production, and made unconditional by A6 (2026-08-07) — there is no longer a
|
||||
switch, an Admin card, or a weekly legacy collector to skip. The local refresh of
|
||||
`fundamental_data` derives `pe_ratio` and `market_cap` from newest valid snapshots ×
|
||||
latest PostgreSQL close, `revenue_growth` from snapshots, and
|
||||
`earnings_surprise`/`next_earnings_date` from `earnings_events`; it marks affected
|
||||
cached fundamental scores stale and runs identically when SEC is unreachable.
|
||||
It consumes `fundamentals_derivation.derive()` outputs, NOT raw snapshot fields —
|
||||
that path carries the split guard (`ttm_diluted_eps`
|
||||
nulls when contaminated, with `ttm_diluted_eps_caveat`) and the multi-class share
|
||||
fallback (`shares_outstanding` + `shares_outstanding_estimated`). Parity and activation
|
||||
share the same candidate builder. Activation is the explicit
|
||||
`fundamental_data_sec_dolt_cutover_enabled` SystemSetting and defaults off. It is
|
||||
managed by the **Fundamentals data source** card in Admin → Settings; while active,
|
||||
the weekly legacy collector skips itself so it cannot overwrite the SEC/Dolt cache.
|
||||
See `docs/fundamentals-deployment.md` for the production flip and rollback procedure.
|
||||
that path carries the split guard (`ttm_diluted_eps` nulls when contaminated, with
|
||||
`ttm_diluted_eps_caveat`) and the multi-class share fallback (`shares_outstanding` +
|
||||
`shares_outstanding_estimated`). See `docs/fundamentals-deployment.md` for current
|
||||
operations and rollback.
|
||||
|
||||
**Task 2 — A6 decommissioning: DONE 2026-08-07.** The cutover ran on and was
|
||||
observed in production, so the legacy providers, their config/env keys, the weekly
|
||||
|
||||
@@ -86,7 +86,7 @@ a reviewed change to `DOLT_VERSION`, followed by the same provision/check flow.
|
||||
|
||||
In Admin → Jobs, wait until no other job is running, then:
|
||||
|
||||
1. Trigger **Dolt Earnings Import (shadow)**. Expect `completed` with import
|
||||
1. Trigger **Dolt Earnings Import**. Expect `completed` with import
|
||||
status `promoted`; a repeat without an upstream change should report `no_op`.
|
||||
2. Trigger **SEC Fundamentals Import**. The first run performs the
|
||||
tracked-universe history backfill and can take materially longer than a daily
|
||||
@@ -197,14 +197,22 @@ rollback window has closed, delete the rows and the `MANAGED_SETTINGS` filter in
|
||||
|
||||
## Failure and rollback
|
||||
|
||||
- **There is no provider fallback any more.** To recover bad `fundamental_data`
|
||||
values, restore the table from the PostgreSQL backup; the next scheduled run
|
||||
will rebuild it from the current snapshots. Confirm a recent backup exists
|
||||
before any change that could corrupt the snapshots.
|
||||
- Disable a failing source-import job in Admin → Jobs only when SEC network
|
||||
access itself must stop — the local cache refresh keeps running, and existing
|
||||
promoted snapshots/events remain available. To freeze the cache as well,
|
||||
disable the job *and* accept that P/E, market cap and earnings dates go stale.
|
||||
- **There is no provider fallback any more, and no Admin switch that freezes the
|
||||
cache.** Disabling **SEC Fundamentals Import** stops SEC network access only;
|
||||
the 04:00 job still rebuilds `fundamental_data` from the stored snapshots,
|
||||
earnings events and closes.
|
||||
- Restoring `fundamental_data` from the PostgreSQL backup is therefore a
|
||||
*temporary* fix on its own: if the bad values come from the snapshots or from
|
||||
the derivation code, the next scheduled run reproduces them. Fix the cause —
|
||||
restore or repair `fundamental_snapshots` / `earnings_events`, or revert the
|
||||
parser change and re-run `scripts/reparse_fundamentals.py --apply`.
|
||||
- To genuinely freeze the cache while you work, stop the service
|
||||
(`sudo systemctl stop signalplatform.service`) — that stops the scheduler with
|
||||
it. There is no finer-grained control, by design: a silently frozen scoring
|
||||
input is worse than an obvious outage.
|
||||
- Disable a failing source-import job in Admin → Jobs when SEC network access
|
||||
itself must stop. Existing promoted snapshots and events remain available, and
|
||||
the job's runtime message still reports the cache result.
|
||||
- Inspect the job runtime, latest `data_import_runs.validation_json`, service
|
||||
logs, and Admin → System Events before retrying.
|
||||
- `unresolved_filing` is emitted once when a filing enters automatic retry. It
|
||||
|
||||
Reference in New Issue
Block a user