dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(dolt): record/alert revision-detection failures + handle cancellation

Browse Source 
Review fixes to the import-run framework (A1):

1. detect_revision ran outside the failure handler, so a failed revision probe
   (the most likely external failure) escaped unrecorded — violating "every
   attempt is recorded". Now the running row is created FIRST, then
   detect_revision + last-revision lookup + stage + validate + promote all run
   inside the same handler; the row converts to no_op when the revision is
   unchanged. New test covers a detection exception → recorded failed + alert.

2. asyncio.CancelledError (BaseException, not caught by except Exception) left a
   permanent running row on deploy/scheduler shutdown. Now caught explicitly:
   best-effort mark failed, then re-raise the cancellation (never swallowed).
   New test asserts the run is failed and the error re-propagates.

Full suite 682 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
dennisthiessen
2026-07-22 09:25:14 +02:00
co-authored by Claude Opus 4.8
parent febd741671
commit 8b45bdb361
2 changed files with 62 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/services/data_import.py
+30 -17
View File
@@ -25,6 +25,7 @@ table (idempotency queries the last run), no aggregate tables.
from __future__ import annotations
import asyncio
import hashlib
import json
import logging
@@ -170,25 +171,11 @@ async def run_import(
logger.info("data_import %s: lock held, skipping", source)
return None
revision = await importer.detect_revision(session)
last_rev = await _last_promoted_revision(session, source)
if revision is not None and revision == last_rev:
run = DataImportRun(
source=source,
revision=revision,
status=STATUS_NO_OP,
started_at=_now(),
completed_at=_now(),
)
session.add(run)
await session.commit()
await session.refresh(run)
logger.info("data_import %s: no_op (revision %s)", source, revision)
return run
# Record the attempt FIRST — before the external revision probe, the
# most likely failure — so anything below is recorded and alerted and
# never escapes unrecorded. Revision is filled in once detected.
run = DataImportRun(
source=source,
revision=revision,
status=STATUS_RUNNING,
started_at=_now(),
)
@@ -197,6 +184,16 @@ async def run_import(
await session.refresh(run)
try:
revision = await importer.detect_revision(session)
run.revision = revision
last_rev = await _last_promoted_revision(session, source)
if revision is not None and revision == last_rev:
run.status = STATUS_NO_OP
run.completed_at = _now()
await session.commit()
logger.info("data_import %s: no_op (revision %s)", source, revision)
return run
staged = await importer.stage(session)
result = await importer.validate(session, staged)
run.source_max_date = result.source_max_date
@@ -230,6 +227,22 @@ async def run_import(
)
return run
except asyncio.CancelledError:
# Deploy / scheduler shutdown: best-effort mark failed so no
# ``running`` row lingers, then let the cancellation propagate —
# never swallow it.
try:
await session.rollback()
run.status = STATUS_FAILED
run.error_details = "cancelled"
run.completed_at = _now()
await session.commit()
except BaseException: # noqa: BLE001 — best-effort during teardown
logger.warning(
"data_import %s: could not record cancellation", source
)
raise
except Exception as exc: # noqa: BLE001 — record + alert, don't crash the job
await session.rollback()
run.status = STATUS_FAILED