Root cause of "price plan needed in bulk but fine on manual reload": on free
tiers FMP returns only market cap (others 402) and the chain merged that as a
partial success — so when the Finnhub/Alpha Vantage fallbacks were rate-limited
during a bulk run, the chain silently returned market-cap-only and the
collector's backoff never engaged. Manual single fetches worked because the
fallbacks weren't throttled at that moment.
Fixes:
- Chain distinguishes RateLimitError from other failures: if a fallback is
rate-limited and fields are still missing, raise RateLimitError (unless
allow_partial=True) so the collector backs off and retries.
- Bulk job paces requests (fundamental_request_spacing_seconds, default 3s) to
stay under Finnhub's ~60/min, and on retry-exhaustion stores partial data and
continues instead of aborting the whole run.
- Manual fetch passes allow_partial=True so a lone 429 doesn't fail the refresh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dennisthiessen
f24e5070ee