1 min lesson
Failure handling
Use "Failure handling" to tell the cases apart, then choose a response for each one.
Step 1 of 2
Failure handling
Providers rate-limit, time out and return partial matches. A waterfall has to degrade rather than break, because a stalled pipeline is worse than a half-filled record.
Treat as a miss for now and fall through to the next provider.
Queue a retry with backoff; don't fail the whole record.
Keep the fields that came back, mark the rest still-open.
A later provider or run can fill the gaps.
Prefer the higher-trust provider or the more recent verified value.
Keep both with provenance so you can audit the choice.
Mark the field unenrichable and move on.
Don't retry forever - log it and let scoring proceed on what you have.
Don't describe the waterfall as a strict ladder that always runs top to bottom. The point of stopping early and caching is that most records never touch the bottom of the chain. If your design re-runs every provider on every record, you've built an expensive single-provider pipeline with extra steps.