Work
23 Aug 2026#ops#data 1 min read
The vendor feed came back empty
A weekly refresh that trusted an upstream feed, and the abort gate it earned.
A weekly multi-cloud price refresh had run without a miss for the better part of a year. Then one upstream feed returned nothing: not an error, an empty result, and the pipeline was happy to publish a catalog with a whole cloud missing. Nothing failed, which was the problem.
The fix was not a retry. It was an abort gate: compare every source against last week's snapshot, and if one shrinks past a threshold, stop, publish nothing, and say which source in the notification.
flowchart LR
A[Fetch feeds] --> B[Compare to last week]
B -->|any source shrank| C[Abort and notify]
B -->|within threshold| D[Publish catalog]Takeaway
An empty success is a failure mode; count rows, not exit codes.
Client engagement, 2021 to 2026. Names, identifiers and internals are generalised.