Which failures should block the pipeline

How a hard validation gate turned into a warning, and why that was right.

A validation-only state machine with detailed notifications, so the checks could be run without moving data. One check, a name mismatch between two records, kept blocking runs for cases that were legitimate. For months it grew exceptions, one per case.

Then it stopped being a gate and became a warning in the success message. The data was never wrong; the check was answering a question nobody was asking at that stage.

flowchart LR
  C[Validation check fails] --> Q{Would the output be wrong?}
  Q -->|yes| B[Block the run]
  Q -->|no| W[Warning in the success message]

Takeaway

A failure should block a pipeline only if letting it through would make the output wrong. Everything else is a report.

Client engagement, 2021 to 2026. Names, identifiers and internals are generalised.