The intake pipeline gets its shape

The intake pipeline got its own repository and its own shape: a preprocessor, an input handler, a validator and a cleaner as separate Lambdas, with the processor split out six weeks later. That shape survived three rewrites and a change of language, and the first version that used it is still in production.

What made it durable was not the code but the contract between stages: each one reads one thing from S3, writes one thing back, and records its status in DynamoDB. Anything that broke could be re-run from the stage that broke.

flowchart LR
  A[Upload] --> B[Validate] --> C[Clean] --> D[Process] --> E[Publish]
  B -.status.-> S[(DynamoDB)]
  D -.status.-> S