The Excel engine moves to Java

Two weeks after the first commit of the Java rewrite I joined it and shipped the substitute, dedup, join and graph actions in the first week. The platform decision behind it: the workbook engine had hit the Lambda time limit and the memory ceiling, and there was nothing left to tune.

A container on Fargate bought a large heap, unlimited runtime and a proper streaming parser. The cost was a new language for the team and a first quarter of fixes. Worth it.

flowchart LR
  U[Workbook in S3] --> T[Fargate task: ETL engine] --> R[Result in S3]
  T -.status.-> D[(DynamoDB)]