Python

This is Part 2 of a series. Read Part 1: Pandas vs Polars in Production - Performance Comparison for the background on the Polars migration. After migrating from Pandas to Polars, CPU performance improved dramatically—but a memory problem persisted. Despite extensive debugging, I couldn’t identify the root cause. So I made a pragmatic decision: architect around it. This is the story of splitting a monolithic Python application into a Go orchestration service with Python workers, not because I fully understood the problem, but because I needed production to be stable. ...

The Problem While working on a freelance project, I needed to generate PDF reports from Excel files. The challenge? The Excel files were wide and long. Here we see a sample Excel file around 15 columns containing employee data like ID, name, email, department, job title, hire date, salary, and more. I used LibreOffice’s CLI utility convert-to in headless mode to convert the files. The conversion worked, but the result was unusable: ...