I've nearly entirely switched to DuckDB for anything more than like 500 or 1,000 rows or if there are a tonne of columns.

Polars is great, but I'm just too used to the Pandas API to use it as a replacement for the cases where DuckDB is overkill.

▲

entropicdrifter 2 hours ago | parent [-]

That's a shame, because Pandas has a really quirky/legacy-burdened API and Polars is super clean by comparison. As someone who had Spark and Pandas experience before switching, Polars felt like Pyspark without the added mental overhead of needing you to think about multi-worker-node parallelism

	▲	jszymborski a few seconds ago \| parent [-]
		It's just muscle memor, I've been using Pandas daily for over a decade, but I'll likely eventually switch over to Polars. Part of the problem, as I said, is that I'm just spoiled by DuckDB when performance matters.