Remix.run Logo
▲ sureglymop 3 hours ago

I've used Polars before and can only recommend it.

It's like you get a really good 'query planner' like a DB would give you, but for your notebooks/scripts/etc. Much better than pandas imo.

▲jadbox 2 hours ago | parent [-]

Why use Polars over just postgresql or sqlite?

▲cle 2 hours ago | parent | next [-]

It is optimized for analytic workloads (default in-mem columnar layout), whereas PG and sqlite are OLTP DBs. It's going to be insanely faster for those use cases.

▲zeristor 15 minutes ago | parent [-]

And with DuckDB?

DuckDB is an in process OLAP, I’ve been using it a lot, and I am keen to use Polars but DuckDB seemings to be flying for me.

▲Tuna-Fish an hour ago | parent | prev | next [-]

Actual databases pay up to two orders of magnitude of speed for durability. If you can regenerate your dataset in case the DB drops it (or there is a power outage, or whatever), being able to complete a complex, write-heavy query in 10s instead of 15 minutes is actually very useful when doing analysis.

▲phillc73 an hour ago | parent | prev | next [-]

If you want to use a database instead of Polars, for similar use cases, duckdb is a much better bet than postgressql or sqlite.

▲entropicdrifter 2 hours ago | parent | prev [-]

Because those are databases? Polars is a data processing engine, not a database. They have different uses.