| ▲ | steve_adams_86 2 hours ago | |
DuckDB has this capability as well: https://duckdb.org/docs/stable/guides/performance/how_to_tun... | ||
| ▲ | falconroar an hour ago | parent [-] | |
Interesting, I wasn't aware; thanks for that. I will say, Polars' implementation is much more centered on out-of-core processing, and bypasses some of DuckDB's limitations ("DuckDB cannot yet offload some complex intermediate aggregate states to disk"). Both incredible pieces of software. To expand on this, Polars' `LazyFrame` implementation allows for simple addition of new backends like GPU, streaming, and now distributed computing (though it's currently locked to a vendor). The DuckDB codebase just doesn't have this flexibility, though there are ways to get it to run on GPU using external software. | ||