| ▲ | alastairr 10 hours ago | |
I'm running duckdb over 500gb of parquet on a largish desktop (50gb ram) and it's been smooth & fast. I guess OOM issues will matter at some point, but I think it's going to be in the top 1% of real world use cases. | ||
| ▲ | jauntywundrkind 10 hours ago | parent [-] | |
With a decent SSD (or eight), spilling to disk is really not bad these days! Yes! And if that's still not enough, if you just need to crunch data a couple times a week, it's not unreasonable to get a massive massive cloud box with ridiculous amounts of ram or ram+SSD. I7i or i8g boxes. Alas, we have cheap older gen epycs & some amazing cheap motherboards but RAM prices to DIY are off the charts unbelievable, but so be it. | ||