I think it was closer to what they need; it comes down to keeping TB of data on local SSD and having a query engine next to it. That takes persistent, shared CPU, caching, sharding, resource management, etc.

Duckdb is an interesting approach to resource management (all ephemeral), but TB's are TB's. I doubt it was finding data where it needed it.

Clickhouse seems slow at getting to the top end of scaling; it's stuck with the 99% of its userbase doing 1 GB queries. I'm hoping they put some of their money into it.

▲

a34729t an hour ago | parent [-]

ClickHouse is fast OLAP. Just do Iceberg with Trino like a normal big data person. WTF?

	▲	datadrivenangel 40 minutes ago \| parent [-]
		DuckDB is now close to competitive with Trino. As the post shares, large distributed multi-tenant data systems require a ton of work no matter what technology you use, but as a huge DuckDB fan, I do have to agree that it is not an out of the box data warehouse, though check back in a year or so when DuckLake and DuckDB 2.0 have had time to mature a bit more for this use case.