Remix.run Logo
jtbaker an hour ago

DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.

Thanks to the team for everything!

fifilura 29 minutes ago | parent | next [-]

My favourite is AWS Athena (backed by Trino).

"If we use this we get indefinite RAM indefinite CPU and do not need to host a server".

I had an impression that DuckDB was not great at distributing work to other machines, but good at doing it locally? Am I wrong?

abirch 6 minutes ago | parent | next [-]

DuckDB out of the box may not be great. But you have DuckLake, Quack, and even DeepSeek made their own distributed DB based on DuckDB: https://github.com/deepseek-ai/smallpond

ericpauley 23 minutes ago | parent | prev [-]

Athena + Clickhouse has been an absolute game changer for us. Perfect combo for OLAP + deeper filtering that we can’t necessarily pre-index for.

throwaw12 an hour ago | parent | prev [-]

Curious to learn more about how people are using it?

Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?

jtbaker 16 minutes ago | parent | next [-]

I've got a couple of different use cases:

- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.

- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.

- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.

arealaccount an hour ago | parent | prev | next [-]

We use DuckDB WASM with parquet to build dashboards in-browser. It's cool to be able to write SQL directly in a browser and not have to rely on REST/Graphql/etc to access the data layer.

drums8787 an hour ago | parent | prev [-]

We use WASM DuckDB as the target for an in-browser agentic feature. Generated SQL runs against the user's individual tables that then feed in-browser dashboards.

Excellent performance.

throw1234567891 39 minutes ago | parent [-]

[flagged]

fg137 a minute ago | parent [-]

Running duckdb as wasm in browser for dashboards is a very common use case.

Does that make this account an alias as well?