My company tried DuckDB-WASM + parquet + S3 a few months ago but we ended up stripping it all out and replacing it with a boring REST API.

On paper it seemed like a great fit, but it turned out the WASM build doesn't have feature-parity with the "normal" variant, so things that caused us to pick it like support for parquet compression and lazy loading were not supported. So it ended up not having great performance while introducing a lot of complexity, and also was terrible for first page load time due to needing the large WASM blob. Build pipeline complexity was also inherently higher due to the dependency and data packaging needed.

Just something to be aware of if you're thinking of using it. Our conclusion was that it wasn't worth it for most use cases, which is a shame because it seems like such a cool tech.

▲

ludicrousdispla 6 days ago | parent | next [-]

DuckDB-WASM supports parquet file decompression though, so if you have a backend process generating them it's a non issue.

How large was your WASM build? I'm using the standard duckdb-wasm, along with JS functions to form the SQL queries, and not seeing onerous load times.

▲

mentalgear 6 days ago | parent | prev [-]

> WASM build doesn't have feature-parity with the "normal" variant

It's a good point, but the wasm docs state that feature-parity isn't there - yet. It could certainly be more detailed, but it seems strange that your company would do all this work without first checking the feature-coverage / specs.

> WebAssembly is basically an additional platform, and there might be platform-specific limitations that make some extensions not able to match their native capabilities or to perform them in a different way.

https://duckdb.org/docs/stable/clients/wasm/extensions

▲

dtech 6 days ago | parent [-]

Note that your docs specifically mentions parquet was supported, but we found out the hard way some specific features turned out not to be supported with WASM + parquet. I did a quick glance at your docs and could not find references to that, so I'm not surprised it was missed.

It was a project that exploited a new opportunity so time-to-market was the most important thing, I'm not suprised these things were missed, and replacing the data loading mechanism was maybe 1 week of work for 1 person, so it wasn't that impactful a change later.

	▲	mentalgear 6 days ago \| parent [-]
		Fair point, thx for sharing your experiences ! You might want to edit the duck-wasm docs in that regard to alert others/the team of this constraint.