| ▲ | shubhamjain 2 hours ago | |
Despite its obvious advantages, the biggest drawback of DuckDB is its concurrency model [1]. If a process opens a database in read-write mode, it acquires an exclusive lock on the file. This prevents even simple read operations from other processes as long as the writer remains open. Maybe there's a simple workaround I haven't come across, but I found it to be quite a productivity killer. So yes, all these benchmarks are great, but it wasn't so fun working with DuckDB when I had to close duckdb cli, just so a query in another script could run. | ||
| ▲ | coldbrewed 2 hours ago | parent | next [-] | |
Duckdb has a server mode[1] which might alleviate some of those pain points. SQLite is a bit more precise in that only a single connection can write to the DB which provides more concurrency but still has pain points. For a single file DB either choice seems justifiable to manage complexity. | ||
| ▲ | threatofrain 37 minutes ago | parent | prev | next [-] | |
This is actually the biggest reason I don't automatically choose sqlite for small project databases. I've built a lot of toy utilities that crossed a bare threshold of usefulness, and then suddenly it's not a toy anymore when people start using it. Now it's debt. Oops. | ||
| ▲ | datadrivenangel 2 hours ago | parent | prev | next [-] | |
Quack is now kind of a workaround for that limitation, as you can have a process with the lock offer read access to other processes. It's not perfect, but for that specific use case it's pretty good. | ||
| ▲ | biophysboy 2 hours ago | parent | prev [-] | |
You can have multiple read only processes. But yes, concurrent read mode and write mode is blocked | ||