Remix.run Logo
beoberha 2 days ago

Curious why pgduck_server is a totally separate process?

rmnclmnt 2 days ago | parent | next [-]

The README explains it:

> This separation also avoids the threading and memory-safety limitations that would arise from embedding DuckDB directly inside the Postgres process, which is designed around process isolation rather than multi-threaded execution. Moreover, it lets us interact with the query engine directly by connecting to it using standard Postgres clients.

pgguru 2 days ago | parent | prev | next [-]

What has been pointed out from the README; also:

- Separation of concerns, since with a single external process we can share object store caches without complicated locking dances between multiple processes. - Memory limits are easier to reason about with a single external process. - Postgres backends end up being more robust, as you can restart the pgduck_server process separately.

dkdcio 2 days ago | parent | prev [-]

from the README:

> This separation also avoids the threading and memory-safety limitations that would arise from embedding DuckDB directly inside the Postgres process, which is designed around process isolation rather than multi-threaded execution. Moreover, it lets us interact with the query engine directly by connecting to it using standard Postgres clients.

beoberha 2 days ago | parent [-]

Thanks! Didn’t scroll down far enough