Remix.run Logo
mjfisher an hour ago

I see a lot of back and forth about postgres' suitability as a queuing system. I wonder if there's a couple of separable problems here. Postgres backed queues - even very scalable ones - might work well for background jobs in a monolithic app backed by a single DB. Things like backgrounding sending an email etc.

But usually when I reach for a queuing system, it's because I want to decouple a part of the architecture. And in that case, it's probably better to use a dedicated queueing system instead of postgres.

I wonder if the two cases are conflated in a lot of online discussion.

dewey 36 minutes ago | parent [-]

We are very heavily using Postgres as a queuing system in production for many years already, not just some quiet background tasks but with millions of tasks in the queue at any given moment. I have yet so see any issues with that so I'm always a bit suspicious when people say they had to reach for something else unless you are at a crazy scale.

I know it's very hard to compare workloads, but famous recent example: https://openai.com/index/scaling-postgresql/

> It may sound surprising that a single-primary architecture can meet the demands of OpenAI’s scale; however, making this work in practice isn’t simple.