Remix.run Logo
dewey an hour ago

We are very heavily using Postgres as a queuing system in production for many years already, not just some quiet background tasks but with millions of tasks in the queue at any given moment. I have yet so see any issues with that so I'm always a bit suspicious when people say they had to reach for something else unless you are at a crazy scale.

I know it's very hard to compare workloads, but famous recent example: https://openai.com/index/scaling-postgresql/

> It may sound surprising that a single-primary architecture can meet the demands of OpenAI’s scale; however, making this work in practice isn’t simple.

tomnipotent 18 minutes ago | parent [-]

That post is about how Postgres doesn't scale for write-heavy workloads and that they had to move those workloads to Cosmos DB. For the rest of the remaining mostly-read workload they have a single primary with 50 read replicas.

dewey 14 minutes ago | parent [-]

The point is that Postgres scales a very long way. Once you arrive at OpenAI / ChatGPT scale there's no shame in reaching for a dedicated queuing system.

tomnipotent a minute ago | parent [-]

> scales a very long way

This ignores the fine print. It scales a very long way under specific circumstances with specific workloads. The more write-heavy your workload the less eloquently Postgres scales.