Remix.run Logo
bitlad 4 hours ago

PostgreSQL is good enough until it's not good enough, when you realize all the bad design decisions that were made before it hits scale. It is the decisions people make around not partitioning, HA, replication that makes it not good enough.

throwawayffffas 4 hours ago | parent | next [-]

These are not bad decisions they are reasonable tradeoffs at the beginning.

Requiring HA, partitioning, and replication are good problems to have.

The alternative is spending engineering time on setting all these up for a failed service with like 100 users.

esafak 2 hours ago | parent [-]

It is not good to have a problem that requires a year of migration by a dedicated team (typical case if you have this 'good problem') when you could have simply spent another day or two to architect it right. You should at least have a path to scale when the time comes.

anitil 4 hours ago | parent | prev | next [-]

I feel like any problem that Postgres can't handle is a good problem to have. Either you've got so many customers that you're hitting sharp edges, or you're working on such an interesting problem that you're out of the domain where Postgres is helpful. That I should be so lucky

nine_k 3 hours ago | parent [-]

A moderate-size queue is already a problem that Postgres struggles to handle %) But fortunately such problems are rare, and relatively well-known, like, say, doubly-linked lists in Rust.

For most other problems, Postgres works well, or at least well enough.

keynha 12 minutes ago | parent [-]

[flagged]

stuaxo 4 hours ago | parent | prev | next [-]

That's fine. There are plenty of projects that don't hit that scale.

thayne 4 hours ago | parent [-]

There are also plenty of projects that do. PostgreSQL is good enough for most projects. But it isn't enough for every project.

jghn 3 hours ago | parent [-]

The problem is that way too many people assume they're in the latter group, when the proportions are very much the other direction

throwaway7783 4 hours ago | parent | prev | next [-]

Re: bad design decisions - This can be said about any technology.

whh 4 hours ago | parent [-]

+1 - when to prematurely optimise, when not to.

jamesfinlayson 2 hours ago | parent [-]

Oh yeah - I remember working with a guy who absolutely insisted that we build application-level sharding into what we were working on (i.e. each primary key would have some metadata that our application logic would use to route traffic to the correct Postgres instance). Our database was not big - 10s of gigabytes - but Instagram did something similar and we needed to be able to scale. I pushed back hard, and fast forward two years and the application was decommissioned without hitting any sort of Postgres bottleneck.

shepherdjerred 3 hours ago | parent | prev | next [-]

Everything is "good enough until it's not good enough". That's engineering.

cyberax 4 hours ago | parent | prev [-]

HA is now pretty good on Postgres.

As for scale... Just use a larger machine. This works for regular transactional data until you're at something like Amazon scale.

Edit:

Think about this, suppose that you store 1 megabyte of data for each of your customers. So if you have a million customers, it's just 1Tb. And these days, you can have a server with 10Tb RAM delivered overnight. Although you might have to sell your firstborn son (offer applies only to royal families) to fund it.

A lot of sharding/no-sql/... development happened in the late 2000-s when computers were about ~100 times less powerful than now. You _could_ get a system with 10Tb RAM in 2010, but as a specially-designed supercomputer.

leoqa 4 hours ago | parent [-]

128Gb of RAM is like 1.5k

cyberax 2 hours ago | parent [-]

For a non-ECC version. A single-module 128Gb ECC stick is about $4500. HBM is even more expensive.