I have dealt with an abomination of a bespoke setup on Redis that was neither simple key/value nor an RDBMS but a grotesque attempt to simulate tables using key prefix with value being a hash containing "references" to entries in other "tables" (with some "special condition" edge cases thrown in to make it spicy). All with next to no documentation. It took adding many unit tests with generous logging to expose the logic and the underlying structure. That complete madness could have been handled in a bog standard way in PostgreSQL with minimal fuss. All for something that didn't even need to be "real-time".

If you see yourself starting with simple key/value setup and then feature requests come in that make you considering having "references" to other keys, it is time to re-consider Redis, not double down on it. Even if you insist on continuing, at the very least, add a service to manage it with some clean abstractions instead of raw-dogging it.

> Both postgres and redis are used with the out of the box settings

Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously.

Some will say that in a container-orchestrated environment, tuning goes out the window since "you never know" where the orchestrator will schedule the service but this is bogus. If you've got time to write a basic deployment config for the service on the orchestrator, you've also got time to at least size the memory usage configs for PostgreSQL and/or Redis. It's just that simple.

This is the kind of thing that is "hard and tedious" for only about five minutes of LLM query or web search time and then you don't need to revisit it again (unless you decide to change the orchestrator deployment config to give the service more/less resources). It doesn't invite controversy to right-size your persistence services, especially if you are going to publish the results.

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion.

I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess, it would cause cache degradation too when there are any problems with the DB.

I’m surprised to see this being somewhat controversial.

In Rails we just got database backed everything with the option to go to special backends if need be.

The only question I have is how do I notice that my current backend doesn’t scale anymore and who or what would tell me to switch.

In similar vein, I’d always thought Redka (https://github.com/nalgeon/redka) was a neat idea since it gives you access to a subset of the Redis API backed by either SQLite or Postgres

Besides using the word "cache" what does this have to do with caching?

It looks like it's just storing the session in postgres/redis.

Caching implies there's some slower/laggier/more remote primary storage, for which the cache provides faster/readier access to some data of the primary storage.

What is all the fuss about? In the ancient times, you put your entry into the database(ANY database), either with TTL or cache tags, and have a cron job that periodically flushes the table. Then in your application you check if you have the entry cached in memory, if not, you check the db entry, if it is not there you build it from scratch and cache it.

Why do people complicate things? We've solved caching ages ago.

Having the ability to set a TTL on the cache key is a critical feature of a cache, not something that can be tacked on later.

I always find these "don't use redis" posts kind of strange. Redis is so simple to operate at any scale, I don't quite get why it is important to remove it.

I've done some benchmarks over the years and 6-7k/s for getting simple data out of a basic Postgres installation seems pretty spot on. The question is when using Postgres as a cache, are you taking away performance for actual, more complex business logic queries. That is going to depend on the level of overlap between endpoints that use caching and those that need more complex queries.

What I'd be interested to see is a benchmark that mixes lots of dumb cache queries with typically more complex business logic queries to see how much Postgres performance tanks during highly concurrent load.

For cache, having TTL is invaluable. Having to tune cleanup jobs in Postgres is annoying. Deletes create dead rows (so do updates) so now you have to deal with vacuum as well. The method the author suggested will run into a lot more problems if the service ever needs to scale up beyond what they estimated than a traditional dedicated cache like Redis in front.

> For postgres, the bottleneck was the CPU on the postgres side. It consistently maxed out the 2 cores dedicated to it, while also using ~5000MiB of RAM.

Comparing throttled pg vs non-throttled redis is not a benchmark.

Of course when pg is throttled you will see bad results and high latencies.

A correct performance benchmark would be to give all components unlimited resources and measure performance and how much they use without saturation. In this case, PG might use 3-4 CPUs and 8GB of RAM but have comparable latencies and throughput, which is the main idea behind the notion “pg for everything”.

In a real-world situation, when I see a problem with saturated CPU, I add one more CPU. For a service with 10k req/sec, it’s most likely a negligible price.

Cache is a very relative term. If I'm caching heavy computation, or perhaps externally acquired resources (e.g web scrapped data), I'd use database as a cache. If I'm caching database results, then I'd obviously use something faster than that.

I hope you remember to add monitoring for your cache expiry cron job so you can be notified if it ever fails to run due to a code change or configuration change. For example, database credentials might be rolled and if you forget to update the job your primary database could fill up.

Perhaps you could have a second cron job that runs to verify that the first one completed. It could look for a last-ran entry. You should put it in the same database, so maybe perhaps you could a key value store like redis for that.

I think the moral here is, if you'll ever have less than 1000 active customers per second, don't do micro services, don't use redis, just use a monolith with a simple stack and you'll still be happy, less maintenance, less knowhows, etc... cheaper all around.

I skimmed the article, but why is everyone always going for distributed cache? What is wrong with in-memory cache? Lowest latency, fast, easy to implement.

Yeah ok, you have 30 million entries? Sure.

You need to sync something over multiple nodes? Not sure I would call that a cache.

I guess my question is: why bother with a benchmark if the pick is pre-ordained? Is it the case that at some point the results would be so lopsided that you would pick the faster solution? If so, what is that threshold? I.e. when does performance trump system simplicity? To me those are the interesting questions.

I've got a similar setup with a k3s homelab and a bunch of small projects that need basic data storage and caching. One thing worth considering is that if someone wants to run both redis and postgres, they need to allocate enough memory for both including enough overhead that they don't suddenly OOM.

In that sense, seeing if the latency impact of postgres is tolerable is pretty reasonable. You may be able to get away with postgres putting things on disk (yes, redis can too), and only paying the overhead cost of allocating sufficient excess RAM to one pod rather than two.

But if making tradeoffs like that, for a low-traffic service in a small homelab, I do wonder if you even need a remote cache. It's always worth considering if you can just have the web server keep it's own cache in-memory or even on-disk. If using go like in the article, you'd likely only need a map and a mutex. That'd be an order of magnitude faster, and be even less to manage... Of course it's not persistent, but then neither was Redis (excl. across web server restarts).

This isn't a great test setup. It's testing RTT rather than the peak throughput of Redis.

I'd suggest using Redis pipelining -- or better: using the excellent rueidis redis client which performs auto-pipelining. Wouldn't be surprising to see a 10x performance boost.

https://github.com/redis/rueidis

I think you just convinced me to drop redis for my new project.

Definitely a premature optimization on my part.

My laptop running OOTB Postgres in docker does 100k requests per second with ~5ms latency. How are you getting 20x slower?

Is this engagement bait? Sigh... fine, at least I bite the bait here:

1. The use-case is super specific to homelab where consistency doesn't matter. You didn't show us the Redis persistence setup. What is the persistence/durability setting? I bet you'd lose data the one day you forgot and flip the breaker of your homelab.

2. What happened when data is bigger than your 8GB of RAM on Redis?

3. You didn't show us the PG config as well, it is possible to just use all of your RAM as buffer and caching.

4. Postgres has a lot of processes and you give it only 2 CPU? Vanilla Redis is single core so this race is rigged to begin with. The UNLOGGED table even things out a bit.

In general, what are you trying to achieve with this "benchmark"? What outcome would you like to learn? Because this "benchmark" will not tell you what you need to know in a production environment.

Side note for other HN readers: The UNLOGGED table is actually very nifty trick for speeding up unit tests. Just perform ALTER to UNLOGGED tables inside the PG that's dedicated for CI/CD: ALTER TABLE my_test_table SET UNLOGGED;

Having native TTL in PostgreSQL would remove so much unnecessary Redis in the wild.

I'd be curious to see how Postgres behaves with numeric IDs instead of strings (or another built-in, faster-to-index/hash type).

Still leaves you with needing to perfomantly hash your strings into those IDs on top, but mostly as the Postgres plateau of performance compared to purposely built KV DBs.

I think there is more performance to be taken from PostgreSQL. Not sure how pgx works internally since I have not used it.

There are async functions provided by PostgreSQL client library (libpq). I've used it to process around 2000 queries on a single connection per second on a logged table.

This seems to be testing how the software is optimized for low core deployments… how does Postgres perform vary as you add more cores and ram? It’s the sort of software where I’d presume more cores and ram yields better performance. Assuming as always that mature systems software sees more many core perf engineering.

postgres message protocol is much complex than redis. i think it's the bottleneck for such difference.

The values in this test are like 20 bytes, right? Wonder how things compare if they're about 1KB.

The big with for Redis is pipelining IMO.

When I last benchmarked Redis vs. PostgreSQL for a simple k/v cache it was about ~1ms for PostgreSQL to fetch a key, and ~0.5ms for Redis with a similar setup as in this post (although I used "value bytea" instead of "value string" – I don't know if it matters, probably not; 1ms was fast enough that I didn't care to test).

I didn't measure setting keys or req/sec because for my use case keys were updated infrequently.

I generally find ms to be a more useful metric than reqs/sec or latency at full load, as this is not a typical load. Or at least wasn't for my use case.

Of course all depends on your use case etc. etc. In some cases throughput does matter. I would encourage everyone to run their own benchmarks suited to their own use case to be sure – should be quick and easy.

As I rule I recommend starting with PostgreSQL and using something else only if you're heavily using the cache or you run in to problems. Redis isn't too hard to run, but still just one less service to worry about. Or alternatively, just use a in-memory DB. Not always appropriate of course, but sometimes it is.

In nginx config you can pretty easily cache web pages or API responses. I would also like to know how far that goes.

Also does anyone like memcached anymore? When I compared with Redis in the past it appeared more simple.

If you use an UNLOGGED table in Postgres as a cache, and your DB restarts, you no longer have a cache. Then your main table gets a huge spike in traffic and likely grinds to a halt.

I'm glad someone did this analysis. I've also been tempted to remove complexity by removing Redis from my stack. But there's a decent speedup from using Redis, so I'll keep it.

(And to the people complaining about this benchmark not being extremely scientifically rigorous: Nobody cares.)

Given there is nothing at all said about the many config options that would contribute to Postgres for this use case, we must assume no configuration has been done.

Also, no discussion of indexes or what the data looks like, so we must assume no attention has been paid to their critical factors either.

So, another case of lies, damned lies and benchmarks.

It seem strange to me people are so willing to post such definitive and poorly researched/argued things - if you're going to take a public position don't you want to be obviously right instead of so easily to discount?

reminds me of Handlersocket in MySQL

PostgreSQL Tuning further for this use case

- Reduce Page Size from 8KB to 4KB, great for write heavy operations and indexed reads. Needs to compile source with those flags, cant configure once installation is done.

- Increase Buffer cache

- Table partitioning for UNLOGGED Table which the author is using

- At connection session level, lower the transaction level from SERIALIZABLE

- The new UUID7 in PG 18 as a key might also help as primary indexed KEY type as it also supports range queries on timestamp

Don't tell DHH

love that the author finished with:

"i do not care. i am not adding another dependency"