I am not a hardware guy, so I am asking this in good faith: excluding people with corporate backing, who actually needs DDR5 RAM? Gamers? Why is DDR4 or DDR3 not good enough?

Because modern CPUs are on platforms that support only DDR5.

If you are a gamer, chances are you want one of the AMD X3D CPUs. Whilst AMD did produce 5600X3D, 5700X3D and the highly sought after 5800X3D, these are effectively unobtainable now (outside of the Used Market, which is already about 2X MSRP).

You are effectively forced into AM5 (or whatever Intel is doing) and they require DDR5. You don't have the "choice" to use DDR4 anymore in most circumstances.

If your question is more of a hypothetical (assuming we could use newer CPUs with DDR4 or even DDR3) the answer is a bit more blurred, but at least in a lot of gaming workloads, you aren't memory speed bound. There is some performance regressions, sometimes up to 15%, but a lot of this is negated with the X3D chips anyways (:

Who really needs more than warmth, shelter and something to eat and drink?