One thing I wonder about is that these CPUs are just so much faster than their memory controllers. So why not add more? Why don't they have a memory controller per core, for example?

Memory controllers are pretty much limited by external pinout.

Zen 4 and newer consumer chips have 4 memory controllers (for legacy reasons shown as 2) which handle 32bit wide LPDDR5 or DDR5 interfaces (2x per dimm for DDR5)

The AM5 (desktop) socket has in addition to the 4x 32bit memory lanes, 28 PCI-E lanes (generally 16x for GPU, two 4x general purpose, one 4x to chipset), 4x combined DP/HDMI outputs, USB 2.0 port, 4 USB 3.2 Gen 2 ports. And that's just the major buses, there's ton of other components exposed (simpler IO, sound, power)

All of that counts to staggering 1718 pins - anyone remember when moving dual-channel K8 from 940 pin to 939 allowed removal of whole layer from motherboard PCB, making them way cheaper?

The high-end performance server socket from AMD has 6096 pins, significant chunk of it being the 24x 32bit DDR5 interfaces.

So unfortunately it's very much not easy to add more memory, and that's before considering issues with routing it on motherboard even if you accept changing shapes

Cost, and especially now, cost of memory! But you can buy AMD chips with 2 controllers (consumer), 4 (threadripper), 6 (8005), 12 (EPYC) memory channels, so in effect you can. Next generation Zen 6 Epyc has 16 channels. Roughly you get about 1 channel per chiplet, but you csn adjust if you need more memory bandwidth by buying fewer chiplets, eg there is an 8 core EPYC if you want to optimisise for bandwidth. But many applications are not bandwidth bound so this isn't the right choice, eg they are IO bound, rarely compute bound.

I think it mostly is for more controllers to be useful that means more PHYs and more actual pins on the package and more traces on the board and more sockets for the memory and... The space and cost of all of that is substantial (to a point, and then it just physically can't be manufactured).

More memory controllers would add more bandwidth. That can help, but few workloads are bottlenecked by memory bandwidth. (Dual memory channels have been a thing in desktops and laptops since the early 2000s single-core era at least, but newer generations of DDR have increased bandwidth to keep up with increased core counts.) What's really needed is less memory latency (less latency = faster), which has not improved as much as bandwidth.

That doesn't help memory latency and most of the time a cpu is waiting it's because of latency, not bandwidth.

It's not always trivial to saturate available memory bandwidth. You may need to use more than one core, or you need to carefully consider the topology of the chip you're working with (see lstopo). For example, what if the L3 cache is actually split in two [0]? In such cases, adding more bandwidth doesn't do much for you.

[0] https://en.algorithmica.org/hpc/cpu-cache/sharing/#saturatin...

Nine women cannot make a baby in one month.

128 cores x 288 pins per DIMM = 36864 pins on your socket, and all the motherboard traces. Good luck with that...