AFAIK, you can't explicitly allocate cache like you allocate RAM however. A bit like if you could only work on files and ram was used for cache. Maybe I am mistaken ? (Edit: typo)

LLMs need memory bandwidth to stream lots of data through quickly, not so much caching. Well, this is basically the same way that a GPU uses memory.

People get on me all the time about not installing programs on my computer. I run everything in the browser, if I can. Partly so I can kill it properly without it misbehaving, and partly because I don't trust their software at all. Zoom, Slack, Gmail, etc-- if I can run it in the browser, then that's the only way I'll run it.

ive found the web versions use a similar amount of memory and have fewer features

my issue is that my company won't issue laptops with more than 16 gbs of ram

guess i'm not virtualizing anything...

The need to use optimal patterns didn't go away, but the techniques certainly did. Just as a quick example, it's usually a bad idea now to use lookup tables to accelerate small math workloads. The lookup table creates memory pressure on the cache, which ends up degrading performance on modern systems. Back in the 1980s, lookup tables were by far the dominant technique because math was *slow.*

It obviously never became completely irrelevant. But I think programmers spend a lot less time thinking about memory than they used to. People used to do a lot of gymnastics and crazy optimizations to fit stuff into memory. I do quite a bit of embedded programming and most of the time it seems easier for me to simply upgrade the MCU and spend 10cents more (or whatever) than to make any crazy optimimzations. But of course there are still cases where it makes sense.

When was the last time you used mergesort because you had to?

It mattered to pass through the interview, but not for the job itself. With all leetcode geniuses in Microsoft why Teams and Windows are so shitty?

Most developers work at banks, insurance companies and other “enterprise” jobs. Even most developers at BigTech and who are working “at scale” are building on top of scalable infrastructure and aren’t worrying about reversing a btree on a whiteboard.

> for fun

Not for fun but for convenience (laziness occasionally?). Someone needed to "pay" for the app being available on all platforms. Either the programmer by coding and optimizing multiple times, or the user by using a bloated unoptimized piece of software. The choice was made to have the user pay. It's been so long I doubt recent generations of coders could even do it differently.

RAM production is highly inelastic and controlled by an oligopoly. They have little desire to increase production considering the lead time and the risk that the AI demand might be transient.

They actively prefer keeping confortable margins than competing between each other. They have already been condemned for active collusion in the past.

New actors from China could shake things up a bit but the geopolitical situation makes that complicated. The market can stay broken for a long time.

RAM actually got more expensive to produce in the medium term because production is bottlenecked. It takes years to expand production.

Also, it's not like we have regressed in the process itself either, which was historically the limiting factor. As you said this is purely an economics thing resulting from a greedy shift in business focus by e.g. Micron.

A lot of that is on the OS vendors (and security requirements drive some inefficiencies that didn't used to be needed either).

This is coming from an insane demand spike, not some nefarious plot by the RAM manufacturers.

Eventually new capacity will come online, and the money the DRAM companies are making are going to accelerate even ,ore new capacity. If you can get your new capacity going before your competitors, maybe you can avoid a bubble burst. If you don’t build new capacity, your competitors will, etc, etc…