... for AI... which people have to convince themselves they 'need'. I am a user, I have a 1m$ rack for my small company for AI, but I have currently 189 racks that cost about 15k one off per rack. And those make us a LOT more money than the AI ones ; most are departmental apps or websites that take very little processing power and are low on everything while people pay nice money to keep them going. Our support is better than the rest; you can just call me or come to my house and we'll do something. Not many clients leave, ever. For the past 30 years. I buy 5-10 year old servers which are complete overkill for 99% of my clients (some are fortune 500 companies but departmental stuff) and they just never leave. Makes vastly more money than AI and we are still cheaper than AWS and the department does not have to get a flexible budget (like they would have to if AWS) with us. We just have the same fixed price + yearly inflation correction and that's it. For 30 years.

Right. It's probably important to distinguish what the hardware manufacturers' incentives are when their supply is constrained and when it isn't. (I am not an expert on anything.) As long as supply is strongly constrained they're naturally going to sell to the highest bidders, which are the big LLM SaaS players. If and when supply is no longer constrained, though, things will look very different:

1) The LLM SaaS companies are a form of vertical disintegration for the hardware providers, a middleman covering costs and taking profits out of the money that comes from customers to the hardware providers. That changes somewhat if there are no longer good models available for local use at no cost to the hardware guys, but only somewhat

2) The LLM SaaS companies are efficient users of their hardware resources. While supply is constrained this helps to make them top bidders and so attractive customers for the hardware manufacturers. When supply is not constrained this should reverse. Which is the more attractive class of customer to a hardware maker: the company full of people with higher degrees who spend their whole working day fighting to pare back resource usage, or the guy who leaves his laptop idle about 18 hours per day on average?

It's notable that nVidia, for instance, has continued to put significant emphasis on AI compact desktops and laptops. And while no doubt that's partly in the service of better developer relations and good PR in general, it's probably also nVidia eyeing the exit, and preparing for a future transition from selling shovels to the army to selling shovels at Walmart. But of course the future isn't clear and obvious. If the hardware makers, maybe the RAM guys in particular, turn out to have underbuilt future capacity starting in the present then we could be stuck in constrained supply for quite a long time. (Futher) government action could affect things etc. etc. And if the frontier labs soon find new ways to use still larger amounts of memory, GPU capacity etc. that isn't butting up against diminishing returns then they'll likely remain kings for some time, though that does not seem probable now.