Your main argument is about better allocation for that RAM, which only makes sense in the conditions of a shortage. If RAM is plentiful, like it’s bound to be once the datacenters craze is over, that argument becomes “why does every apartment need a washer/dryer when we can have shared washers/dryers in a common area that are being used 100% of the time”. The answer is “because we can and it’s not that huge of a difference” plus you can have both. Like you can have a server cluster for large batch jobs, and personal ones for personal use.

For individuals though, it’s the same reason plenty of people pay for the 2tb or 4tb SSD for their laptops or desktops even though NAS has been a thing for 40 years. Maintaining a “server” of any sort is a completely separate endeavor for most. Just like a NAS is a no-brainer for many and non-starter for many more.

▲

flaunf221 2 days ago | parent [-]

RAM was dirt cheap before and yet until M3 Apple made laptops starting with 8 GB of RAM. Why do you think Apple will suddenly become generous later?

The problem with LLMs outside of server is that people typically have many devices. So you can run LLM on your laptop. You want to make a query from your phone. Start your laptop? You want to do something from your smart home? Start your laptop? What about your partner? Will they need to buy another top of the line laptop for themselves? Your kids?

Alternative to server is of course that everything will have over 100 GB of memory, eat battery like crazy, be dumber and slower than single server in your household.

4 TB ssd was like $300 until recently. Even if RAM prices were to go to previous values, I don't think AI capability will be any comparable. Besides you also need proper compute, unless you want your prefill to be snail slow.

▲

eddythompson80 2 days ago | parent [-]

It has nothing to do with generosity. Apple (and other laptop and PC makers) just have different segments of the market that they catered for. Plenty of people use laptops just to browse the web. They didn’t need a lot of RAM. Those who needed the RAM (usually branded as pro-users or “creators” or engineers) always had the option to get more RAM both from Apple and others at a premium. It’s just that ~32GB was the upper amount for almost any use case. Why would Apple put 256GB or 512GB in every laptop back when there was no use case for it?

Every scenario you described applies the same to a NAS. What if you need a file from your phone? Open your laptop? What if your partner needs a file? Open your laptop? The answer to all of them is either yes, or they have it on their laptop/phone too.

Apple (and other laptop makers) weren’t charging $300 for 4tb even before the shortage. They realized that not everyone needed that and they way overcharged for that but still less than today’s prices. People who needed it bought it even though it would have been a lot cheaper for them to build a NAS. A current-gen M5 is perfectly capable of doing decent tk/s given enough RAM. In 2 or 3 years it would be even more capable, it just needs RAM.

Plenty of people will use clou hosted models like they do with storage today. Plenty of people will build home servers for AI. Just like plenty of people have home NASs. But the number of people who just want a large SSD on their laptop make the previous group a rounding error and those people would rather have a beefy laptop or desktop with enough RAM in it. If anyone can solve the Phone<->Desktop communication seamlessly it’s Apple after all.

Unlike storage however, a lot of people are now aware of how much data collection tech and AI companies are doing and the idea that AI models are training on their personal photos and prompts is deeply unappealing to many. With just storage, what could Google or Apple do? Certainly they wouldn’t just print your photos and put them in an ad or on their website. But with AI models, it’s a different equation

▲

flaunf221 2 days ago | parent [-]

> Every scenario you described applies the same to a NAS.

No. Files are your property. Inference is a service.

There's also the fact that working with files over network is often ill-advised and borderline impossible for some workloads. Even when that NAS is connected over wire and not wireless. Meanwhile even if you're on so-so connection, LLMs work great.

▲

eddythompson80 2 days ago | parent [-]

And storage can’t be a service?

	▲	flaunf221 2 days ago \| parent [-]
		A. Like I said - unless you invented magical communication channel that will give latency and speed and and reliability of local NVMe drive to your remote NAS, no, remote storage can't replace local storage in full. NAS is mostly good for backups, torrenting linux isos and serving them with Plex or Jellyfin. You are not installing software on your NAS instead of your machine. Or doing video editing. Or playing demanding games. I have file server at home and I still have ~8 TB of storage in my desktop, of which at leat 3 TB I can't move out. B. If I have 1 TB of my work files, my wife has 1 TB of her work files, that's 2 TB of total storage. If both us need LLM, it doesn't mean we need twice the LLM as one of us.