Apple has a huge opportunity here to lead the market for machines to run local models if they step into it. Their stuff is already better than what nVidia is offering with stuff like the DGX Spark.

It's a niche market but it's a market that overlaps heavily with professionals in the AI space and lead developers, so it's a market that gets them customers in those roles.

If I were running Apple I'd call the RAM price bubble for what it is and temporarily eat some margin to offer machines with more RAM than competitors, especially these models that are great for edge AI, and capture market share.

▲

haskman 13 hours ago | parent | next [-]

A Mac Mini with M5 Pro, half the memory of a DGX Spark, half the disk of DGX Spark, and no CUDA, is already more expensive than a spark. The Spark runs Linux natively, has a tiny form factor, and is usable as a general purpose computer.

I use a DGX Spark with NixOS on it as my daily driver and it's fantastic.

▲

mwcampbell 21 hours ago | parent | prev | next [-]

> Their stuff is already better than what nVidia is offering with stuff like the DGX Spark.

How so? In tokens per second when running major open-weights models, or something else?

▲

Danox a day ago | parent | prev | next [-]

Apple isn’t the company that eats margins but they are company that would design around the problem and I think that’s what they will do after all, they have the design and engineering and plenty of money because they didn’t burn it on AI models or data centers.

	▲	api 13 hours ago \| parent [-]
		With AI Apple is doing their classic strategy I guess: wait and then follow with something that learns all the lessons the pioneers learned. They're letting everyone else pay for model training, with the obvious end of the road being open weights. They're letting everyone else overspend on first-generation AI data centers before the chip industry has truly optimized for today's AI designs (which will reduce data center footprint). They're letting everyone else play with and pioneer UI ideas. Meanwhile all they've done is put a few toes in the water: "Apple Intelligence" which is barely anything, and adding AI acceleration to their GPUs and doing a bit of up-market marketing to AI. So yeah they'll probably follow with a second generation Apple Intelligence that incorporates everything everyone else pioneered that worked, and an M7 or M8 line of chips with a GPU augmented with whatever approaches the other chip companies found worked best for running models... and built around the state of the art model architecture the industry finally converges on. By then RAM will probably be cheap again, so you'll get a 14" MacBook Pro with an M8 with a tensor-GPU and 512GiB of RAM.

▲

bigyabai a day ago | parent | prev [-]

Apple doesn't design GPUs on-par with Nvidia's efficiency yet. They need an architectural overhaul to be a serious competitor, which is what I'm expecting is queued up for M7.

Nvidia has CUDA, AMD has CDNA, and Apple has... compute shaders, I guess?

▲

nicce a day ago | parent | next [-]

> Apple doesn't design GPUs on-par with Nvidia's efficiency yet

How much it matters in inference? Most GPUs have enough computing for that and the bottleneck is the RAM speed and size. And M5 Ultra is becoming to challenge this.

	▲	bigyabai a day ago \| parent [-]
		For prefill, it's basically all that matters. Long-horizon agent tasks, session compaction, file reads and context manipulation will all hit the compute bottleneck in regular usage, incurring several minutes of latency on most Apple Silicon chips, regardless of RAM. It's kinda why memory bandwidth is an enormous red herring, even for datacenter applications. Nvidia's huge advantage is a compute-optimized GPU architecture and their Infiniband networking, their memory controllers aren't really the star of the show.

▲

Danox a day ago | parent | prev | next [-]

Probably two generations away. I’m more interested in how much uplift/speed and more importantly what is the power usage is required for the new computers Apple is shipping particularly for the Studio versions.

▲

newsclues a day ago | parent | prev [-]

Apple has metal and mlx