Silicon Mac’s are great for running LLMs. Unified memory and memory bandwidth of the Max and Ultra processors is very useful in doing inference locally.

▲

bigyabai 6 days ago | parent | next [-]

Great news, but entirely lost on commercial hyperscalers and much of the PC market. Apple's recalcitrance towards supporting Nvidia drivers basically killed their last shot at real-world rackmount deployment of Apple Silicon. Now you can go buy an ARM Grace CPU that does the same thing, but cheaper and with better software support.

	▲	seanmcdirmid 5 days ago \| parent [-]
		You really can’t. NVIDIA’s arm chip still looks nerfed compared to apple’s offering, and…I can run 40GB sized LLMs on the plane with no internet…it’s not something that you can do with any other platform.

▲

pstuart 5 days ago | parent | prev [-]

I imagine they recognize the need for increased memory bandwidth and will bump that up significantly -- they are well positioned to do so.