Yeah, that's because the original npus were a rush job, the amd AI Max is the only one that's worth anything in my opinion.

▲

SomeHacker44 19 hours ago | parent | next [-]

I have a Strix Halo 395 128GB laptop running Ubuntu from HP. I have not been able to do anything with the NPU. I was hoping it could be used for OpenCL, but does not seem so.

What examples do you have of making the NPU in this processor useful please?

	▲	pseudosavant 11 hours ago \| parent [-]
		All the videos I've seen of AI workloads with an AMD Strix Halo with 128GB setup have used the GPU for the processing. It has a powerful iGPU and unified memory more like Apple's M chips.

▲

mrinterweb a day ago | parent | prev | next [-]

The Apple M series chips are solid for inference.

▲

hamdingers a day ago | parent [-]

Correct me if I'm wrong, but I thought everyone was still doing inference on the GPU for Apple silicon.

▲

mrinterweb a day ago | parent [-]

The Apple M series is SoC. The CPU, GPU, NPU, RAM are all part of the chip.

	▲	13 hours ago \| parent \| next [-]
		[deleted]
	▲	wtallis a day ago \| parent \| prev [-]
		The RAM is not part of the SoC. It's a bunch of separate commodity RAM dies packaged alongside the SoC.

▲

a day ago | parent | prev | next [-]

[deleted]

▲

Dylan16807 a day ago | parent | prev [-]

Is that because of the actual processing unit or because they doubled the width of the memory bus?

	▲	zamadatix a day ago \| parent [-]
		It's because it comes with a decent iGPU, not because of the NPU inside of that. The NPU portion is still the standard tiny 50 TOPS and could be filled with normal RAM bandwidth like on a much cheaper machine. On the RAM bandwidth side it depends if you want to look at it as "glass is half full" or "glass is half empty". For "glass is half full" the GPU has access to a ton of RAM at ~2x-4x the bandwidth of normal system memory an iGPU would have and so you can load really big models. For "glass is half empty" that GPU memory bandwidth is still nearly 2x less than a even a 5060 dGPU (which doesn't have to share any of that bandwidth with the rest of the system), but you won't fit as large of a model on a dGPU and it won't be as power efficient. Speaking of power efficiency - it is decently power efficient... but I wouldn't run AI on battery on mine unless I was plugged in anyways as it still eats through the battery pretty quick when doing so. Great general workstation laptop for the size and wattage though.