The M6 doubled the neural engine from 16 to 32 cores. I would expect that the M7 doubles that again to 64 from 32? That would make sense.

I believe that the CPUs are actually limited by ram bandwidth more than the neural engine right when it comes to LLM processing?

Maybe the M7 introduces something new to get around the current ram bandwidth problems on the non-Ultra chips.

▲ bigyabai 2 hours ago | parent | next [-]

Apple's biggest bottleneck for real-world inference is prefill processing. They need a better GPGPU architecture, which is what I'm expecting M7 to reveal.

	▲	tedd4u 2 hours ago \| parent [-]
		Agreed, seems like the M5 has already made steps in that direction, with 4x prompt processing / prefill performance vs. M4. [1] And token generation also got a 10% boost. The data below is only for Pro & Max but I think the base M5 got the same relative boosts vs. M4 base. `Chip BW (GB/s) GPU Cores Q4_0 Prompt Q4_0 Gen M4 Pro (20c) 273 20 439.78 50.74 M4 Max (40c) 546 40 885.68 83.06 M5 Pro (20c) 307 20 ~1500 to 1700 ~56 M5 Max (40c) 614 40 ~3000 to 3500 ~92` [1] https://www.hardware-corner.net/m5-pro-m5-max-local-llm-4x-f...

▲ wmf 2 hours ago | parent | prev [-]

LPDDR6 is coming.

▲

bhouston an hour ago | parent [-]

I understand that will boost read rates to around 14 Gbps as compared to the current 10 Gbps for LPDDR5X, so a 40% improvement.

▲

wmf 26 minutes ago | parent [-]

The bus is also 50% wider so the bandwidth is double.

	▲	bhouston 19 minutes ago \| parent [-]
		Or they could make use of LPDDR5X-PIM? That would be such a killer feature and competitive advantage.