Apple's biggest bottleneck for real-world inference is prefill processing. They need a better GPGPU architecture, which is what I'm expecting M7 to reveal.

Agreed, seems like the M5 has already made steps in that direction, with 4x prompt processing / prefill performance vs. M4. [1] And token generation also got a 10% boost. The data below is only for Pro & Max but I think the base M5 got the same relative boosts vs. M4 base.

    Chip         BW (GB/s)   GPU Cores   Q4_0 Prompt   Q4_0 Gen
    M4 Pro (20c)    273         20          439.78        50.74
    M4 Max (40c)    546         40          885.68        83.06
    M5 Pro (20c)    307         20      ~1500 to 1700    ~56
    M5 Max (40c)    614         40      ~3000 to 3500    ~92