Can you please elaborate a bit more about your issues with PyTorch on M4 mac. I read PyTorch has some support for Mac GPU with MPS backend, but not sure how extensive it is. I am looking for a new machine, and use of PyTorch and LLM inference are one of the main uses. Sorry for being a bit off-topic from the thread. Thanks.

I couldn't get pytorch to use the mac GPU at all. I didn't spend ages on it - like about a day. In general I find the build environment on mac really annoying compared to say Linux so it may be that with more patience it could be done. IT was also on my work computer and setting up Xcode command line tools on my work mac is super annoying for whatever dumb reason, but basically getting any kind of prebuilt python version (like say the ones that ship with rye) to talk to your Xcode if it's not in the same path as the one that built the python version is annoying, so then you're looking at building the whole of python which is not that hard but (when I tried it) meant it broke the next time Xcode upgraded. I tried symlinks to the sdk etc but that was no bueno for reasons I don't quite remember. And then even having done that and patched pytorch it didn't actually run GPU accelerated.

So bottom line is it might be possible to make it work but in my brief attempt I couldn't get it to work.

If you're at all linux-capable I would say it's hands down a better dev experience in every way (and that's coming from someone who's used macs for years and years in both a professional and private capacity).