“We believe AI is for everyone” *who can afford two M5 Studios with a total of 1 terabyte of memory.

Very big LLMs can still be useful even when you run them only at the speed of around 1 token per second that can be reached with almost any mini-PC or laptop with the weights streamed from SSDs.

While at that speed they will not be your main LLM, there are tasks that are still useful even when requiring many hours, e.g. an overnight run, for instance for checking what has been produced after many interactive iterations of a fast but small model.