| ▲ | EagnaIonat an hour ago | |
> The joke is that macs are famously slow at prompt prefill and you are not getting anything back in 3 seconds. Your knowledge is out of date. In truth it depends on the Mac and the models used. I asked this question on M5 Max 128GB, using Ollama model Quen3.8:27b-mlx, with thinking enabled. Question: "Give me a python code snippet that opens a file and sorts the lines of text. " In 2.4 seconds it gave me 4 examples that work with different sorting configurations and a summary of when to use each. Compare that to an older model of gpt-oss:20b, took 5 seconds to finish thinking and 2 seconds to stream the answer. It gave me one python example snippet and two one liners that do the same thing. | ||
| ▲ | desterothx an hour ago | parent [-] | |
We are talking about models of the flash size, 100s of billions of parameters, don't listen to the media, size does matter | ||