| ▲ | Bonsai: A 27B reasoning model on a 16 GB M2 Mac, with Ferrox(medium.com) | |
| 3 points by antonellof 9 hours ago | 1 comments | ||
| ▲ | antonellof 9 hours ago | parent [-] | |
Run a 27B reasoning model locally on a 16GB M2 Mac. Ferrox + ternary quantization delivers 5.9GB models without sacrificing much quality. | ||