| ▲ | cmiles8 2 hours ago | |
Small open weight local models are the future. While hosted mega models make headlines for doing cool stuff, the vast majority of applications for AI simply don't need all that power, and thus cost. That’s a big part of why businesses are screaming that there’s no ROI from AI. Brining this tech down into small local models is likely where this all converges for the vast majority of use cases and what solves the present ROI crisis for LLM-based AI. | ||
| ▲ | scotty79 2 hours ago | parent [-] | |
If you are into small local models I highly recommend vibe thinker. It's a model trained specifically for reasoning. Basically a problem solver. When compared with other models, on math problems benchmarks, it's closer to models hundred times its size than ten times its size which it beats comfortably. It supports long contexts on limited VRAM and is blazing fast. | ||