| ▲ | myrmidon 5 hours ago | |
Are there significant usecases for the really small LLMs right now (<10b distills and such)? My impression so far was that the resulting models are unusably stupid, but maybe there are some specific tasks where they still perform acceptably? | ||
| ▲ | arkmm 5 hours ago | parent [-] | |
They're still very good for finetuned classification, often 10-100x cheaper to run at similar or higher accuracy as a large model - but I think most people just prompt the large model unless they have high volume needs or need to self host. | ||