Remix.run Logo
myrmidon 5 hours ago

Are there significant usecases for the really small LLMs right now (<10b distills and such)?

My impression so far was that the resulting models are unusably stupid, but maybe there are some specific tasks where they still perform acceptably?

arkmm 5 hours ago | parent [-]

They're still very good for finetuned classification, often 10-100x cheaper to run at similar or higher accuracy as a large model - but I think most people just prompt the large model unless they have high volume needs or need to self host.