Is there any usefulness with the small large language models, outside perhaps embeddings and learning?

I fail to see the use-case on a Pi. For learning you can have access to much better hardware for cheaper. Perhaps you can use it as a slow and expensive embedding machine, but why?

▲

kouteiheika 8 hours ago | parent [-]

A natural language based smart home interface, perhaps?

Tiny LLMs are pretty much useless as general purpose workhorses, but where they shine is when you finetune them for a very specific application.

(In general this is applicable across the board, where if you have a single, specific usecase and can prepare appropriate training data, then you can often fine-tune a smaller model to match the performance of a general purpose model that is 10x its size.)

	▲	michaelmior 8 hours ago \| parent [-]
		I think there's a lot of room to push this further. Of course there are LLMs being used for this case and I guess it's nice to be able to ask your house who the candidates were in the Venezuelan presidential election of 1936, but I'd be happy if I could just consistently control devices locally and a small language model definitely makes that easier.