You can run a model with substantially similar capabilities to Claude or ChatGPT locally, with absolute data privacy guaranteed. Whereas with Claude or ChatGPT, all you can do is trust and hope they won’t use your data against you at some point in the future. If you’re more technically minded, you can hack on the model itself, the sampling method, etc., and have a level of fine-grained control over the technology that isn’t possible with a cloud model.

▲

ukuina 10 months ago | parent [-]

> You can run a model with substantially similar capabilities to Claude or ChatGPT locally

I am all for local models, but this is massively overselling what they are capable of on common consumer hardware (32GB RAM).

If you are interested in what your hardware can pull off, find the top-ranking ~30b models on lmarena.ai and initiate a direct chat with them on the same site. Pose your common questions and see if they are answered to your satisfaction.

	▲	anon373839 10 months ago \| parent [-]
		Two points: 1) I actually think that smaller models are substantially similar to frontier models. Of course the latter are more capable, but they’re more similar than different (which I think the ELO scores on lmarena.ai suggests). 2) You can run much larger models on Apple Silicon with surprisingly decent speed.