The interesting thing about models this small is they should be able to be put on a single Taalas chip (the HC1 already runs a Llama 3.1 8B model). We're already at the point where half-decent reasoning could be run on an ASIC (and at mind-boggling speeds).

▲

pants2 6 hours ago | parent [-]

Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.

	▲	le-mark an hour ago \| parent [-]
		Given this and the quality of open models, it makes no sense to me that there’s a future for Anthropic et all?