To be honest, I'm getting tired of a "laptop" in every one of these clickbait titles turning out to be $3000 Macbook. Sure, it's impressive to achieve this degree of the LLM compression, but I really don't like that the title implies local LLM becomes a viable for an average person with the actual hardware being out of reach for 99%.

▲

Computer0 2 hours ago | parent | next [-]

Yeah I understand the sentiment, I think it should’ve been “,on a laptop!” instead of “on a laptop”

▲

throw284959 2 hours ago | parent | prev [-]

I ran full version of this model without any swapping on cluster of 2x $3000 laptops (strix halo zbook 128GB) at about 20 tokens per second.

I would say it is in reach for normal person. If anything buying it was great investment, it is work tool, I will probably sell it for more than what I bought it for :)

	▲	prmoustache an hour ago \| parent [-]
		> I would say it is in reach for normal person. That is a very wealthy country centric thing to say.