You are aware that a 1 trillion parameter model won’t run on any quantity of consumer hardware, even heavily quantized… you need a data center for that. And that is not even to talk of the Kimi K3 model which is in the ballpark of 3 trillion

▲

throw1234567891 2 days ago | parent | next [-]

You don’t need a datacenter. 1T model requires 1T of RAM. Not easy to come by today in a compact factor but nowhere near the datacenter.

	▲	nullc 21 hours ago \| parent [-]
		> 1T model requires 1T of RAM assuming 8 bits per weight, but 4bits is commonly used once you get to that scale.

▲

layla5alive 2 days ago | parent | prev [-]

You do not need a data center - just a single rack will do to run it fast. Or about 100x 5090s.