3T parameters! I've been really impressed with Kimi's coding performance. It's replaced a lot of my Claude Code workflows, which just got way too expensive. Wonder how big Fable and Opus are in comparison.

▲

thway15269037 5 hours ago | parent [-]

Chinese labs seem to think that fable/mythos is a 10T+ model, and target that amount of parameters with next iteration of qwen/kimi/ds/glm. Proves nothing tho.

▲

zozbot234 5 hours ago | parent [-]

With common use of n-gram tables, comparing "params" count has gotten harder. Those are exceptionally sparse and light on compute, way beyond even the sparsest MoE experts. You could host them on a slow spinning disk and still have a viable local inference platform.

▲

thway15269037 4 hours ago | parent [-]

So that's why I can't buy a replacement 16TB HDD for less than a 1000$ when three years ago I bought them for 229$ a pop.

I wondered why do they even need slow HDDs, like, do they store training data on them? User chats? Do they assemble crazy 12-way stripes for fast speed? Do they store what, failed attempts?

	▲	selectodude 4 hours ago \| parent [-]
		I think that’s just because hoovering up the entire worlds worth of data and saving it to a RAID multiplied by every frontier lab is a lot of fucking hard drives.