I wonder how the Chinese labs are training a 3 trillion parameter model on what has to be vastly smaller compute resources. If the U.S. compute advantage is persistent, it's hard to imagine that Chinese labs will be able to keep pace forever, as a matter of physics, but... so far they seem to be doing just fine.

▲

dopa42365 2 hours ago | parent [-]

It's not like same parameter count models are identical, so that doesn't appear to be an indicator for quality, or even compute requirements?

There seems to be more to producing a better model than brute forcing parameter count after all.

	▲	tibbar an hour ago \| parent [-]
		Training and serving large models does require increasingly more compute, though. (The Chinese labs have clearly found some massive optimizations, but my point was that you'd think at some point even those optimizations wouldn't be enough to keep up with exponentially increasing model sizes.)