Isn't specialized hardware also a big risk? GPUs are more amenable to any big changes that may happen in the next 5, 10 years of AI research. Maybe we won't even be talking about LLMs anymore. Maybe matrix multiplication won't even be the main primitive.

▲

moduspol 2 hours ago | parent [-]

If matrix multiplication isn't the main primitive, I think we have a lot of pain coming our way.

	▲	amlib an hour ago \| parent [-]
		Maybe that is far fetched, but I could see them specializing for some super high dimension multiplication and meanwhile 5 years later turns out "all you need" are 3x3 matrices and suddenly 90% of your specialized hardware is now dark silicon :)