The same could be said about deepseek given they had to innovate with limited compute?

Exactly, but with the miniaturisation frontier still ahead of us.