A transistor is very simple too, and here we are. Don’t dismiss something because it’s simple.

You got to look at how it scales. LLMs have already stopped increasing in parameter count as they don't get better by scaling them up anymore. New ideas are needed.