I won't try to defend Chomsky. (Not really a big fan even before this.) But if the mere mention of him is sus to you then I advise you to not study either linguistics or computer science because it's Chomsky normal forms and Chomsky hierarchies all the way down. There's even still people clinging to some iteration of the universal grammar despite the beating it has taken lately.

He's also one of the most prominent political thinkers on the American hard left for the last half century.

There's a joke going around for a while now that you either know Chomsky for his politics, or for his work in linguistics and discrete mathematics, and you are shocked to discover his other work. I guess we can extend that to a third category of fame, or infamy.

The merge operation in the later Chomsky modern linguistics program is similar in a lot of ways to transformer's softmax merging of representations to the next layer.

There's also still a lot to his arguments that we are much more sample efficient. And it isn't like monkies only learn language at a gpt-2 level, bigger brains take us to gpt-8 or whatever. There's a step change where they don't really pick things up linguistically at all and we do. But with a lot more data than we ever get, LLMs seem to distill some of the broad mechanisms what may be our innate ability, though still seems to have a large learned component in us.