https://github.com/GAIR-NLP/ASI-Arch

This is being done, with great results so far. As models get better, architecture search and creation and refinment improves, driving a reinforcement loop. At some point in the near future the big labs will likely start seeing significant returns from methods like this, translating into better and faster AI for consumers.

I skimed the repo and only found sloop. Can you point out where I can find those new architectures you talk about?