> even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today

I hate to dogpile on this statement but I can think of two major issues right now:

* Small context windows, and serious degradation when pushing the limits of existing context windows. A human can add large amounts of state to their "context window" every day.

* Realtime learning. My humans get smarter every day, especially in the context of working with a specific codebase.

Maybe the AI companies will figure this out, but they are not "same technique more processor power" kinds of problems.