Your history is all valid, but I don't think it really hits on the main motivations for how we got here.

Thread per request works perfectly fine if your application is CPU constrained.

However the observation was made, that most web applications are IO constrained, the majority of the time spent serving a web request is spent waiting for a database or downstream API.

Since most of the threads are idle waiting, your application needs many threads to optimally utilize the servers resources.

There was a perception(valid or not) that OS threads have too much memory and scheduling overhead.

Nginx came out using async io, and it could handle much more concurrent requests than apache, which used a threaded model, it sparked a lot of interest in different kinds of application managed scheduling.

It inspired initiatives like the reactive manifesto[0], which spawned tools like RXJava.

[0]: https://reactivemanifesto.org/