In reality, I feel like this tends to translate to "we need to scale horizontally and make sure our code runs on a large number of machines" with little consideration for actually optimizing the code itself.

In fact, devoting time to code optimization tends to be explicitly discouraged in my experience. The prevailing wisdom seems to be, "write shitty code... just make sure it runs on as many AWS machine instances as possible."