> it's so cheap as to be nearly free

Both chips and developer time are expensive. Massively so, both in direct cost and secondary and tertiary elements. (If you think hiring more developers to optimise code has no knock-on effects, I have a bridge to sell you.)

There isn't an iron law about developer time being less valuable than chips. When chip progress stagnates, we tend towards optimising. When the developer pipeline is constrained, e.g. when a new frontier opens, we tend towards favouring exploration over optimisation.

If a CS programme is teaching someone to always try to optimise an algorithm versus consider whether hardware might be the limitation, it's not a very good one. In this case, when it comes to AI, there is massive investment going into trying to find more efficient training and inference algorithms. Research which, ironically enough, generally requires access to energy.