>We've automated ourselves into Goodhart's law.

Yes.

This does not however mean that progress is not being made.

It just means the progress is happening along such dimensions that are completely illegible in terms of the culture of the early XXI century Internet, which is to say in terms of the values of the society which produced it.

Feels like a parallel with https://en.wikipedia.org/wiki/Constructivism_%28philosophy_o... where "it's not valid until you checked"