> Like maybe we should have dev servers that are deliberately throttled back

This is a good point. Often datasets are smaller in dev. If a reasonable copy of live data is used, devs would have an intuition of what is making things slow. Doesn't work for live data that is too big to replicate on a developer's setup though.