This is just another example of the bitter lesson. In a year a model will come out that will make none of these model specific optimizations you made matter.

Yeah

But the point is that at any moment, there is friction in switching