There are even companies starting to offer distillation as a service https://inference.net/explore/model-training