Distillation does not require access to weights.