Can someone explain what are teachers in Multi-Teacher On-Policy Distillation? I can imagine math, coding and other verifiable domains, but they also have biology? Is it where distillation from bigger models come in?

▲

sosodev 4 hours ago | parent | next [-]

They reference https://thinkingmachines.ai/blog/on-policy-distillation/

If I understand correctly, it's distillation via having a teacher model score each of the student's tokens for a problem based on their own probabilities of generating that token at each step in the sequence. The reward/loss is then applied as RL.

The multi-teacher bit seems to imply they're distilling from multiple models. It's light on the details, but it seems like it could be part of distilling from frontier/closed models. Provided they calculate the logprobs, which OpenAI seems to allow via API but not Anthropic. Maybe they have a way of estimating the logprobs externally?

This method can be used to learn any domain from the teacher. Biology included.

	▲	htrp 2 hours ago \| parent \| next [-]
		> which OpenAI seems to allow via API but not Anthropic Does openai still allow logprobs in their current gen models?
	▲	mike_hearn 4 hours ago \| parent \| prev [-]
		It's not that light on the details. I read the paper and they train several different models in parallel over a few different domains and then they distill from their own models to get the final model.

▲

porridgeraisin 4 hours ago | parent | prev [-]

They're other models yes.