You can’t build a frontier model with one single thing. This is an incremental improvement but it doesn’t explain the entire success of the model. The training set is immensely important, regardless of how you feel about distillation.

▲

EGreg 3 hours ago | parent [-]

Reminds me of this btw:

https://www.bbc.com/news/technology-12343597

Microsoft replied that Bing uses “many different signals” —- including cribbing from Google :-)

	▲	krlx 2 hours ago \| parent [-]
		I remember 15 years ago or so, one of my first student job was to evaluate Bing results compared to the same query on Google. Didn't know then that I was a distillation attacker.