Remix.run Logo
▲ furyofantares 28 minutes ago

Would local models catch up if frontier models paused new training for a year?

I'm not sure for. It might just be you can get these nice local models because of distillation from a much more powerful model.

▲swiftcoder 20 minutes ago | parent [-]

Adversarial distillation hasn’t been a major factor in the past year. If the SOTA stops moving, all the other labs catch up to them in 6 months to a year, and distillation into smaller models continues to close the gap between small/large models during that time