| ▲ | furyofantares 28 minutes ago | |
Would local models catch up if frontier models paused new training for a year? I'm not sure for. It might just be you can get these nice local models because of distillation from a much more powerful model. | ||
| ▲ | swiftcoder 20 minutes ago | parent [-] | |
Adversarial distillation hasn’t been a major factor in the past year. If the SOTA stops moving, all the other labs catch up to them in 6 months to a year, and distillation into smaller models continues to close the gap between small/large models during that time | ||