| ▲ | srcreigh 2 hours ago | |
No. The bitter lesson is about capabilities. GP is talking about efficiency. GP isn’t suggesting that focused narrow model(s) will be more capable than large model, but that many small focused models can have sufficient capability while being more optimal. Also, the bitter lesson is just wrong. The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions (eg tools internet docs). LLMs would not be as good without expertly written context, and expert context without LLMs aren’t as good either. | ||