This seems to be the inherent trade-off with smaller models. They can be just as "bright" as their larger contemporaries but they necessarily can't encode as much knowledge.

I think over time we'll realize this is the way forward, you want the smartest model with the least amount of embedded knowledge and you provide the knowledge externally or merge it into the weights for the domain you're interested in. Then you don't have to train a ginormous model every 6 months to pull in new information.

I think this is what the n-gram table in Qwen 4 (and 3.8-Flash-Next) is for. We’ll see if it comes down to the 27b variant.

A RAG solution would make time to first token awful.