MiniMax M3 and DeepSeek v4-Pro are highly capable long context open weight multi-modal models. But long-context is a trap, because performance still falls dramatically after 150k-200k context.

> But long-context is a trap, because performance still falls dramatically after 150k-200k context.

I often see this repeated, and it is not true task to task. I work on this daily and we have several tasks where long context is advantageous and our evals against a whole battery of models with different windows show it as being so.

This is why having good evals for the tasks you're working on is so important.

I do grant it's a good rule of thumb.

> But long-context is a trap, because performance still falls dramatically after 150k-200k context.

I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.