Thanks - still not clear what they did really. Some inference time hacks?

That would imply the model always had a 1m token context but they limited it in the api and app? That's strange because they can just charge more for every token past 250k (like google does, I believe).

But if not shouldn't it have to be completely retrained model? it's clearly not that - good question!

Most likely still 32k tokens under the hood, but with some context slicing/averaging hacks to make inference not error out on infinite input.

(That's what I do locally with llama.cpp)

They already had 0.5M context window on the enteprise version.