I hope their open source variants are just as good, having a 1 million token window for a fully offline model would be VERY interesting.

I don't know how well it performs, but you can extend Qwen3.5 to 1 million token context using YaRN. Also, Nemotron 3 Super was recently released and scales up to 1 million token context natively.