The context window has been shattered: Subquadratic debuts a 12M token window

refibrillator 7 hours ago | parent | next [-]

Previous discussion here (with links to actual primary source):

https://news.ycombinator.com/item?id=48023079

No technical report published yet, unlikely code or weights will be either given VC funding.

Chance-Device 6 hours ago | parent | prev | next [-]

It’s probably something like deepseek’s native sparse attention with content based granularity. They might not be publishing anything because it’s not such a strong value proposition and doing so would lead to commentary that would tank their investment opportunities.

▲

SilverElfin 6 hours ago | parent [-]

Or maybe because giving it away would tank their investment opportunities.

	▲	regularfry 5 hours ago \| parent [-]
		There's ways and means. Pushing something out in the sub-30B range would gain them mindshare and they could keep bigger models to themselves. I can't see any indication of what size their model is though.

▲

flowerthoughts 4 hours ago | parent | prev | next [-]

For Claude Code, I feel 1M is enough. I've had a compaction once, but that was because I was forcing Claude to do something it clearly had a hard time understanding.

For general chat bots where the user doesn't understand what a context window is, what do you do about context? Latest few messages and then a memory tool? Compaction?

	▲	anon22981 an hour ago \| parent \| next [-]
		I feel the 1M context is way too large —- the model gets ”drunk” way before it gets anywhere near 1 million. Imo the 1M context window is a huge downgrade.
	▲	whoodle 3 hours ago \| parent \| prev \| next [-]
		This is so interesting to me, I frequently experience compaction on long running features and still find Claude is better than starting with fully fresh prompts. Every dev seems to use these tools differently.
	▲	brianush1 4 hours ago \| parent \| prev [-]
		Claude does compaction in the regular web chat interface now, too

▲

Alifatisk 2 hours ago | parent | prev | next [-]

Waiting for the paper and model card. I believe it when I see it.

▲

roger_ 7 hours ago | parent | prev | next [-]

Have they published?

▲

Bombthecat 7 hours ago | parent | prev | next [-]

I believe it, when I see it.

▲

t4sty 4 hours ago | parent | prev | next [-]

Feels like this is to AI what JPG is to image

▲

dfilppi 6 hours ago | parent | prev [-]

[dead]