I’m very surprised this isn’t getting more attention. Am I missing something?

It seems at or above SOTA on the given benchmarks, doesn’t have context rot, is orders of magnitude faster, and uses less compute that current transformer models. I suppose it’s just an announcement and we can’t test it ourselves yet.

▲

alexsubq 2 days ago | parent | next [-]

We are SOTA in some ways and not in others, continuously working to make it better! We need a little more time to scale, as we are working on things like disaggregated prefill, etc., the norms of large-scale model infra.

I am happy to answer any questions!

▲

supern0va 2 days ago | parent | next [-]

This seems super cool if as described, but I'm sure you can understand the skepticism.

Do you anticipate having any kind of public accessible chat interface for testing in the near future?

Also, what, if any, benefits are there for smaller context windows? Is there still a material improvement in cost to serve under say 256k? I'm curious about the broader implications for the space beyond improvements for very large context windows.

▲

alexsubq 2 days ago | parent [-]

I do, for sure! Yes, we have a few product rollouts lined up. The differentials for latency are posted in our blog post, so that should provide an idea of where the scaling law differentials kick in.

	▲	dvfjsdhgfv a day ago \| parent [-]
		> I do, for sure! Yes, we have a few product rollouts lined up. When, more or less?

▲

dirtyalt a day ago | parent | prev [-]

I have questions.

Can you back up your claims?

Why did you not release the white paper in parallel with the product?

Feels really fishy.

▲

jakevoytko 2 days ago | parent | prev | next [-]

The proof is in the pudding. At this point, there have been plenty of models that overperformed on benchmarks and underperformed on real work. So my stance is that I'm curious, I'm excited to see where it goes, and I don't believe it until I can try it.

▲

amw-zero a day ago | parent | prev | next [-]

Yes you're missing something: the snake oil.

▲

dvfjsdhgfv a day ago | parent | prev | next [-]

> Am I missing something?

Yes, this product doesn't exist.

And the last time a company claimed something similar it disappeared after taking money from investors.

▲

remaximize 2 days ago | parent | prev | next [-]

I agree, it's a real architectural breakthrough if true

▲

shdh 2 days ago | parent | prev [-]

no one has access to it yet

no published benchmarks

no paper

no demonstrations of capabilities