Did Anthropic give you third-party benchmarks? Is that what you said to them? Yes, they're important, but the attitude is wrong.

bloppe 9 hours ago | parent | next [-]

Anthropic always publishes 3p benchmarks every time they announce a new model

	▲	MostlyStable 9 hours ago \| parent [-]
		And even if they didn't, they have a track record. Even if we did have benchmarks in this case I would still wait until people got there hands on it and formed a more holistic opinion.

fwipsy 5 hours ago | parent | prev [-]

Fudging benchmarks is a cheap way to get attention. If the model is really that good, it will have plenty of attention soon enough.

	▲	greenavocado 4 hours ago \| parent [-]
		Yeah, what happened to that scam startup that alleged to have made a model context window breakthrough a few weeks ago?