Tunnel vision? If your model can handle big context, why divide into lesser problems to conquer - even if such splitting might be quite trivial and obvious?

It's the difference of "achieve the goal", and "achieve the goal in this one particular way" (leverage large context).

▲

wat10000 8 hours ago | parent [-]

I meant, if the claim here is that small models can accomplish the same things with good scaffolding, why didn’t they demonstrate finding those problem with good scaffolding rather than directly pointing them at the problem?

▲

mattmanser 5 hours ago | parent [-]

They don't have to.

Lot of people in this thread don't seem to be getting that.

If another model can find the vulnerability if you point it at the right place, it would also find the vulnerability if you scanned each place individually.

People are talking about false positives, but that also doesn't matter. Again, they're not thinking it through.

False positives don't matter, as you can just automatically try and exploit the "exploit" and if it doesn't work, it's a false positive.

Worse, we have no idea how Mythos actually worked, it could have done the process I've outlined above, "found" 1,000s of false positives and just got rid of them by checking them.

The fundamental point is it doesn't matter how the cheap models identified the exploit, it's that they can identify the exploit.

When it turns out the harness is just acting as a glorified for-each brute force, it's not the model being intelligent, it's simply the harness covering more ground. It's millions of monkeys bashing type-writers, not Shakespeare at one.

	▲	wat10000 an hour ago \| parent \| next [-]
		It’s strange to see this constant “I could do that too, I just don’t want to” response. Finding an important decades-old vulnerability in OpenBSD is extremely impressive. That’s the sort of thing anyone would be proud to put on their resume. Small models are available for anyone to use. Scaffolding isn’t that hard to build. So why didn’t someone use this technique to find this vulnerability and make some headlines before Anthropic did? Either this technique with small models doesn’t actually work, or it does work but nobody’s out there trying it for some reason. I find the second possibility a lot less plausible than the first.
	▲	LordDragonfang 2 hours ago \| parent \| prev \| next [-]
		> If another model can find the vulnerability if you point it at the right place, it would also find the vulnerability if you scanned each place individually. They didn't just point it at the right place, they pointed it at the right place and gave it hints. That's a huge difference, even for humans.
	▲	Oranguru 3 hours ago \| parent \| prev [-]
		[dead]