I mean, you don't need to run it all the time, right? You do it once over your entire existing codebase to start and then once over the diff in your CI/CD pipeline when you make a new change. I'm sure it's not literally that simple but I doubt these need to churn 24/7/365 either.

▲

xerxes249 4 hours ago | parent | next [-]

In the Mythos blogpost they revealed to run the model like a 1000 times on the same code-base maybe with slightly different prompt or temperature. That suggests it will just be pay to win. If the 'attacker' spends more money/tokens than the 'defender' you will eventually be outclassed.

▲

sofixa 3 hours ago | parent [-]

It's even worse, it's loot box style. Not pay to win, but pay to have the chance to win. The result will always be non-deterministic, so for some cases it can give you what you're looking for from the first time, or it can take 1000 tries.

	▲	beering 2 hours ago \| parent [-]
		It’s never not been “loot box style”. None of your past hired security audits were guaranteed to catch all issues?

▲

vb-8448 4 hours ago | parent | prev | next [-]

You are supposed to run it on full codebase before any single PR gets merge.

▲

jazz9k 4 hours ago | parent | prev [-]

Companies don't make production pushes yearly. For many, it's two week sprints..and that's one project.

This doesn't make any sense cost-wise. It would be cheaper to just hire a security engineer.