Proper verification would be deterministic rules the JIT-ed code would be checked against, so it would have to be non-JIT. Since the comment is talking about using more & more JIT for security reasons, I assumed it extended to verification too.

I don't get why you'd generally switch to JIT for security reasons.

▲

daishi55 a day ago | parent | next [-]

Why can’t you JIT- generate the code and then run the verification?

▲

Ekaros a day ago | parent | next [-]

Because after proper verification you very much aren't at JIT territory. Take the network stack. From my memories a proper commercial test suit from say Defensics is from a few thousand to tens of thousand cases. For each of Ethernet, IP, TCP and UDP. Any decent coverage however fast will take some time. Most likely lot more than you want with just in time...

	▲	mihaelm a day ago \| parent [-]
		Yeah, the cost of running the test suite every time an agent generates code JIT would be the killer for running it at scale. I didn't even get into practicality of that approach too much because the gist of it for me was the question of agents writing code JIT having security benefits (?) - the verification thing is a sidetrack. Certainly, adding agentic JIT code invites more complexity to properly do it, so what are the benefits that call for introducing it.

▲

mihaelm a day ago | parent | prev [-]

Where did I say you can't do that? I answered you why I assumed the original comment was leaning towards JIT for everything.

▲

daishi55 a day ago | parent [-]

Ok so if it is indeed possible to verify the code after JIT generating it, I still don’t understand why you would assume that in the hypothetical scenario we aren’t doing that verification?

You literally said it can’t be JIT, I’m just saying it seems like it can.

	▲	mihaelm a day ago \| parent [-]
		Again, I answered that in the earlier reply: > Since the comment is talking about using more & more JIT for security reasons, I assumed it extended to verification too. I wouldn't consider feeding the JIT-ed code to a JIT-ed verifier code proper verification. Even ignoring the verification bit, I don't see the security benefits of having agents write JIT code, hence the original question. Because there might be some edge cases I don't know about, but I can't see this applying generally.

▲

patmorgan23 a day ago | parent | prev [-]

Who writes the formal verification spec and when?

	▲	daishi55 a day ago \| parent [-]
		I suspect there are already plenty of test suites out there for something as universal as a networking stack.