Rather than patiently referencing syscall tables by hand which is error prone and hard to review as a human, the LLM wrote tests to actually compare each of these to the kernel syscall tables to make my life a lot easier and avoid regressions or mistakes. Complex tests to write, that saves me a lot of time as a security reviewer. Can change any value and watch tests fail.

https://codeberg.org/lrvick/nit/src/branch/main/tests/kernel...

That is the point. Getting to results that satisfy security and functionality requirements in much less time.

Also, I was able to generate a bare minimal and hardened kernel config by using the LLM to write a tool to trace all kernel calls. Something that I would have normally been forced to punt on that LLMs made easy to justify.

LLMs are just as Linus Torvalds likes to say "Auto complete on steroids".

▲

fwip 4 days ago | parent [-]

The parsing code isn't even correct, though. I'm sure it works for the current input source code, but it's a clear sign of an LLM developing by iteration, rather than actually understanding the problem and coming up with a correct solution.

How often does "this table of constants was transcribed incorrectly" even come up as a bug class? Is it worth writing a buggy C parser for a failure mode that doesn't happen? Do you expect that Linux is going to change their constant definitions out from under you?

	▲	lrvick 4 days ago \| parent [-]
		Correct depends on what problem one is trying to solve though the parsing can certainly be cleaned up before release. It is a working WIP. > Do you expect that Linux is going to change their constant definitions out from under you Unlikely but I have been surprised before. Hardcoding to pretty much any modern kernel source version forever for this narrow use case is probably fine. The goal is about helping me and a reviewer quickly form confidence none of the magic numbers are mistaken or malicious in an automated way against the source of truth.