> It would be too annoying for a human to ever write code to standards this high,

These are standards of a normal university operating systems course (assuming a university worth its salt anyway). Ridiculous to present it as some kind of feat that was "impossible" or even that rare.

▲

lrvick 4 days ago | parent [-]

Sure, a very smart and patient human could write this by hand in a long enough time horizon and unlimited budget. I should have said impossible in any realistic amount of time any responsible sponsor or employer would pay for.

▲

nicebyte 4 days ago | parent [-]

I'm sorry, this is simply not true.

this https://codeberg.org/lrvick/nit/src/branch/main/src/system/r...

is textbook definition of boilerplate. you need to be patient to cross-reference syscall tables but certainly not smart.

Again, students learning about operating systems do similar things as a normal part of their education process.

I'm not saying LLMs wouldn't help with this (in fact, this is the type of thing that becomes pretty much a no-op with them) but the notion that this is "impossible by any realistic standards" is risible, and if you have actually contributed to the kernel like you say, you should know better.

▲

lrvick 4 days ago | parent [-]

Rather than patiently referencing syscall tables by hand which is error prone and hard to review as a human, the LLM wrote tests to actually compare each of these to the kernel syscall tables to make my life a lot easier and avoid regressions or mistakes. Complex tests to write, that saves me a lot of time as a security reviewer. Can change any value and watch tests fail.

https://codeberg.org/lrvick/nit/src/branch/main/tests/kernel...

That is the point. Getting to results that satisfy security and functionality requirements in much less time.

Also, I was able to generate a bare minimal and hardened kernel config by using the LLM to write a tool to trace all kernel calls. Something that I would have normally been forced to punt on that LLMs made easy to justify.

LLMs are just as Linus Torvalds likes to say "Auto complete on steroids".

▲

fwip 4 days ago | parent [-]

The parsing code isn't even correct, though. I'm sure it works for the current input source code, but it's a clear sign of an LLM developing by iteration, rather than actually understanding the problem and coming up with a correct solution.

How often does "this table of constants was transcribed incorrectly" even come up as a bug class? Is it worth writing a buggy C parser for a failure mode that doesn't happen? Do you expect that Linux is going to change their constant definitions out from under you?

	▲	lrvick 4 days ago \| parent [-]
		Correct depends on what problem one is trying to solve though the parsing can certainly be cleaned up before release. It is a working WIP. > Do you expect that Linux is going to change their constant definitions out from under you Unlikely but I have been surprised before. Hardcoding to pretty much any modern kernel source version forever for this narrow use case is probably fine. The goal is about helping me and a reviewer quickly form confidence none of the magic numbers are mistaken or malicious in an automated way against the source of truth.