A RewriteInRustBench would be unironically useful at this point since all the main agents can write it reasonably well despite its relative scarcity in the input data.

▲

LarsDu88 2 hours ago | parent | next [-]

Rust is the best language for LLMs b/c it gives by far the best debug messages. Just tons of verifiable reward signal for post-training. Even the most rudimentary LLMs can school me on idiomatic Rust

▲

rafram 2 hours ago | parent | next [-]

On the other hand, Rust's borrow checker is very picky, and even a frontier LLM still sometimes struggles to respond to roadblocks sensibly (refactoring so whatever it's trying to do can be done safely) rather than stupidly (introducing some horrible global arena thing so it can make the borrow checker go away). A lot depends on how good your instructions are, and how good the existing code is, since bad input begets bad output.

	▲	nchie an hour ago \| parent [-]
		I've (more or less; I've read quite a bit of the code) vibecoded several houndred thousand lines of Rust and I've not seen this happen a single time. It sounds like something it'd do when you ask it to "write a linked list while satisfying the borrow checker". Are you sure you haven't (possibly unknowingly) been giving it instructions which ended up luring it into doing these things?

▲

bitexploder 2 hours ago | parent | prev | next [-]

Evidence needed. I think for certain kinds of outcomes it has very strong advantages, but these advantages are not a given as 'best for LLMs' :)

▲

lossolo 42 minutes ago | parent | prev [-]

Not always. In my experience, if you're not working on a small, trivial codebase, LLMs will sometimes just create spaghetti unreadable, inefficient code to satisfy the constraints of the type system/borrow checker.

▲

6thbit an hour ago | parent | prev | next [-]

Have each agent rewrite openssl in $lang and call it the RollYourOwnCrypto bench.

▲

culi 2 hours ago | parent | prev | next [-]

I wouldn't be surprised if we're already at the point of more LLM-written Rust than hand-written. Models training off models

▲

adamrezich 2 hours ago | parent | prev [-]

All the main agents can write Jai code reasonably well despite being even more scarce in input data!