I take this idea even further: After the LLMs have critiqued each other, I introduce a third critique and review it myself as a human. This third party review is most effective at highlighting problems that the LLMs miss, in my experience.

Jokes aside, I agree about having LLMs iterate. Bouncing between GPT and Opus is good in my experience, but even having the same LLM review its own output in a new session started fresh without context will surface a lot of problems.

This process takes a lot of tokens and a lot of time, which is find because I’m reviewing and editing everything myself during that time.

This is astrology for devs.

This is precisely how I used to use Beads before I made GuardRails (I wanted something slightly simpler, but similar with more 'guard rails'). I braindump everything I want to build, I ask Claude to do market level research. I then ask Claude to ask clarifying questions, when I ask Claude to be critical of its conclusions and provide the top options and to justify it. I also question Claude and say its okay to disagree with me, be critical, I just want to understand.

By the end you have piecemeal "tickets" for your coding agent, if you have multiple developers you can sync them all up into github, and someone could take some locally, or you can just have Claude work on all of them with subagents. The key feature there is because its all piecemeal the context stays per task.

Then I run a /loop 15m If you're currently working ignore this. Start on the next task in gur if you have not. If you finished all work and cannot pass one gate, work on the next available task.

(Note: gur is my shorthand for GuardRails)

I also added a concept called "gates" so a task cannot complete without an attached gate, gates are arbitrary, they can be reused but when assigned to a task those specific assignments are unique per task. A task is basically anything you want it to be: unit test, try building it, or even seek human confirmation. At least when I was using Beads it did not have "gates" but I'm not sure if it has added anything like it since I stopped using Beads.

Claude will ignore the loop if it's currently working, and when its "out of work" it will review all available tasks.

If anyone's curious its MIT Licensed and on GitHub:

https://github.com/Giancarlos/guardrails

I strongly believe you don’t need to call another model for that. The same model can do result fine. Just not as part of the same context.

I mean that if you ask codex on gpt 5.5 to submit to a plan reviewer subagent that uses gpt5.5, this is enough to have a very good reviewing and reassessment of the plan.

My hypothesis is that it’s even better than opus.

The reason why submitting the product of one LLM to another to review is that you need a fresh trajectory. The previous context might have “guided” the planer into some bias. Removing the context is enough to break free from that trajectory and start fresh.

I hate how seriously people take the output of an LLMs or how reliable they think it is.

Have Claude produce that spec 10 times, use the same prompt and same context. Identical requests, but you'll get 10 unique answers that wil contradict each other with each response seeming extermely confident.

Its scary how confident you people are in these outputs.

The return of pair programming.

Considering the domain being distributed systems, and aiming to implement "a Rust-based multi-Paxos consensus engine that not only implements all the features of Azure’s Replicated State Library (RSL)", I don't think we even have to look so deep into it, it's severely lacking tests.

If you're building a distributed system and you don't have more tests and testing code than actual code, by an order of magnitude most likely, then you're missing test coverage.

IIUC only 50k LoC are non-test code, which improves the metric. Whether that's enough tests still depends on the code. If most are getters and setters, the coverage might be ok.

I may have missed it but are those tests written by person or generated? Otherwise how do you know they even test anything (like actually test, not appear to test)

> I'm also shifting to an vibe coding workflow, but I have a genuine question: whenever I use AI for Rust, it makes an insane amount of lifetime errors. I have no idea how people are churning out so many lines of code so quickly.

What harness and model you've been using? For the last few months, essentially since I did the whole "One Human + One Agent = One Browser From Scratch" experiment, I've almost exclusively been doing cross-platform native desktop development with Rust, currently with my own homegrown toolkit basically written from scratch, all with LLMs, mostly with codex.

But I can't remember a single time the agent got stuck on lifetime errors, that's probably the least common issue in regards with agents + Rust I come across. Much bigger issue is the ever-expanding design and LLMs being unable to build proper abstractions that are actually used practically and reduces the amount of code instead of just adding to the hairball.

The issue I'm trying to overcome now is that each change takes longer and longer to make, unless you're really hardcore about pulling back the design/architecture when the LLM goes overboard. I've only succeeded in having ~10 minute edits in +100K LOC codebases in two of the projects I've done so far, probably because I spent most of the time actually defining and thinking of the design myself instead of outsourcing it to the LLM. But this is the biggest issue I'm hitting over and over with agents right now.

A lefthook:

format: glob: ".rs" run: cargo fmt -- --check

lint: glob: ".rs" run: cargo clippy -- -D warnings

tests: run: cargo test

audit: run: cargo audit

+ hooks that shove the lefthook automatically in the ai's face

---

rustfmt.toml:

edition = "2021" newline_style = "Unix" use_small_heuristics = "Max" max_width = 100

The irony of the machines having no mechanical sympathy is just too good

The feedback loop is the interesting part, if you use standard software engineering practices (modularise, test/document your interfaces, etc) then I find things like Claude Code do an exceptional job: since they can actually run cargo check/test themselves and can validate the tests too.

I see the complete opposite. The lower level the language, the less babysit the agent. Pure asm is the best, only with very advanced SIMD flags it has problems. C is excellent.

But python or typescript are full of errors all the time. I rather fallback to perl than python. Perl has been excellent all along.

I'm surprised to hear this. I have not had any issues here at all. The AI might clone things but I don't really care/ mind, I can ask it to refactor to make things zero-copy after, which is how I've often written Rust myself. I've never seen it overly wrap things in Rc.

I've not done any particular/ special prompting.

What kinds of programs are you writing and with what models? I'm curious if the lifetimes your programs require are trickier than most.

> Honestly, despite all the hype around Rust in the community, the fact that AI can't handle lifetimes reliably makes me reluctant to use it. The AI constantly defaults to spamming .clone() or wrapping things in Rc, completely butchering idiomatic Rust and making the output a pain to work with.

This hasn't been true since around gpt-4.5 on the OpenAI side of things. The 5.x models have been pretty much solid on Rust for a while now.

I’ve been writing almost exclusively Rust with LLMs and rarely ever hit this. I guess maybe the kind of work you are doing?

Yeah, LLMs suck at named lifetimes. The number of times I have seen Claude reach for indices and clones instead of just using proper named lifetimes is too many to count at this point. Not great for high-performance code!

I wrote and maintain this library of skills and workflows called Rust Bucket[0]

It sets up your repo to ensure agents use a workflow which breaks your user requests down into separate beads, works on them serially, runs a judge agent after every bead is complete to apply code quality rules, and also strict static checks of your code. It's really helpful in extracting long, high-quality turns from the agent. It's what we used to build Offload[1].

0: https://github.com/imbue-ai/rust-bucket : A rusty bucket to carry your slop ;)

1: https://github.com/imbue-ai/offload

I think it's due to the lack of quality instructions on what is good Rust code; AI often literally doesn't know what idiomatic Rust is. It can be good to have a reference where you write the basic rules that you want it to follow (ideal to assume it has no idea why spamming clone is bad and you're speaking to someone who has just watched one of those youtube videos with a dude in black t-shirt speaking very slowly and going over basic programming concepts as if they're breaking you out of the matrix).

> whenever I use AI for Rust, it makes an insane amount of lifetime errors.

What model are you using, and what frameworks are you using?

This is not a hard problem for LLMs to solve.

Rust is nearly the perfect language for LLMs.

It's exceptionally expressive, and it forbids entirely the most common globally complex bugs that LLMs simply do not (and won't for some time) have the context window size to properly reason about.

Dynamically typed languages are a disaster for LLMs because they allow global complexity WRT to implicit type contracts (that they do not and cannot be relied on to withhold).

If you're going to add types, as someone pointed out earlier, why are you even telling an LLM to write Python anyways?

Rust is barely harder to read than Python with types. It's highly expressive.

You have the `&mut` which seems alien, verbose (safe) concurrency, and lifetimes - which - if you're vibe coding... you don't really need to understand that thoroughly.

You want an LLM to write code in a language where "if it complies, it works" - because... let me tell you, if you vibe code in a language where errors are caught at runtime instead of compile time... It will definitely NOT work.

Lots and lots of guardrails to not allow slop.

In tsz I have hard gates that disallow doing work in the wrong crate etc.

https://github.com/mohsen1/tsz

Honestly Rust is an UGLY language. For whatever powers it possesses in memory safety, its cryptic symbology is reminiscent of assembly.

This is a problem when language designers are mathematicians and don’t understand typographical nuance and visual weights.

Clone is not "butchering idiomatic Rust", we gotta stop this nonsense

I feel like there's very little blogs that actually follow up on their experiment. It's just dopamine city.

It just an anchor. If it were 50k would you say the same down to 25k? And if so how many more times would it apply?

The interesting thing is that it was manageable solo (in many ways it's _more_ manageable solo+AIs than with coworkers+(their)AIs), and in such a short amount of time.

> It's almost guaranteed with agents you could do the same job with less than half of 100k lines.

That's great, non-test code is only ~47k lines of code.

For a startup with limited funding, building a product is no more a bottleneck. every one doesn't have the same access to funding!

It is described in the "Code Contracts" section of the article: "Code contracts specify preconditions, postconditions, and invariants for critical functions. These contracts are converted into runtime asserts during testing but can be disabled in production builds for performance". The .NET framework article that he links to: https://learn.microsoft.com/en-us/dotnet/framework/debug-tra...

All unit tests can pass if you don't assert anything. Just have to make sure to read through all 130k lines of code to check.

Learnings is irritating to me. The way kids use the word aesthetic is irritating too. I wonder if I might be that old man shaking his fist at the clouds, but I have gotten over begs the question, and literally, so maybe not yet...

I understand the instinct, but that's a bit too prescriptive for me.

https://en.wiktionary.org/wiki/learnings

A lesson would be a specific learning activity happening at a specific place and time, administered by a person more knowledgeable than you; like a teacher or mentor "giving a lesson".

If you're fine with the generalized form "learned a lesson", then surely "learnings" is fine too. There's no point in trying to police a completely normal and sensible use of language.

Yes, but it comes with much better “built-in” guardrails to rein in the autocomplete. Especially if compared to something runtime-surprise-prone-if-lovable like Ruby.