One of the most helpful usages of CC so far is when I simply ask:

"Are there any bugs in the current diff"

It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.

▲

GuB-42 3 days ago | parent | next [-]

I added this to my toolbox in addition to traditional linters.

My experience is that it is about 10% harmful, 80% useless and 10% helpful. Which is actually great, the 10% is worth it, but it is far from a hands off experience.

By harmful I mean something like suggesting a wrong fix to code that works, it usually happens when I am doing something unusual or counter intuitive, for example having a function "decrease_x" that (correctly) adds 1 to x. It may hint for better documentation, but you have to be careful not to go on autopilot and just do what it says.

By useless I mean something like "you didn't check for null" even though the variable can't be null or is passed to a function that handles the "null" case gracefully. In general, it tends to be overly defensive and following the recommendations would lead to bloated code.

By helpful I mean finding a real bug. Most of them minor, but for some, I am glad I did that check.

LLMs complement traditional linters well, but they don't replace them.

▲

csomar 3 days ago | parent [-]

> it usually happens when I am doing something unusual or counter intuitive,

That’s usually your signal that your code needs refactoring.

	▲	GuB-42 3 days ago \| parent \| next [-]
		I wouldn't say it needs refactoring. Maybe more documentation, or some work on naming. But I believe that code you write has to be at least a bit unusual. Every project worth making is unique. Otherwise, why not use something off the shelf? For example, let's say you want to shuffle songs for a music player, you write your shuffling algorithm and it is "wrong", but there is a reason it is "wrong": it better matches the expectations of the user than a truly random shuffle. A LLM trained on thousands of truly random shuffles may try to "fix" your code, but it is actually the worst thing you can do. That "wrong" shuffle is the reason why you wrote that code in the first place, the "wrongness" is what adds value. But now, imagine that you realize that a true random shuffle is actually the way to go, then "fixing" your code is not what you should do either, instead, you should delete it and use the shuffle function your standard library offers. The unusual/unique/surprising parts of your code is where the true value is, and if there is none of that in your codebase, maybe you are just reinventing the wheel. Now, if a LLM trips off these parts, maybe you need some documentation, as a way to tell both the LLM and a human reading that part that it is something you should pay attention to. I am not a fan of comments in general, but that's where they are useful: explaining why you wrote that weird code, something along the lines of "I know it is not the correct algorithm, but users prefer it that way".
	▲	Vegenoid 3 days ago \| parent \| prev [-]
		While this can be true, I think a lot of people are far too eager to say that because someone is doing something in an unusual way, it's probably wrong. Not everything fits the cookie cutter model, there is tons of software written for all kinds of purposes. Suggesting that they're writing code wrong when someone says "an LLM struggles with my unusual code", when we aren't actually looking at the code and the context, is not helpful.

▲

KTibow 3 days ago | parent | prev | next [-]

I'm surprised that works even without telling it to think/think hard/think harder/ultrathink.

▲

aurareturn 3 days ago | parent | prev | next [-]

I do the same with Github Copilot after every change.

I work with a high stakes app and breaking changes cause a ton of customer headaches. LLMs have been excellent at catching potential little bugs.

▲

bertil 3 days ago | parent | prev | next [-]

That matches my experience with non-coding tasks: it’s not very creative, but it’s a comprehensive critical reader.

▲

elcritch 3 days ago | parent | prev | next [-]

Recently I realized you can add Github Copilot as a reviewer to a PR. It's surprisingly handy and found a few annoying typos and one legit bug mostly from me forgetting to update another field.

▲

swyx 3 days ago | parent | prev | next [-]

maybe want to reify that as a claude code hook!

▲

hahn-kev 3 days ago | parent | prev | next [-]

We use Code Rabbit for this in our open source project.

▲

neucoas 3 days ago | parent | prev [-]

I am trying this tomorrow

	▲	lpa22 3 days ago \| parent [-]
		Let me know how it goes. It’s a game changer