> beside-the-point semantic nits all over the place.

This is also a problem with Copilot Reviews on GitHub.

We have them enabled (but opt in) and they have, multiple times, spotted quite useful things.

Sure often the thing they spot is just half right, like it spots the place where a problem is but not quite the relevant problem but by reading it (and taking it serious) you then notice the actual problem.

This involved finding a bunch of nasty race conditions.

And many ways where doc and code was out of sync which could have caused pretty bad outcomes further down the line.

But the problem is it is too obsessed with finding 2-4 but not more things, leading to two issue:

1. even if there are 10 non overlapping issues it often will tell them to you bit by bit over 2-3 runs after you fix the previous issues. This is very annoying/high friction.

2. once there isn't much to find anymore it comes up with increasingly more annoying nit picks not one cares. Thinks like minor unclearness in formulation no one would get wrong, spell correcting non-doc comments for things like `foos => foo's` and similar etc. All indeed wrong, but also all things where fixing them adds 0 business value. Obsessing that for an aliased function name where, both names are equally good, one specific name must be used and naturally always the name you didn't use even if this is the most widely used name in the code base. And similar non-bussiness value nonsens. Worse it will starting classifying such minor non business value issues as "high" and hallucinate reasons why supposedly minor style issues will lead to very bad runtime error or other nonsense.

This has me very split about the feature, on one hand is has proven quite useful, on the other hand it can very annoying, high friction and pushes people to wast time on non-business value nit pick (which are fine to fix if you anyway touch to code but not fine if you don't and sometimes it's just wrong).

Ironically with how it work it is more like a bad unreliable and inconsistent employees which is sometimes good at spotting things others overlook. That just isn't what you want from an automated code review :/, but also is to useful to fully ignore :(.