Remix.run Logo
colechristensen 8 hours ago

One thing I've found to be successful is to

1) give it a number of things to list in order of severity

and

2) tell it to grade how serious of a problem it may be

The human reviewer can then look at the top ten list and what the LLM thinks about its own list for a very low overhead of thinking (i.e. if the LLM thinks its own ideas are dumb a human probably doesn't need to look into them too hard)

It also helps to explicitly call out types of issue (naming, security, performance, correctness, etc)

The human doesn't owe the LLM any amount of time considering, it's just an idea generating tool. Looking through a top ten list formatted as a table can be scanned in 10 seconds in a first pass.