The 'flashlight not a blocker' distinction is the right call. Curious to know - how do you handling false positive rate in practice?

In our experience with LLM-based code analysis, the signal-to-noise ratio is the thing that determines whether teams actually use the tool or just forget about it after a week.

For us, this was a very fast 0-60 project meant to help people quickly identify if they were breached by the LiteLLM supply chain attack (with other detection support). That's part of the reason our tool runs recursive checks, so developers can go to their e.g. ~/source directory and quickly see if they were pwned.

We've had almost zero false-positives with the AI detection in our configuration -- granted we haven't had a whole lot of testing given the short timeframe we started in, so take this with a grain of salt

Disclaimer: I work on this code