Not an astronomer, but I'm a non-programmer who spent two months building a fairly big web app with Claude, so the false-positive question is the one I'd ask too.

What saved me wasn't asking Claude whether it was right (it almost always says yes). It was checks that don't depend on its own report: run it on data where you already know the answer, and give a fresh session only the result and ask it to break it.

For a planet hunt I guess that means hiding a few fake planets in real data to see if it finds them, and running the same pipeline on shuffled data to see how many "planets" it finds in pure noise.