A professional scientist I know (tenured, professor) recruited me to set up a backtesting framework for a predictive finance model. When the results were not as they expected (this person does not work in finance and never has), they asked to see the code, then told me that claude had found a problem with the way some of the calculations were done (there was actually no problem), supplied the claude comments, and told me to change the code to match what they thought was correct. I did it anyway. Had they had more expertise in the domain (finance), they likely would have been able to leverage claude as a tool rather than inadvertently pursuing a very stupid mistake. Domain experts tend to doubt their ability to excel in other domains which is amplified by LLMs.

This sounds rather similar to the form of scientific fraud where you first create a conclusion, then invent/manipulate the data until it supports your conclusion.

I work with a bunch of PHD's and have been since before ai coding.

Their code is aways terrible, and they constantly think it's good.

The exercise is always the same: explain the math to me, like I'm 5, then we profile it and see what is faster.

Oddly Claude Code, integrated into their IDE's has made this situation happen much less.

I never want to work in a place again where the fun way to start the Monday meeting is a "math problem".

PS: Don't even get me started on their SQL.