I'm curious if you are a coder and have used an LLM to review your code. It is like something like shining a black light around a hotel room, and that seems to be the case even for highly regarded software.

It is really easy to have tunnel vision while coding. LLMs have a working memory with a capacity an order of magnitude greater than ours. I wouldn't trust an LLM to write the code, but at this point it is malpractice not to use one for review.

I have been, yes. For a field that has engineering in the name there sure has been a lot of critical mistakes.

You have to call a spade a spade — the profession accepts this sort of tradeoff in the name of speed and cost.

A well designed system would have never allowed those mistakes to occur. I feel like using an llm to catch these sorts of things is just because it wasn’t built right in the first place.

I think ai systems will be able to build systems of abstraction that are formally verified, and we won’t be needed(eventually).

Right now it’s being used as a bandaid.