> never rely on anything that wasn't at least seriously looked over by an LLM

I can imagine LLMs becoming a mainstay, but what you are describing isn't wholly different from sufficiently advanced static code analysis - where you'd want more determinism than most LLMs normally provide.

The problem is that such a thing might take a decade and billions of dollars of investments to create per-language (e.g. actually useful code analysis for Java, for Spring Boot, for processing and validating form data, and DB schemas and document processing and rendering reports etc., literal domain checks for anything and everything that is common across various enterprises) so nobody wants to do that, so it's easier to throw LLMs at it and call it good enough.

▲

moffkalast a day ago | parent [-]

I remember back in the pre-2023 days where SonarQube was a big deal for Java static analysis, and I let it rip across an entire 120k line project at one point upon which it found something like seven issues, out of which only one or two were actual bugs. It was almost entirely useless. I think even Qwen would've done leagues better today.

Most bugs are far too nuanced to be caught by static analysis imo, you do need to actually understand what's going on in the program, the intent, the environment, etc. instead of blindly verifying if everything technically checks out, compilers already do a perfect job at that.

	▲	KronisLV a day ago \| parent [-]
		> everything technically checks out So who's responsible for all of the Spring Dependency Injection bullshit with circular dependencies and AOP issues, stuff like @Transactional only working when called from a different bean, as well as the other hundreds of issues I've seen throughout the years? One can't just ignore that, because in many places that is most of the job market (alongside maybe .NET or PHP). There's got to be some traditional way to spot every single one of the states that can be represented in code by the frameworks available in a given language, surely the correct answer is not "Yeah, an LLM said it looks okay because it's close enough to some training data that we have." It might be the practical answer, but only because all of our tech is built wrong. Then again, writing provably correct code might be impossible in Java, at least with the currently available tools, because the ecosystem is such that the compiler can't do anything about all of the dynamic stuff that evil developers make you deal with at runtime.