I think part of this comes from the fact that LLMs are surprisingly good at logic but roughly about as good as expected on information accuracy.

LLMs are not convincing to me in the domain I did grad school...but neither is Wikipedia, or Reddit, or random pop sci books. And LLMs are basically just summarizing those things.

But when made to work through difficult arbitrary logic (like coding), they are very impressive.

I think this also explains why people gripe a lot about LLM coding _style_, but concede that LLMs do totally fine on coding _correctness_ in 2026

This sounds plausible. And it's also very fixable!

These days people don't interact with raw LLMs: they interact with systems and harnesses that deal with chain-of-though and tool calls etc.

I don't think we can honestly expect an LLM's weights to encode a large amount of information accurately. But we can expect the whole system that you interact with that includes the LLM to be able to cite its sources and go digging etc.

So the LLM-system can become as accurate as our best sources.

Of course, figuring out how to get the maximum of information from the sources available is a big deal. See eg how many economists or epidemiologists can build entire careers out of noticing 'natural experiments', ie figuring how to use data that 'nature' created and that might already be collected to answer interesting questions about causal relationships.

> I think this also explains why people gripe a lot about LLM coding _style_, but concede that LLMs do totally fine on coding _correctness_ in 2026

I actually have gripes about correctness, too. But I suspect here the answer is also: more proving, more automated test generation (like fuzzing and property based testing etc), more formal methods.

As a really simple and somewhat silly example: I have much better results getting AI agents to write good Rust code, than I have with Python. A good part of that is that for Rust I can ask the agent to make both the compiler and clippy::pedantic happy. That gives a lot of good feedback, that I didn't have to engineer myself.

Makes me wonder if training weighted social media text close to older and higher grade webpages (colleges, research labs, national statistics)