I'm actually curious about AI progress:

There's no way the AI is a priori understanding codebases with millions of LoC now. We've tried that already, it failed. What it is doing now is setting up its own extremely powerful test harnesses and getting the information and testing it efficiently.

Sure, its semantic search is already strong, but the real lesson that we've learned from 2025 is that tooling is way more powerful.

That's cool! I've always wanted to learn how kernel devs properly test stuff reliably but it seemed hard. As someone who's dabbled in kernel dev for his job. Like real variable hardware, and not just manual testing shit.

Honestly, AI has only helped me become a better SWE because no one else has the time or patience to teach me.

▲

tptacek 3 hours ago | parent [-]

What do you mean "a priori understanding codebases"? Quantify it and let's test specifically what you mean. Linux is huge.

▲

par1970 3 hours ago | parent [-]

> What do you mean "a priori understanding codebases"?

I took him to be distinguishing between (1) just reading the code/docs and reasoning about it, and (2) that + crafting and running tests.

▲

tptacek 3 hours ago | parent [-]

I don't think that's it; both reading the code and running tests are a posteriori capabilities.

	▲	sigbottle an hour ago \| parent \| next [-]
		No you're right. I initially thought you were wrong but it is sus. My intuition for a priori cut something along the lines of, "Even if you had the entire source code in your head at once, there's limits to reasoning about it". Computability is one hard result. You also have to interact with the real world on a wide variety of hardware systems, or even just a wide variety of systems if you create an API - how do you reason past the abstraction boundary reliably without actually having tests and interacting with systems and getting feedback? Not really possible unless LLM's control everything. More philosophical questions (such as "is our 'correct' actually the right thing?") we grant the easy case that everybody's in consensus - the "easier" problems show up either way. But getting to the point of "understanding in principle every piece of linux" is pretty undefined and practically doesn't seem possible for a singular LLM or a human. This also seems really hairy for smuggling in whatever implicit premises you want to swing the issue either way. But personally I (and many other people) have seen late 2025 models get extremely good, and that precisely is because they actually started doing deep tooling and like, actually running and testing their code. I was not getting nearly as much value out of them (still a decent amount of value!) prior to the tooling explosion, not even MCPs were good. It was when they actually started aggressively spawning subshells and executing live tests. But I guess using a priori/posterioi isn't really a useful split here?
	▲	par1970 an hour ago \| parent \| prev [-]
		Yeah, maybe you are right. But is doing math and reasoning about Turing machines a priori? If so, then it seems plausible to me that reasoning about a codebase (without running it) is also ‘a priori’.