" if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now."

? it can never be the case.

AI has a 'noisy channel problem' and it's due to the ambiguity inherent in language.

If we have a 'way to express ourselves perfectly' in a language-like concept, at a higher level of abstraction than code, we'd have our solution.

But we don't.

We could develop really, really strong ways of writing specifications that work really well, combined with other kinds of paramterized specs, that could work - but as of now, we're not quite there.

We are 'missing a layer' - and do we make do with what we used before.