Karpathy is butchering the metaphor. There is no abstraction here. Backprop is an algorithm. Automatic differentiation is a technique. Neither promises to hide anything.

I agree that understanding them is useful, but they are not abstractions much less leaky abstractions.