So we have here a massive evidentiary brief, the point of which is to argue that it is easier than we thought to, like, "sequence the AI genome" (in my estimation). We revisit famously-messy neurons, examine various tool failures, check places where messy things still hide, and catalog how "polysemanticity" is represented variously in the network.

Once sequenced, the thinking goes, we'll be able to do things like: explain and anticipate failures; predict what happens when you modulate the pathway an input takes on its way to becoming output; reconstruct algorithms the models have learned.

I'm not 100% sure this will work out like the mechinterp people think. If the neural representations and pathways are "clean" (organized into a a meaningful-to-humans hierarchy), it seems at least very plausible that the behaviors that AI xrisk people are worried about will mostly boil down to straightforward synthesis (i.e., mostly-linear-interpolation) of mostly-well-cataloged knowledge. And if that is true, then it does not seem hopeful that there would be a way to prevent the output they are worried about, other than just removing a bunch of the knowledge. So maybe there is no alignment solution at all.

What I envision is the ability to find the circuits and neurons evaluating morality and ethics and being able to figure out how they work at the boundaries and ensure safe deferral to human judgement or stopping outside of in-distribution behavior, which further is vetted against actual human values.