| ▲ | aesthesia 4 hours ago | |
Typically the analysis works in the reverse direction: we have a neuron (or SAE feature, or whatever) and we look at examples of data that makes it activate, then inferring an interpretation. This is a lot harder than probing for a known concept, and one of the points of this article is that if you try harder to understand the activation pattern, you will very often find that there is something interpretable there, even if at first glance it looked random. | ||