It’s important to understand that it had pytorch to actually run subsets of the weights. So its more like a dedicated person running the model to find the token it shouldn’t generate, then following the weight chain back up while calculating the NN outputs to find a place where if it changes some weights (again with a lot of testing via pytorch) the result is better.

Its not natively reading the weights in the Matrix “symbols scrolling on a screen” sense. Still bloody impressive and not an outcome I would have bet on!