That paper you linked has all of the information you need. The linear algebra they do on the weights there to null out one direction reduces to a single (dot-product + broadcast-multiply + add) on the activations.