I called this out to Thomas Pornin a few months ago (that the forward pass of a neural network and a block cipher rhymed in the sense of being an iterated complex linear function punctuated by a nonlinear function that keeps the whole system from collapsing into linearity) and he intimated that it was not the profound or useful observation that I thought it to be. I feel somewhat vindicated.