The complexity of actual biological neural networks became clear to me when I learned about the different types of neurons.

https://en.wikipedia.org/wiki/Neural_oscillation

There are clock neurons, ADC neurons that transform analog intensity of signal into counts of digital spikes, there are neurons that integrate signals over time, that synchronizes together etc etc. Transformer models have none of this.