the active layers concept seems to be experiencing convergent evolution to how synapses in a human function

huge parameter models with many small but efficient layers can work quickly on low resource hardware

similar to how neurons experience chemical spiking to activate small portions of the brain at once