Remix.run Logo
cztomsik a day ago

hm, residual is what I would not expect, can you elaborate why?

simsla a day ago | parent [-]

Avoids vanishing gradients in deeper networks.

Also, most blocks with a residual approximate the identity function when initialised, so tend to be well behaved.