Remix.run Logo
GolDDranks 2 days ago

I wondered the same. Seems like it would just make a V-shaped loss around the zero, but abs has that property already!

fancyfredbot 2 days ago | parent [-]

RELU would have made it flat below zero ( _/ not \/). Adding the abs first just makes RELU do nothing.