That's exactly why it's called a 'rectified' linear unit! It's a half-wave rectifier. The ReLU function is just what you'd see if you put an (ideal) diode on a curve tracer.

▲

CamperBob2 2 days ago | parent [-]

(Although to be more accurate, it would be what you'd see on an I-V curve tracer if you measured an ideal diode in series with a 1-ohm resistor. The diode by itself would just go vertical at the Y axis, and the ML people would mutter into their beards about exploding gradients.)

▲

hshdhdhehd 2 days ago | parent [-]

And that diode would be an exploding grenadiant?

	▲	CamperBob2 2 days ago \| parent [-]
		Yes, because the slope of current versus voltage goes infinite as soon as the voltage goes positive. The math would explode, and so would the diode, given enough current.