A fuse can sit 99.9999999% of its lifetime in a circuit doing nothing, should we get rid of fuses?

I've literally heard someone say that we should remove the protection circuits because it's two dollars of parts protecting under a dollar of amplifiers.

Yes, the amplifiers are cheap, but the rest of the board isn't and if the amplifiers get blown up that's 1000 dollars in support costs and RMAs, bad customer opinion and a line stoppage to boot.