Remix.run Logo
flir 4 hours ago

Hah. I think of it as a slime mold. There's the main body (bodies?), but it's always shooting out little bits of itself that try weird stuff - founding underwater communes, or climbing mountains in Crocs or something. Most of these offshoots don't have that much of an impact, but occasionally one lucks out and discovers America or peanut butter and the main body saunters off that way.

godelski 3 hours ago | parent [-]

Yeah we find this type of optimization all over nature. Even radiation is important to create noise. We need it in machine learning. A noisy optimizer is critical for generalized learning. Too much noise and you learn nothing but no noise and you only memorize. So there's a balance