> IMO the much more likely explanation is that the brain uses contrastive learning somewhere. [...] Instead of using calculus and outer products to calculate derivatives, you feed real data and false data forward through the network and each neuron tracks and trains on the difference.

Kinrany 4 hours ago | parent | next [-]

Separately, do I understand the idea correctly:

- when learning on the real examples, just propagate forward

- when learning on the fake examples, first generate them by propagating white noise backward

	▲	inigyou 3 hours ago \| parent [-]
		That's a technique that was tried in machine learning and worked (although obviously not optimally or we'd all be using it by now) and then we hypothesised that animal brains use it because it would explain sleep.

▲

Kinrany 4 hours ago | parent | prev [-]

I'm curious if there is an explanation for it not being more common to sleep like dolphins do, small parts of the brain at a time.