The chain rule was explored by Gottfried Wilhelm Leibniz and Isaac Newton in the 17th century. Either of them would have ”invented” backpropagation in an instant. It’s obvious.

▲

_fizz_buzz_ 3 days ago | parent | next [-]

Funny enough. For me it was the other way around. I always knew how to compute the chain rule. But really only understood what the chain rule means when I read up on what back propagation was.

▲

Lerc 3 days ago | parent | next [-]

That's essentially it. Learning what the chain rule does, and learning what it can be used for, and how to apply it.

Neither are really inventions, they are discoveries, if anything the chain rule leans slightly more to invention than backdrop.

I understand the need for attribution as a means to track the means and validity of discovery, but I intensely dislike it when people act like it is a deed of ownership of an idea.

▲

Jensson 3 days ago | parent | prev [-]

You don't think the people who invented the chain rule understood what it means?

▲

_fizz_buzz_ 3 days ago | parent [-]

Obviously, Newton and Leibniz and many other Mathematicians (and other people) understood the chain rule before back propagation. But unfortunately I am very far from a Newton or Leibniz, so it took me a lot longer to grasp why the chain rule is the way it is. And back propagation just made it click for me. I was really just talking about me personally.

▲

cutlilacs 2 days ago | parent [-]

What insight did you gain from back propagation that you didn't have from just the formula of the chain rule?

▲

_fizz_buzz_ 2 days ago | parent [-]

What clicked for me was drawing the chain rule as a graph. When I was in school I just applied the chain rule without thinking about it. I really didn't mean this to be some deep insight or anything. Just an anecdotal comment.

	▲	cutlilacs 6 hours ago \| parent [-]
		Ah makes sense, I was thinking there was some deeper insight I was missing. Thanks!

▲

Kranar 3 days ago | parent | prev [-]

It's not at all obvious, as the article points out it was assumed for 40 years that backpropagation was not an efficient approach for training neural networks.

	▲	nothrowaways 3 days ago \| parent [-]
		It still is not efficient approach.