To be fair, any multivariable regulator or filter (estimator) that has a quadratic component (LQR/LQE) will naturally yield a solution similar to backpropagation when an iterative algorithm is used to optimize its cost or error function through a differentiable tangent space.

▲

bgnn 3 days ago | parent [-]

So yeah, this was what I was thinking for a while. What about a more nonlinear estimator? Intuitively seems similar to me.

▲

andyferris 3 days ago | parent [-]

I believe the reason it works in nonlinear cases is that the derivative is “naturally linear” (to calculate the derivative, you are considering ever smaller regions where the cost function is approximately linear - exactly “how nonlinear” the cost function is elsewhere doesn’t play a role).

	▲	bgnn 3 days ago \| parent [-]
		that makes a lot of sense actually. thank you.