Yep. This is also how you solve differential equations with analog computers. (You need to recast them as integral equations because real-world differentiators are not well-behaved, but it still works.)

https://i4cy.com/analog_computing/

How does this compare to the Picard-Lindelof theorem and the technique of Picard iteration?