If anyone here wants to do this but don't want to implement all of this yourselves, this "field" is called Durable Execution. Frameworks such as Temporal, Restate and DBOS do a lot of the heavy lifting to get the idempotency, exactly once and recovery to a known state logic here.

Second this. It's only been a few months since I started deploying Temporal at work, and there's no way that I would try implementing all this in-house.