This article glosses over the hardest bit and bike sheds too much over keys.

> Critically, these two things must happen atomically, typically by wrapping them in a database transaction. Either the message gets processed and its idempotency key gets persisted. Or, the transaction gets rolled back and no changes are applied at all.

How do you do that when the processing isn’t persisted to the same database? IE. what if the side effect is outside the transaction?

You can’t atomically rollback the transaction and external side effects.

If you could use a distributed database transaction already, then you don’t need idempotent keys at all. The transaction itself is the guarantee

I'm not sure if TFA implies this (it uses too much of his personal jargon for me to understand everything, and it's Friday) but consider this solution based on his transaction log section: you should use the same database that persists the idempotency key to persist the message, and then consume the messages from the CDC/outbox-style. Meaning, the database simply acts as an intermediate machine that dedupes the flow of messages. Assuming you're allowed to make the producer wait.

The external side-effects also need to support idempotency keys, which you propagate. Then you use something like a message queue to drive the process to completion.

If you need "transactions" with microservices the traditional answer is sagas - eg multiple transaction boundaries that dont fully commit until their entire set of things is complete by passing a message or event to the next system, and having the ability to rollback each thing in a positive manner, either by appending new correct state again or "not ever adding" the original state.