> 2. Whenever a service retries but has to give up, the error it sends to its callers should never be retried. There has to be some agreement that that HTTP code will never be retried. This prevents the multiplicative factor of retry on top of retry, which is why those storms can generate so much load.

Ooh, I like the idea of propagating "no retries" hints in the responses back upstream. Have you seen it implemented in the wild, or in public discussions about the practice?

I've only seen it in bigcorp cross-service typedefs, or in startup's code that re-implements the checks in every service.