Remix.run Logo
likium 6 days ago

Just curious if the processes got tuned/adjusted as a result? And what were the negative side effects?

ethbr1 6 days ago | parent [-]

Absolutely! That was one of the common productive outcomes: this policy / approach is screwed up, and we could do it better.

Negative side effects were about what you'd imagine. Some low performers unjustly shielded themselves. Safeguards were overbuilt as proof "something" was changed to prevent a failure repeat. Executive promotion criteria could get squirrelly. Etc.

But on the whole, I think the individual/team productivity boost and agility created by honesty was a huge net win.

heelix 6 days ago | parent | next [-]

The first time I'd seen a blameless post mortem, I thought it was a load of bs, as another organization had just caused the first significant production outage our app had ever had. Convenient... no blame. We went along with the process and it did not take very long to understand how this changed the culture. If someone horked a step on a manual deploy, the real question is why is this not automated. People stopped hiding mistakes - so the old snipe hunts where information to trouble shoot might 'disappear' faded and made it easier to debug and then figure out what could be done better. It helped the business understand that 'running in production' did not mean done.

Ryan, if you are out there reading this - ty.

ethbr1 6 days ago | parent [-]

100%. It was an eye opening experience. Felt somewhat akin to running an RCA on "Why do people hide mistakes?" Well, because it's in their self interest to do so!

MichaelZuo 6 days ago | parent | prev [-]

Wouldn’t a hybrid system make more sense?

To only assign blame to people/teams when they’ve guaranteed in writing that it would be so and so, avoiding the downsides.

And blaming the process when there were no such guarantees?

ethbr1 6 days ago | parent [-]

The issue with any hybrid system is you have to play the incentives out at scale.

E.g. if blame is assigned when there's a written guarantee, why would anyone ever make a written guarantee?

And not trying to be obtuse, but I've only ever seen blameless cultures work in absolute. Compromises let back in all the nasty mal-incentives you see driving unproductive CYA behaviors.

MichaelZuo 6 days ago | parent [-]

E.g. Some people with lower amounts of credibility will be eager to make guarantees in writing, if they want to sound more convincing than someone with higher credibility who disagrees.

Something like a new engineer disagreeing with a PM, a PM disagreeing with higher management, etc…

There’s many reasons why that would be a favourable choice.