I don't write a lot of super low level stuff, so maybe things are different there, but at least in the normal user space level I've found it pretty rare that explicit mutexes ever beat the performance of an (in my opinion) easier design using queues and/or something like ZeroMQ.

Generally I've found that the penalty, even without contention, is pretty minimal, and it almost always wins under contention.

To be fair the code in question was written many years ago - before anyone I know had heard of zeromq (it existed but wasn't known). it would be possible to optimize the mutexs out I'm sure - but the big problem wasn't speed it was complexity of code that I now maintain. Since local sockets are easier and faster I'm safe deleting the more complex code that should have never been written.