Back in the dinosaur days (around 2005) I was working on a PHP CMS used by a big registrar. Occasionally page loads would block for seconds. It appeared randomly (natch) and was relatively unreproducible.

I couldn’t find any good way to debug it and a friend suggested GDB. I had never thought of using such a low level debugger on a scripting language, but what choice did I have? Fired it up, found a blocked process and sure enough it was blocked on reads to /dev/random.

I leaned two things that day: the decision to make and keep /dev/random blocking was dumb and GDB (or lldb, or valgrind, etc.) is useful for debugging just about anything.