Remix.run Logo
gopalv 8 hours ago

> That would be like asking a driver, "what's your favorite pothole you've ever driven over?"

The bugs that matter aren't potholes, they're trails with no markers, cut by someone you don't know who has left the organization years ago.

This is more like "What was your harder part of the Rubicon trail?"

I spent 7 months debugging a single issue which happened once a billion requests across 100k web servers or so.

For those 7 months, most machines did a hard restart every 4 hours and the bug never reproduces, so I didn't even know if I fixed it until I had enough confidence to turn that restart loop off.

Finding the issue had a fork of valgrind in the middle, found a CPU errata with AMD hardware when 2 locks placed in the same cache line (but not all failing machines had that CPU version), found a "bug" in UFS FreeBSD where deleting a file + recreating it uses the same inode id (not a bug, but not how linux worked)

The fix was 2 lines.

Your answer is not the wrong answer, but the bugs that you encounter in the wild is often like a tiger hunt in a jungle, alone.