Remix.run Logo
josephg 8 hours ago

Bugs in nontrivial software aren’t potholes. They’re more like sinkholes. Seemingly small inconveniences can open up to a much more interesting world underneath.

About 20 years ago when I was still in school I remember a bug in C++ that took me a whole week to find, where I was accidentally returning a pointer to a local variable. The embarrassing thing was that the compiler warned me about it, but I was so laser focussed on finding why my program was crashing that I decided to leave fixing the compiler warnings til later.

About 10 years ago I spent 3 months on and off trying to track down a memory leak in a nodejs server process. I can’t remember what the fix was - I’m sure it was something stupid. But along the way I learned to connect the chrome dev tools to nodejs and I figured out how to use heap snapshot diffing, and what roots are, and how V8 thinks about memory. I still use that knowledge to this day.

I could tell lots of stories like this. That time I left a debugging statement in some performance critical code and for 3 months it ran 20x slower than it should have. How a diffing function actually needed to be a 3 way diff to function correctly - else some weird corner cases would cause problems. And so on.

Do you seriously not have interesting stories? Are you just not curious when things go wrong, why they go wrong? This sounds like a great interview question to weed out people who don’t have any passion to become excellent engineers.

do_not_redeem 8 hours ago | parent | next [-]

I'm amazed when people can remember specific details like that from decades ago. If you asked me to describe the specific leak I was chasing when I learned to use a memory profiler, there's no way I could tell you. It's just something I know now, and I use that knowledge when I need it.

It seems like some people have this sequential narrative of their experience in their head, where events are strictly ordered—but for me it's more like a pool, or an unordered set. I learn things Just In Time, they go in the pool, and eventually they fade away if no longer relevant (goodbye, VB6). That doesn't mean I don't care about the craft, it just means I'm focused on the future rather than the past.

pdpi 6 hours ago | parent | next [-]

There's several bugs from some 8+ years ago that I remember fairly well, because they were truly interesting.

One I didn't work on, but was the sort of stuff legends are made of. A very specific revision of Blink that was never shipped as a Chrome release but got shipped as BlackberryOS's default browser had a bug around one of the bitshift operators (can't recall which), which would give a wrong result around once in a million operations. The desktop build of that particular revision would only trigger that bug semi-reliably, but the Blackberry build would trigger it deterministically. Bitshifts are not particularly common in most JS codebases, but this was a bitcoin wallet, and bitshifts are used by the thousands as part of e.g. key derivations. "Key derivation" is exactly where this bug triggered for a user, and he lost a whole bunch of BTC to that bug. We were almost 100% sure the guy was trying to defraud us with that bug report but something made us itchy and we had somebody keep digging until we found the bug. The good news is that, because the Blackberry version was deterministic, we managed to reproduce the bug to the point of recreating the badly-generated keys and successfully recovered the guy's BTC.

One I worked on myself, diagnosing it involved attaching both IntelliJ's debugger and GDB to a running Java application, so I could set breakpoints both in Java land and in the C++/JNI component of a JDBC driver. My colleague tracked it down to an NPE at that boundary, and was stuck on how to proceed from there.

A fun one from a couple of years involved a script that ran fine on Firefox but failed on Chrome. This was around the time when fat arrow notation was introduced for JS functions. Introducing that involved changes to the parser, of course, and a vendor's minified scripts triggered a bug in the Chrome's JS parser such that it was seeing a fat arrow where there wasn't one. We solved that one by re-minifying the vendor's minified script with a different minifier that wouldn't generate the offending pattern.

mathgeek 4 hours ago | parent | prev [-]

> I'm amazed when people can remember specific details like that from decades ago.

I suspect this comes down to biological or neurological differences. As someone who remembers lots of trivial facts but cares not for the details of bugs, it might also just be a matter of whether your passion is your work.

foldr 8 hours ago | parent | prev [-]

It’s more of a memory test than anything. I can easily believe that some people happen to remember bugs in more detail than others, for whatever reason. I’m not sure it makes sense to draw inferences from this about their ability as engineers.

josephg 6 hours ago | parent [-]

It sounds harsh to say this, but why not? All else being equal, I’d rather hire & work with engineers with a good memory for engineering.

Your brain remembers things that you think about a lot, or that had a strong emotional experience. That’s how memory works. People who think a lot about engineering become better engineers.

foldr 6 hours ago | parent [-]

You can test people's memory if you want to, but it doesn't seem like that was the original intention behind the question. I'd suggest giving people advance notice for questions like this so that they can look through their past work and pick out some interesting cases. That way you don't select for people who randomly happen to remember something interesting within 30 seconds, or (more likely) people who are good at making things up. Surely the real point of the question is to talk about something interesting that someone has worked on, not to administer an ad-hoc and unscientific test of their memory.

As for strong emotional experiences, it seems obvious to me that people will have a wide range of different emotional reactions to their work. None of these reactions is inherently better or more correct than any other. Here I think you are falling into the common interviewer trap of judging people negatively for not being exactly like you.