From my experience a lot of the hardest problems in this space are either 1. edge cases or 2. integration-related and that makes them hard to validate across systems or draw boundaries around what's in the dummy mode. This type of parallel, live, full system integration test is hard to pull off.

In 1997 I was working on an integration between AOL and Circuit City (ha I outlived them both) to enable free AOL accounts for people buying PCs or some such; about a week before launch I changed the data returned from encoding spaces as "+" to "%20" and broke their integration (perl script). Very upsetting for them, and I felt bad.

I also had some weird bug when we started registrations from German accounts and I didn't handle umlauts (or UTF-16 with nuls in the string) in passwords properly.