That is the exact problem that got me to abandon my splendidly working, self-hosted email stack. I had so many instances of mail just not getting delivered, despite doing absolutely everything correctly, that I gave up.