From experience, it's not overstated. Running your own email server is pain, and even if you do everything right you may get delivery problems. And if you want to improve your chances, you have to do whatever big tech wants you to. And if you ever get onto the bad side (for example, your site is hacked and distributes malware for a few days) you may never recover.

It's not impossible, but it's not something you run once and forget.