I haven’t had this issue. It might be due to the long-standing reputation of my email domain(s) or of the mail server’s IP address, I don’t know. But I know other self-hosters who don’t have that issue either.