> Is there a way to only host the receive portion?

Sure. Set up your email server and have DNS of your domain point the MX record at it so you receive all email directly without any third party involvement. Then, set up outgoing email to forward to some third party which handles delivery for you. IIRC there are some free options even, if your outgoing volume is low.

That said, my email server handles incoming and outgoing email for a handful of domains for self and family, no problem.