Interesting that you mention this, as I also switched to Fastmail recently and got more spam than before, but after marking it as spam for some time it now died down I think. This may also be a symptom of changing providers, where the previous provider knew the kind of spam I tended to get from past years, while Fastmail needed some time to get up to speed.

Fingers crossed that the experience will be the same for you.