You are correct, but my point was that it seems to me, that even though engagement algorithms aren't really a thing in the Fediverse, such filter bubbles still appear via self-sorting and de-federation. So yes, perhaps algorithms are to blame for filter bubbles in centralized social media, but removing algorithms doesn't remove filter bubbles, because people create it by themselves anyway.

TL;DR: Blaming the algorithm (correct or not) doesn't actually matter in the end, because filter bubbles happen with or without them.