It's a downside for some nebulous strawman that just barely exists outside of the support system UBI would provide. Far enough out to not experience the benefits but not far enough to be above it all.

Of course the strawman's proponents will use it as an example of why we should write the whole idea off entirely, rather than test the validity of the concern and discuss options that provide some form of equity to a wider population.