The voucher system is seen as an attack on IEP / special ed children, since public schools rely on distributing money away from the general population of students and into special ed and IEP students who, by various measures, can consume 3x the amount of money per student. They need lots of non-special-ed students to subsidize the special ed ones at the current levels.

If enough people use the voucher system it basically forces the per student spending to get closer to a purely egalitarian spending per student, with the result that public schools have to spend about the same amount on special ed kids as the voucher kids get (in the extreme, that's all the students they're left with). While this is objectively fairer in my opinion, it's viewed as an entitlement that the special ed kids can take more money at the expense of everyone else.

Obviously though this has to be carefully framed to sell it properly. Very few are going to knowingly sign up to lose funding for their own student to help some other student who is already getting 3x the money as them, so instead it's framed as some sort of evil capitalist agenda against public schools.