I agree with your argument but this particular case is for the legal workers, just like UK significant portion of the Swiss population don’t want legal workers from the rest of Europe. They attempted to limit the legal rights of the EU citizens that currently have.