There are several reasons. I believe, many would acknowledge that it absolutely has good sides. But there are worrysome sides. As I life at the moment in Switzerland I know a little bit about the discussion there:

The implementation of an E-ID could just not good. In Switzerland people voted against E-ID already once and I believe now everyone agrees nowadays that it would have been an bad bad implementation back then (too much reliance on external companies). The same was true e.g. for Covid Certificates. The different implementations around Europe had different qualities and e.g. Switzerland ended up with one of the better (or maybe best) ones, where the identity of people were protected.

Let's just take the example of voting. It is already hard to explain to people that voting works as intended. Look e.g. at the US were I've the impression people do not trust regular voting anymore, despite having people from other countries checking if voting works correctly in the US. But overall it is a system almost everyone is able to understand. But the moment you bring cryptography into the game it's over. 99% will never ever understand why they should trust this. And honestly I feel with them. There are a lot of software people here and we all know how awful our whole industry cares about security overall and how critical software components depend sometimes on a few people. At least the whole implementation should be open source, everything else should not be tolerable.

What I have the impression most people fear, is not the E-ID itself, it is how it will be misused. Suddenly websites will now request verification for dubious reasons. While it is not the case with a regular ID, it will be trivial to do so in the future. The same with mass surveillance, it was not practical before internet, now it is, so governments do it. I think here comes one of the main arguments against it people would bring up, there is no simple instrument for people how they can fight back in case they dont like to identify with their E-ID.

To some degree there is mistrust in government (in Switzerland less then in Europe I believe) for very valid reasons. But still e.g. in Switzerland they had records of many people years ago. After the whole topic came to the surface it was a debacle and new laws were created to explicitly forbid this. E.g. in Switzerland it is not allowed by law to just store some information because are from the left-wing or right-wing (just regular left-wing/right-wing, not extremist), just as one example of something simple. Despite of this government still started to do again. Several newspaper requested this information, which now has to given out, and found it, despite being against the law, the government is doing it again. This kind of thing you can find for other European countries as well, and for the US I assume I don't even have to start.

Then what about people without Internet? At the 38C3 in germany last year was a presentation about this topic (Don't remember the full name, just that is is somehwere on https://media.ccc.de/c/38c3): that we always think it is just the old people, but this is not true.

Sure you could argue, that people give away they privacy willingly anyway, but I'm not sure if this a good reason to argue against all the suspicious some people have.

Here an article from a Online newspaper in Switzerland, tough its German: https://www.republik.ch/2025/08/29/ein-klares-jein-zur-e-id

At least in Switzerland I believe, if they just slightly would change the law it would benefit everyone. E.g. that in case an internet page expects an E-ID, that first it needs to through (a probably costly) evaluation what data is really, really needed, with many privacy experts at the table, to always reduce it to the absolut minimum (the E-ID has this feature to be even better than an ID regarding this). Additionally that there must be e.g. always a possibility to somehow call and have a possibility to do it without E-ID.

You can implement any of this without digital ID. Are there any real privacy concerns in Estonia after their digital citizenship? These arguments could be made against passports/driving licenses. It just doesn't seem like real legitimate concerns. Governments already have multiple databases on who you are. If digital ID unlocks more possibilities I am all for it - there is unbelievable amounts of missed opportunities in Government. One example - using a pen tablet to sign your name with the job centre to 'prove' who you are: utterly ridiculous!