But the government inherently has that data, as it comes from there. They're the ones issuing the IDs in the first place.

Theres no avoiding this, structurally. So the best thing you can do is not to introduce any additional points of failure.

All previous systems avoided this. The government issued me an ID in the past, yet had no record of when I used it, or for what.

I am so much more afraid of monopolies invading my privacy than roving hackers, or my corner store. Governments are the ultimate monopoly.