I mean, if you consider "building the whole gas station" to be the infrastructure, then sure, but building a gas station requires significantly more complex infrastructure than installing an EV charger or two.

You need to install several specialized sealed tanks underground, and the pumps that will get the gas & diesel (and sometimes kerosene) in them up to the cars.

You then need to be able to make secure, safe deliveries of highly flammable liquids to the site regularly, forever.

Once an EV charger has been installed, it'll need occasional maintenance, and if some yahoos vandalize it it might need repair or replacement, but otherwise it's pretty much good to go, potentially for decades.