It's not quite that bad. There are about 100 known stars within 20 light years; with the nearest only about 5 away.

https://www.stellarcatalog.com/stars.php?offset=100

Actually getting a spacecraft to them in a reasonable amount of time is still well beyond us. But if we could get a probe up to relativistic speads, that would put missions to nearby stars on a similar timescale to existing space missions.

Our ability to actually get

I agree and appreciate that, but I would venture a guess that most of the nearest ones to us likely don't have "earth-like" planets, even accounting for the bias in our viewing capabilities you described. The moons, however! Maybe that bumps the chance of having a habitable celestial body to 0.00001% per solar system? That's my current rough guess. It would be remarkable if "actually life is very easy to form in the universe, it turns out Earth was just first".