Technically you're right. But, you would need to create a space that becomes a magnet so people would drive/Uber there. Personally, these places charmed me from being "small, quaint, and relaxed spot in the middle of a city" (with a couple of exceptions). And then, you only have a very small subset of people who might actually enjoy it, so an average potential client would have to drive/Uber significant amount to get there.

Another thing is, in most of the Canada (i'm more familiar with laws here, but assuming it's similar in the states), zoning laws actually would prevent such place to exist in the middle of housing compounds. I live in a city right now, but if there was such place, and it would take me about 15 minutes to drive there, I would absolutely never go other than for novelty factor. It's great, but it's not "i'll drive there by myself after work, have a drink and drive back" kinda great. If I could subway there for 20 minutes, I would probably do it once every other month. If it's 20 minute walking distance, then probably once a month.

So yeah, distance from your local neighbourhood would matter when it comes to niche businesses. But again, I might be extremely off my base, since I've never owned a cafe/bar.