A lot of people are reasonably well-known in certain circles because of some show, podcast, book, etc. that's become something of a hit often with some calculated controversy. And, as you say, collects something of a following.

There are also a ton of people who have never especially groomed the mass market though they're pretty well known in their industry.