I wouldn’t personally refer anyone I’ve not worked with. What’s the point? Someone being smart isn’t enough, they have to be actually sane and normal to work with. You can be friends with people you wouldn’t want to work with. Someone I’ve not met would be a totally unknown quantity.

I would refer someone I'm friends with if I know them well, understand what they would be like to work with, and understand the job/company well enough.

After all, I've lived with some people that I never worked with. Surely I know them well enough to recommend them for jobs, even though I have not ever worked alongside them.

I wouldn't trust a referral from someone I don't work with either. Your reputation is on the line, if you don't have one to lose why should anyone care.