As another commenter said, at lot of the fraternal organizations are religious. The Elks site says to be eligible for membership, you must 1) Be at least 21 years of age; 2) Believe in God; 3) Be a citizen of the United States who pledges allegiance to and salutes the American Flag; 4) Be of good character. The Knights of Columbus says membership is limited to practicing Catholic men.

That works for some people. I like the activity-based groups. Besides the sports groups, a community garden is also good.

That's strange, I know a bunch of guys in my hometown who hang out at the Elks who aren't religious. Most of them are probably confirmed or at least baptized though.