If I were in college, that would make sense, but I like to spend evenings with my family.

You just need to find the right family it sounds like ;) We both have the same sleeping schedule so works out for us, and we hang out more than ever. But YMMV.