A sitemap.xml file could get you most of the way there.