In all their cities they could see buildings that they did not know how to build. And before that, public services would have broken down. It would have become impossible to find people who knew how to repair your heated floor (if you were rich), etc. The city of Rome declined from 1 million people to something like 20,000. In the late 500s, Pope Gregory the Great thought that the world was ending because of all the trouble (including vicious barbarian invasions). Monks (and presumably anyone educated) had access to a lot of ancient texts, it was only some that got lost in the West. I think they would have had a distinct sense that that past was more advanced.