I mean heterodox as seen by medieval monks, so deeply unchristian things, for example.

It's hard to imagine something more heterodox than Ovid (he managed to get himself exiled by Augustus), and that survived. Medieval readers didn't seem to mind that sort of thing in Greco Roman writing, it was part of their heritage, no one was seriously worshipping those gods so it wasn't seen as a threat. The people in the past behaving in a way that was seen as immoral wasn't a problem.