You could think of supervised learning as learning against a known ground truth, which pretraining certainly is.