I think Go-Explore (https://arxiv.org/abs/1901.10995) is promising. It'll provide automatic scaffolding and prevent catastrophic forgetting.

If one can frame the problem into a competition, then self-play has been shown to work repeatedly.