It would lose knowledge about existing subjects unless it’s continually retrained on those too. It could help inform the next training dataset though.

Good thoughts here. Forgetting is important, but that's too advanced for modern LLMs.