Remix.run Logo
torginus 3 days ago

You can make GPT rewrite all existing textual info into chatbot format, so there's no loss there.

With newer techniques, such as chain of thought and self-checking, you can also generate a ton of high-quality training data, that won't degrade the output of the LLM. Though the degree to which you can do that is not clear to me.

Imo it makes sense to train an LLM as a chatbot from the start.