There is a quiet poetry to GPT1 and GPT2 that's lost even in the text-davinci output. I often wonder what we lose through reinforcement.

They were aiming for a fundamentally different writing style: where davinci and after were aiming for task completion, i.e. you ask for a thing, and then it does it. The earlier models instead worked to make a continuation of the text they were given, so if you asked a question, they would respond with more questions, pondering, reflecting your text back at you. If you told it to do something, it would tell you to do something

You can run GPT1 and 2 on consumer hardware so nothing is preventing you from exploring that art :)