LLM-isms are much less prevalent in base models, which is what GPT-2 was. It had significant problems with maintaining coherence, but GPT-2 generated text did not have the obvious tells of today's LLMs.