GPT-2 was 'sufficiently capable' according to Amodei: https://explainx.ai/blog/dario-amodei-gpt2-openai-open-sourc...

There are... multiple blog posts online now about how to use freely available data and modest amounts of compute to train a custom GPT-2-sized model from scratch. It would be quite a policing effort to prevent.