Accurate to my experience hacking on this model today, but I don't think anyone's blowing smoke about it.

Thinking back to where GPT-3 was 5 years ago, I can't help but be a little bit excited. And unlike GPT-3 this is Apache.

I'd put this closer to gpt2 tbh. GPT3 was already quite impressive and functional. We haven't come particularly far since imo. More small noticable steps, but no significant jumps.