Remix.run Logo
▲ phire an hour ago

I'm not sure we would have found this impressive back then, the output is pretty nonsensical. Back then, we had other (more manual) approaches that could produce somewhat equivalent results.

It's a bit of an illusion, it looks quite impressive to us because we know how good the output from a full-sized LLM can be. We are willing to give it a lot of leeway because of its small size as long as it produces output that looks kind of like chatgpt.

Well... computational linguistics academics should have been impressed at how well it understands language with nothing more than ingesting a huge amount of untagged data.