Just the idea of generating text by removing noise is so profound. Maybe each step is a level of hierarchy. Linguists must be astonished at the things happening these past years. I have to read more about it

I feel the same. On reflection, it's how I think I experience thoughts emerging in my head. Language gets derived from initially noisy embeddings. It's quite beautiful that we've ended up closer to a de-noising architecture than auto-complete on steroids.

You might find this article interesting:

https://www.quantamagazine.org/when-chatgpt-broke-an-entire-...

How transformers pretty much blindsided the whole field of NLP.