> Both LLMs and n-gram models satisfy the markov property, and you could in principle go through and compute explicit transition matrices (something on the size of vocab_size*context_size I think).

Isn’t it actually (vocab_size)^(context_size)?

Yes, you're right. I typed "**" (exponentiation) but HN ate the second star since I forgot to escape.