Well yeah, but N+2 but the generation of the +2 loses the first part of N.

If you use a context window of 2, then yes, you might know that word C can follow words A and B, and D can follow words B and C, and therefore generate ABCD even if ABCD never existed.

But it could be that ABCD is incoherent.

For example, if A = whales, B = are, C = mammals, D = reptiles.

"Whales are mammals" is fine, "are mammals reptiles" is fine, but "Whales are mammals reptiles" is incoherent.

The longer you allow the chain to get, the more incoherent it becomes.

"Whales are mammals that are reptiles that are vegetables too".

Any 3-word fragment of that sentence is fine. But put it together, and it's an incoherent mess.

That are reptiles!