> The point was that it's just predicting based upon the previous token.

Well that's just wrong. None of the LLMs of interest predict based upon the previous token.