I as a human being can of course not be replaced by a next token predictor.

But I as a chess player can easily be replaced by a chess engine and I as a programmer might soon be replaceable by a next token predictor.

The only reason programmers think they can't be replaced by a next token predictor is that programmers don't work that way. But chess players don't work like a chess engine either.