> Are they? I mean I wouldn't say they are strictly deterministic, but with a temperature and topk of 0 and topp of 1 you can at least get them to be deterministic if I'm correct.

the mathematics might be

but not on a GPU, because floating point numbers are an approximation, and their operations are not commutative

if the GPUs internal scheduler reorders the operations you will get a different outcome

remember GPUs were designed to render quake, where drawing pixels slightly off is imperceptible