It's very possible but somewhat slower. PyTorch and CUDA have flags for determinism. It won't work across all different GPU models though, but it will get you bitwise equal results on the same GPU.

▲

prohobo 3 hours ago | parent [-]

Both of your comments are illuminating :p

So, we could technically debug a prompt's output? I get that there are too many steps to actually step thru, but what if there were checkpoints? At least you could isolate behaviors to specific sections of a neural network?

	▲	bonoboTP 2 hours ago \| parent [-]
		Of course. And mechanistic interpretability research is a thing.