A large part of this is because the latency on modern TVs can be anywhere between 4.7ms and 150ms so games have to allow for a lot of slack in their input.

The NES and SNES had 1-3 frames of latency depending on the game.