> There must be some sweet spot there, since if it lasts too little

Absolutely, and therein lies the essence of game design, right?

For a simple game like this, I'd say 8-10 seconds is a good time to shoot for, for the player's first session. It'll naturally get longer as the player gets better. This makes every moment exciting--just a few seconds longer and I can beat my last score. But yes, this is an art not a science.