If you don't focus stack and try to train on partially unfocused images, the optimizer will try to match the rendered view to be also partially unfocused.

You would have to mask out the blurry areas for each image. I guess one could just implement a feature where the optimizer only optimizes gaussians within the sharp distances relative to the camera.

Other way of looking at the question is if you could make focus stacking better by using the full multi-view dataset? Afaik focus stacking essentially does depth estimation so it seems like multiple views would help with that.

Another way would be some kind of 4d GS where one dimension is the focus distance. But I'd guess the renders would inherently have shallow depth then, which is less useful usually.