The millisecond added by Wi-Fi isn't changing the experience.

Just avoid or throttle downloads while playing.

Average latency difference isn't much, but wireless's inherent sensitivity to interference makes the worst-case latency difference a lot higher. I don't want to lose a game because somebody in the vicinity turned on a poorly shielded microwave oven. If you're going to all the effort of getting everybody together in person, why not do it right and set up a wired network?