The only thing that comes to mind for me is simpler header, but not sure if it makes much of a difference anyway.

Yes, it makes a difference: about 8 milliseconds. Properly implemented IPv6 has a lower latency. (and is more efficient, though i believe the energy savings are negligible) See this map: https://stats.labs.apnic.net/v6perf