The performance was reduced without a settings change. That is still a regression even if huge pages mitigates the problem.

I'd be curious to know if there's still a regression with hugepages turned on in older kernels.

If you are benchmarking something and the only changed variable between benchmarks is the kernel, that is useful information. Even if your environment isn't correctly setup.