Fun data point though, I just ran three data points of the Phoronix nginx benchmark and got these results:

- Pinned to 6 cores: 28k QPS

- Pinned to 12 cores: 56k QPS

- All 24 cores: 62k QPS

I'm not sure how this applies to realistic workloads where you're using all of the cores but not maxing them out, but it looks like hyperthreading only adds ~10% performance in this case.

Here's results of the Nginx benchmark pinned to 1-24 cores: https://docs.google.com/spreadsheets/d/1d_OK_ckLT1zTA_fG4vkq...

At 51% reported CPU utilization, it's doing about 80% of the maximum requests per second, and it can't get above 80% utilization.

I also added a section: https://www.brendanlong.com/cpu-utilization-is-a-lie.html#bo...

I'd imagine in this case it's just uncounted usage from OS networking stack

You missed to try 18 cores.