> More CPUs pegged at 100% is always a good sign of smooth flows, but optimizing further requires data layout changes from that point.

True. This is true by experience. I want to add that, additionally this is not just true based on experience, rather it is fairly a straightforward logic.

If the cpu is already saturated at 100% doing work in the most "efficient" way possible for a given data structure (some sort of a shannon limit for the given data structure), then there is zero further improvement possible.

Any additional improvement must therefore happen either on the thing that the CPUs operate on (data structure improvements) or on the hardware themselves.