If you are willing to give up incremental compilation, concatenating all C++ files into a single file and compiling that on a single core will often outperform a multi-core compilation. The reason is that the compiler spends most of its time parsing headers and when you concentrate everything into a single file (use the C preprocessor for this), it only needs to parse headers once.

Merely parsing C++ code requires a higher time complexity than parsing C code (linear time parsers cannot be used for C++), which is likely where part of the long compile times originate. I believe the parsing complexity is related to templates (and the headers are full of them), but there might be other parts that also contribute to it. Having to deal with far more abstractions is likely another part.

That said, I have been incrementally rewriting a C++ code base at a health care startup into a subset of C with the goal of replacing the C++ compiler with a C compiler. The closer the codebase comes to being C, the faster it builds.