Now I wonder how much performance is being left on the table elsewhere in the OpenSSL codebase...

Given the massive regression with 3.x alone, you'll probably be happier if you don't know :/

haproxy has an article on the subject

https://www.haproxy.com/blog/state-of-ssl-stacks

TLDR - on the TLS parts, quite a lot, up to 2x slower on certain paths. Amusingly, openssl 1.1 was much faster.

libcrypto tends to be quite solid though, though over the years, other libraries have collected weird SIMD optimizations that enable them to beat openssl by healthy margins.