I mean it's still C++ that's compiled and executed, surely the compiler would be able to provide a way to hook into that?

I don't recall the source, but I don't believe most (any?) c++ compilers implement compile-time code evaluation by compiling and running code.

For one thing they are required to disallow all undefined behavior for compile time execution, and some forms of UB only occur when the code is run.