As long as you don't statically link you can easily replace malloc (LD_PRELOAD). Many debug libraries do. Why is this so special in openssl? (I don't know if there is some special reason, though openssl is a weird one to begin with)

Several reasons: using OpenSSL's malloc may bypass hardened libc mallocs like those on OpenBSD.

If memory crosses the boundary between OpenSSL and your app, or some other library, freeing it with a different allocator than the one it was allocated with is undefined behavior.

OpenSSL's allocator doesn't free in ways which prevent memory sanitization tools like valgrind from finding memory bugs.

OpenSSL has a completely separate idea of a secure heap, with it's own additional malloc implementation, which can lead to state leakage or other issues if not used perfectly at the (non-existent) security boundary and is accidentally intermingled with calls to the (insecure?) malloc.

It's just a big can of security worms which may have had a use when VMS was supported, though that's questionable, and only serves to add additional layers of inscrutability and obfuscation to an already messy codebase today. There's no good reason for it.