So if I understand correctly 3 modules are involved:

- esp4 (kernel config "CONFIG_AF_RXRPC")

- esp6 (kernel config "CONFIG_INET_ESP")

- rxrpc (kernel config "CONFIG_INET6_ESP")

Is this correct?

You mixed up the names vs. config options but yes killing those 3 options should make you "safe". No warranty.