See https://bdash.net.nz/posts/sandboxing-on-macos/ for more details on how sandboxing works on macOS. It touches on how the SBPL Scheme source code is interpreted in userspace to build a bytcode representation of the policy, and the kernel MAC hooks that the Sandbox kernel extension uses for enforcing sandbox policies.