They could ask and/or parse the tests for the information rather than run them to output it. I’m honestly still not seeing a killer feature here that makes the security implications worth it!

yeah me as well. at least have the untrusted code allow certain plugins or certain features of plugins to run that you whitelist. not having vim keybindings or syntax highlighting is too barebones.