Deno is a V8 wrapper, the same JS engine as Chrome. Vulnerabilities are very common there, not necessarily because it's poorly designed but more because there's massive financial incentives in findings them.

This plus what you mentioned is why I would never trust it to run arbitrary code.

Now in the context of yt-dlp it might fine, google isn't going to target them with exploits. I would still prefer if they didn't continue to propagate "DeNo iS SaFe BeCauSe It HaS sAnDbOxInG" because I've seen projets that were actually executing arbitrary JS rely on it thinking it was safe.