Remix.run Logo
toomuchtodo 2 hours ago

Would you happen to know where the latency comes from between upload and scanning? Would more resources for more security scanner runners to consume the scanner queue faster solve this? Trying to understand if there are inherent process limitations or if a donation for this compute would solve this gap.

(software supply chain security is a component of my work)

simonw 43 minutes ago | parent | next [-]

I don't know that myself but Mike Fiedler is the person to reach out to, he runs security for PyPI and is very responsive. security@pypi.org

TheDong 2 hours ago | parent | prev [-]

He said, "pypi doesn't block upload on scanning"; that's part of where the latency comes from. The other part is simply the sheer mass of uploads, and that there's not money in doing it super quickly.

I agree that's a bad idea to do so since security scanning is inherently a cat and mouse game.

Let's hypothetically say pypi did block upload on passing a security scan. The attacker now simply creates their own pypi test package ahead of time, uploads sample malicious payloads with additional layers of obfuscation until one passes the scan, and then uses that payload in the real attack.

Pypi would also probably open source any security scanning code it adds as part of upload (as it should), so the attacker could even just do it locally.

toomuchtodo 2 hours ago | parent [-]

I suppose my argument is that pypi could offer the option to block downloads to package owners until a security scan is complete (if scanning will always take ~45-60 minutes), and if money is a problem, money can solve the scanning latency. Our org scans all packages ingested into artifact storage and requires dependency pinning, and would continue to do so, but more options (when cheap) are sometimes better imho. Also, not everyone has enterprise resources for managing this risk. I agree it is "cat and mouse" or "whack-a-mole", and always will be (ie building and maintaining systems of risk mitigation and reduction). We don't not do security scanning simply because adversaries are always improving, right? We collectively slow attackers down, when possible.

("slow is smooth, smooth is fast")