> I'm a bit confused by these claims about the GIL? The expensive IO operations release the GIL.

FYI: The extraction of files is largely python code that doesn't release the GIL. (cf. the zipfile class from the python interpreter has large layers of abstraction with a massive overhead in python code).

Regardless, python packages are thousands of tiny files, so pip never gets to release the GIL for any meaningful duration.

If you were writing an app that only extracted large GB files, you could take advantage of some I/O operations and some zlib operations freeing the GIL for a bit. Unfortunately pip is the opposite use case, lots of tiny files.

> Interesting, I would be very surprised if this made a significant difference? but I'll take a look.

Optimizing empty files was actually quite worthwhile for pip, because about 10% of python packages are empty init files.

This might not give the same result for uv though. pip is fully linear, every single open/read/write/stat operation we removed was a direct performance gain. uv does parallel async IO, you could very well remove 10% of filesystem calls and barely affect the overall duration. :D

> FYI: The extraction of files is largely python code that doesn't release the GIL. (cf. the zipfile class from the python interpreter has large layers of abstraction with a massive overhead in python code).

TIL. I ran some benchmarks and confirmed this is the case for many small files as you'd see in wheels — the GIL is released in a meaningful way for larger files though. Thanks!

> This might not give the same result for uv though.

Yeah, I built a prototype and ran some benchmarks. It makes a big difference if the entire wheel is empty files but for any real world examples it's within noise of the baseline.