> The core problem is the lack of a lockfile. Every other package manager figured this out decades ago

Well... not Pip!

I would recommend you to try uv for this.