Remix.run Logo
▲ p_l 2 hours ago

The problem is packages that use the build backends, and Gentoo is generally not in the business of maintaining that kind of patch.

Admittedly rustification like that is a problem not only for gentoo, I have commercially deployed systems where one of the most annoyings elements of updating is presence of rust in python deps...

▲woodruffw an hour ago | parent | next [-]

I think that’s actually a different problem: packages that use uv-build for their backend should use a semver range, which enables the kind of “pick a version and use it” strategy I mentioned before.

(This is the default when you initialize a project with uv, so I think someone would need to go out of their way to configure their build backend to request a specific patch version of uv-build.)

▲pdonis an hour ago | parent [-]

> packages that use uv-build for their backend should use a semver range

This wouldn't necessarily solve Gentoo's problem. The only way to be sure to solve Gentoo's problem would be for every Python project that uses uv-build to specify one particular version. Then Gentoo could be sure that once that release of that project was built once, it would never change.

In other words, for a distro like Gentoo, the build backend is a dependency, and if it changes, that triggers a change in the build, even if the build backend change would not actually change the final wheel that got built.

▲woodruffw an hour ago | parent [-]

That would appear to be the polar opposite of effective: that turns a global constraint problem (everybody needing to have or install a compatible build backend version) into a global coordination problem (everybody needing to have or install an exact version). I can imagine most of what causes problems for Gentoo is people trying to do the “right” thing of pinning an exact build backend version, which then invalidates whatever compatible build backend version they’ve picked.

It seems much easier to solve the problem of compatibility than the one of getting every open source maintainer to upgrade their build backend version (whether uv-build, or flit, or anything else) every Tuesday.

▲pdonis an hour ago | parent | next [-]

> I can imagine most of what causes problems for Gentoo is people trying to do the “right” thing of pinning an exact build backend version, which then invalidates whatever compatible build backend version they’ve picked.

This doesn't make sense. Gentoo doesn't "pick" any build backend version. They build each Python project exactly the way the project owner tells them to, using whatever build backend is specified in pyproject.toml for that project. That's why it's such a burden for them (or more precisely their users, since each Gentoo user is having to do this) if build backend versions change so often and projects don't pin a specific version but just say "use the latest"--because then they have to use what the project owner says, and the project owner says, "Update to a new build backend every week because that's how often my chosen backend, uv-build, likes to push updates".

▲woodruffw 42 minutes ago | parent [-]

> if build backend versions change so often and projects don't pin a specific version but just say "use the latest"

This isn’t what’s happening: uv defaults to semver bounds, so a maintainer would have to go out of their way to introduce a troublesome backend constraint.

And note: there is no “use the latest” constraint in Python packaging. If you leave the build backend version unconstrained, the build frontend is allowed to resolve any version it sees fit, including the version it already has.

(I also know from first-hand experience that Gentoo and other distributions do not build Python packages as-is; they patch them in all kinds of ways. But that’s neither here nor there.)

▲pdonis 37 minutes ago | parent [-]

> uv defaults to semver bounds

But a new patch release will be within the same semver bound, so it amounts to the same thing.

> If you leave the build backend version unconstrained, the build frontend is allowed to resolve any version it sees fit, including the version it already has.

What do build frontends actually do, though?

▲woodruffw 28 minutes ago | parent [-]

> But a new patch release will be within the same semver bound, so it amounts to the same thing.

Sorry, can you lay out the actual problem you’re envisioning for me? If uv-build goes from 0.13.1 to 0.13.2, then anybody with a semver bound of `uv-build>=0.12,<0.14` doesn’t need to update anything. They can keep using whatever version of the build backend they already have that satisfies that bound.

The main thing that would be troublesome is if a user pins their build backend to a specific range or version, e.g. `uv-build==0.13.1`. Then, upon bumping it, everyone who has already built 0.13.1 will need to invalidate and rebuild their build backend. But that’s why that isn’t the default.

> What do build frontends actually do, though?

uv will definitely use a compatible version. It won’t even download a copy of that that compatible version by default, since the uv binary itself is typically compatible and can invoke the build backend directly.

I’m not sure what pip and others do, but I would expect they do the same. It’s the obvious way to implement backend resolution.

▲pdonis an hour ago | parent | prev [-]

> a global coordination problem (everybody needing to have or install an exact version).

This coordination problem is already solved by PyPI. That's where Gentoo looks to find the build backend--just like everyone else that builds packages. The project owner had to be able to find whatever version they used there or they couldn't even build their project locally. So they should be able to depend on the fact that anyone else trying to build their project can also find that same version at the same place.

> getting every open source maintainer to upgrade their build backend version (whether uv-build, or flit, or anything else) every Tuesday.

You're missing my point. If I know that my project builds properly with build backend X version Y, and I pin that version in my pyproject.toml, I don't care how many newer versions you want to release. Unless I make a change to my project that merits updating my build backend to a newer version, I'm going to simply leave my project alone. I already know it will build fine with the version I have. Why should I push an update every Tuesday just because you decided to release a new version of the build backend, which I don't need anyway?

▲jitl 23 minutes ago | parent [-]

gentoo does not appear to download uv from PyPi https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/uv...

▲arto an hour ago | parent | prev [-]

I hear you, but realistically the rustification of Python package internals is only going to strongly increase over the next year (I'm working on a number of such packages, myself), so systematic solutions to such annoyances are going to become increasingly necessary

▲p_l 8 minutes ago | parent [-]

Well, it did force me to work on better build caching and the like, but at the same time, it accelerated my wish to remove python where possible