| ▲ | hyperpape 4 days ago |
| > But if you want an existence proof: Maven. The Java library ecosystem has been going strong for 20 years, and during that time not once have we needed a lockfile. And we are pulling hundreds of libraries just to log two lines of text, so it is actively used at scale. Maven, by default, does not check your transitive dependencies for version conflicts. To do that, you need a frustrating plugin that produces much worse error messages than NPM does: https://ourcraft.wordpress.com/2016/08/22/how-to-read-maven-.... How does Maven resolve dependencies when two libraries pull in different versions? It does something insane. https://maven.apache.org/guides/introduction/introduction-to.... Do not pretend, for even half a second, that dependency resolution is not hell in maven (though I do like that packages are namespaced by creators, npm shoulda stolen that). |
|
| ▲ | Karrot_Kream 4 days ago | parent | next [-] |
| When I used to lead a Maven project I'd take dependency-upgrade tickets that would just be me bumping up a package version then whack-a-moling overrides and editing callsites to make dependency resolution not pull up conflicting packages until it worked. Probably lost a few days a quarter that way. I even remember the playlists I used to listen to when I was doing that work (: Lockfiles are great. |
| |
| ▲ | RobRivera 4 days ago | parent | next [-] | | > I even remember the playlists I used to listen to when I was doing that work (: Im a big fan of anything Aphex Twin for these type of sessions. | |
| ▲ | yawaramin 4 days ago | parent | prev [-] | | How do lockfiles solve this problem? You would still have dependency-upgrade tickets and whack-a-mole, no? Or do you just never upgrade anything? | | |
| ▲ | hyperpape 4 days ago | parent | next [-] | | I think the difference is that since libraries do not specify version ranges, you must manually override their choices to find a compatible set of dependencies. The solution is version ranges, but this then necessitates lockfiles, to avoid the problem of uncontrolled upgrades. That said, there's an option that uses version ranges, and avoids nondeterminism without lockfiles: https://matklad.github.io/2024/12/24/minimal-version-selecti.... Note: maven technically allows version ranges, but they're rarely used. | |
| ▲ | chowells 4 days ago | parent | prev | next [-] | | The difference is that the data is centralized with a single source of truth, and you have tools for working with it automatically. It doesn't mean lockfiles are cheap to update, but it does mean it's a much more streamlined process when it's time. | | |
| ▲ | yawaramin 4 days ago | parent [-] | | The data is also centralized without lockfiles though...it's in the package spec file itself, where all version can be upgraded together. If you are saying the only difference is some tooling for automation, that's a temporary problem, not a fundamental one. | | |
| ▲ | chowells 4 days ago | parent | next [-] | | By "centralized" I mean a single file that contains all transitive dependency version information and nothing else. This is very different than having to recursively examine tiny portions of your dependencies' package specs recursively to find all transitive dependencies. | |
| ▲ | growse 4 days ago | parent | prev [-] | | Without a lock file your transitive dependencies are.... by definition not centralised? Or have I misunderstood? |
|
| |
| ▲ | Muromec 4 days ago | parent | prev [-] | | you press a button which triggers a pipeline which does npm update on all root dependencies and produces a new lockfile, then creates a PR you need to approve. creating a PR triggers running all the tests you bothered to write to also flag things that didn't go well automagically. | | |
| ▲ | yawaramin 4 days ago | parent [-] | | Is this a joke? What's the difference between this and pressing a button which triggers a pipeline which does npm update on all root dependencies and produces a new package.json file which you approve in a PR? Is the only difference that some convenient tooling already exists so you don't have to update each root dependency by hand? | | |
| ▲ | crote 4 days ago | parent | next [-] | | A package spec defines your desires for your dependencies, a lockfile defines a resolution of those desires. For example, as a developer I want Spring to stay on 6.3.x and not suddenly jump to 6.4 - as that is likely to break stuff. I do not care whether I get 6.3.1 or 6.3.6, as they are quite unlikely to cause issues. I do not care the slightest what version of libfoobar I get 5 dependencies down the line. However, I do not want packages to suddenly change versions between different CI runs. Breakage due to minor version bumps are unlikely, but they will happen. That kind of stuff is only going to cause noise when a rebuild of a PR causes it to break with zero lines changed, so you only want version upgrades to happen specifically when you ask for them. On top of that there's the risk of supply chain attacks, where pulling in the absolute latest version of every single package isn't exactly a great idea. The package spec allows me to define "spring at 6.3.x", the lockfile stores that we are currently using "spring 6.3.5, libfoobar 1.2.3". I ask it to look for upgrades, it resolves to "spring 6.3.6, libfoobar 1.4.0". There's also a "spring 6.4.0" available, but the spec says that we aren't interested so it gets ignored. All tests pass, it gets merged, and we'll stay at those versions until we explicitly ask for another upgrade. The whole flow exists for things which aren't root dependencies. The major versions of those are trivial to keep track of manually, and you'll only have a handful of them. It's all the minor versions and downstream dependencies which are the issue: tracking them all manually is a nightmare, and picking whatever happens to be the latest version at time-of-build is a massive no-no. | |
| ▲ | Muromec 4 days ago | parent | prev [-] | | The difference is transitive dependences with ranges, which results in less frequent updates of direct dependences. And tooling of course too |
|
|
|
|
|
| ▲ | adrianmsmith 4 days ago | parent | prev | next [-] |
| If you use two dependencies, and one requires Foo 1.2.3 and the other Foo 1.2.4 then 99% of the time including either version of Foo will work fine. (I was a Java developer and used Maven for about 10 years.) For those times where that's not the case, you can look at the dependency tree to see which is included and why. You can then add a <dependency> override in your pom.xml file specifying the one you want. It's not an "insane" algorithm. It gives you predictability. If you write something in your pom.xml that overrides whatever dependency your dependency requires, because you can update your pom.xml if you need to. And because pom.xml is hand-written there are very few merge conflicts (as much as you'd normally find in source code), vs. a lock file where huge chunks change each time you change a dependency, and when it comes to a merge conflict you just have to delete the lot and redo it and hope nothing important has been changed. |
| |
| ▲ | zdragnar 4 days ago | parent | next [-] | | > You can then add a <dependency> override in your pom.xml file specifying the one you want. Isn't that basically a crappy, hand-rolled equivalent to a lock file? | | |
| ▲ | smrtinsert 4 days ago | parent | next [-] | | A single override does not equate to an entire lockfile of dependencies. | | |
| ▲ | zdragnar 3 days ago | parent [-] | | And yet, that one manual override and an auto-generated lockfile require basically the same level of effort, and serve the same purpose. Edit: actually, depending on the package manager, the auto generated lockfile takes less work than the single override, as they don't have the same issue maven does to require an override in the first place. |
| |
| ▲ | Muromec 4 days ago | parent | prev [-] | | Of course it is |
| |
| ▲ | int_19h 4 days ago | parent | prev | next [-] | | What happens when one requires Foo 1.0 and the other requires Foo 2.0, and the two are incompatible on ABI level? | | |
| ▲ | tpm 3 days ago | parent | next [-] | | When you are building some app in an ecosystem not entirely managed by you, things like this are bound to happen, so there are always ways to solve this. You use Foo1 in Project1 (= one pom.xml) and create Project2 (=second pom.xml) where you use Foo2 (but package it in such a way that Foo2 is not exported from Project2), and depending on the usecase create a thin wrapper which you can then use from Project1, as an absolute worst case. | |
| ▲ | jameslars 4 days ago | parent | prev | next [-] | | Notably, a lockfile does not solve this problem either. | | |
| ▲ | omcnoe 3 days ago | parent [-] | | True, but the lockfile is imposed at build time. Swapping out the version of a transitive dependency might build totally fine, but also might result is broken behaviour at runtime if the behaviour of the dependency changed. |
| |
| ▲ | Muromec 4 days ago | parent | prev [-] | | Then you sit and cry of course |
| |
| ▲ | ffsm8 4 days ago | parent | prev | next [-] | | Depending on the dependency, you can also use shadow versions right? Essentially including both versions, and providing each dependency with it's own desired version. I believe it's done with the maven shade plug-in Never used it myself though, just read about it but never had an actual usecase | |
| ▲ | lostmsu 3 days ago | parent | prev [-] | | Yes, but if later Dep1 and Dep2 stop depending on Foo you will never know about it. |
|
|
| ▲ | xg15 4 days ago | parent | prev | next [-] |
| The irony is that it even has something like lockfiles as well: The <dependencyManagement> section: > Dependency management - this allows project authors to directly specify the versions of artifacts to be used when they are encountered in transitive dependencies or in dependencies where no version has been specified. It's just less convenient because you have to manage it yourself. |
|
| ▲ | arcbyte 4 days ago | parent | prev | next [-] |
| I have YEARS of zero problems with maven dependencias. And yet i cant leave up a node project for more than a month without immediately encountering transitive dependency breakage that take days to resolve. Maven is dependency heaven. |
| |
| ▲ | creesch 4 days ago | parent | next [-] | | You might not have issues, but they definitely happen. Especially once you bring in heavy frameworks like Spring, which for some reason ship with a ton of dependencies, some of which are surprisingly outdated. I had this happen with JUnit after a JDK upgrade. We needed to update to a newer major version of JUnit to match the new JDK, so we updated the test code accordingly. But then things broke. Methods were missing, imports couldn't be resolved, etc. Turned out something else in the dependency tree was still pulling in JUnit 4, and Maven's “nearest-wins” logic just silently went with the older version. No error, no warning. Just weird runtime/classpath issues. This something else turned out to be spring, for some odd reason it was an ancient version of Junit 4 as well. And yeah, you can eventually sort it out, maybe with mvn dependency:tree and a lot of manual overrides, but it is a mess. And Maven still doesn't give you anything like a lockfile to consistently reproduce builds over time. That's fine if your whole org pins versions very strictly, but it's naive to claim it "just works" in all cases. Certainly because versions often don't get pinned that strictly and it is easy to set up things in such a way that you think you have pinned the version while that isn't the case. Really fun stuff.. | | |
| ▲ | arcbyte 3 days ago | parent [-] | | After more than a decade of working on spring projects in fortune 500 companies, I've literally never had this problem with Maven dependencies. I agree its theoretically possible, but I disagree that its difficult to resolve or somehow "hell". It works extremely well. |
| |
| ▲ | nemothekid 4 days ago | parent | prev [-] | | That's a problem with node's culture, not with lockfiles. I've never experienced the level of bitrot Node suffers from in Rust, Ruby, Go, or PHP, which do have lockfiles. |
|
|
| ▲ | kemitchell 4 days ago | parent | prev | next [-] |
| npm got around to `@{author}/{package}` and `@{org}/{package}` beyond just global `{package}`, albeit midstream, rather than very early on. The jargon is "scoped packages". I've seen more adoption recently, also with scopes for particular projects, like https://www.npmjs.com/package/@babel/core |
| |
| ▲ | reactordev 4 days ago | parent [-] | | The issue is what happens when libX@latest is updated and uses libd@2.0 but your other dependency libA@latest uses libd@1.3.1? In maven, crazy things happen. Sometimes it’s fine but if you have any kind of security, the version mismatch has different signatures and blows up. Ask any spring developer what happens when they have more than 1 slf4j in their classpath. |
|
|
| ▲ | Perepiska 2 days ago | parent | prev | next [-] |
| The author of the article asked the same question in his personal blog in Telegram a year and a half ago [1], and received in response exactly the same example with maven. Probably, to build a personal brand and increase visibility, he needs to ask the same thing many times in different places. 1. https://t.me/nikitonsky_pub/597 |
|
| ▲ | smrtinsert 4 days ago | parent | prev | next [-] |
| The java ecosystem never went through the level of pain node ecosystem has. For a while it was simply insanity in node. I've worked heavily in both, and the developer experience in java was always way better. |
|
| ▲ | oftenwrong 4 days ago | parent | prev | next [-] |
| To add: if you would like to avoid depending on Maven's dependency mediation behaviour, then a useful tool is Maven Enforcer's dependencyConvergence rule. https://maven.apache.org/enforcer/enforcer-rules/index.html |
|
| ▲ | Bjartr 4 days ago | parent | prev | next [-] |
| Seems like there's room then in the Maven ecosystem that does what maven-enforcer-plugin does, but which just looks at a lockfile to make its decisions. |
|
| ▲ | potetm 4 days ago | parent | prev [-] |
| The point isn't, "There are zero problems with maven. It solves all problems perfectly." The point is, "You don't need lockfiles." And that much is true. (Miss you on twitter btw. Come back!) |
| |
| ▲ | hyperpape 4 days ago | parent | next [-] | | I think Maven's approach is functionally lock-files with worse ergonomics. You can only use the dependency from the libraries you use, but you're waiting for those libraries to update. As an escape hatch, you end up doing a lot of exclusions and overrides, basically creating a lockfile smeared over your pom. P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again. | | |
| ▲ | Alupis 4 days ago | parent | next [-] | | > P.S. Sadly, I think enough people have left Twitter that it's never going to be what it was again. Majority of those people came back after a while. The alternatives get near-zero engagement, so it's just shouting into the wind. For the ones that left over political reasons, receiving near-zero engagement takes all the fun out of posting... so they're back. | | |
| ▲ | stack_framer 4 days ago | parent [-] | | I'd be willing to bet that 95% of my "followers" on Twitter are bots. So I get near-zero engagement, or engagement that is worth zero. | | |
| ▲ | floydnoel 3 days ago | parent [-] | | it is worth it to go through and block all bot followers. your engagement will likely improve afterwards. |
|
| |
| ▲ | potetm 4 days ago | parent | prev [-] | | Of course it's functionally lock files. They do the same thing! There's a very strong argument that manually managing deps > auto updating, regardless of the ergonomics. P.S. You're, right, but also it's where the greatest remnant remains. :( | | |
| ▲ | shadowgovt 4 days ago | parent | next [-] | | I fear it says something unfortunate about our entire subculture if the greatest remnant remains at the Nazi bar. :( (To be generous: it might be that we didn't build our own bar the moment someone who is at least Nazi-tolerant started sniffing around for the opportunity to purchas the deed to the bar. The big criticism might be "we, as a subculture, aren't punk-rock enough.") | | |
| ▲ | LAC-Tech 3 days ago | parent [-] | | If Twitter is a Nazi site then why do I have to block Laura Loomer & Ben Shapiro manually? |
| |
| ▲ | KerrAvon 4 days ago | parent | prev [-] | | JFC, get off Twitter. It's a Nazi propaganda site and you are going to be affected by that even if you think you're somehow immune. |
|
| |
| ▲ | jeltz 4 days ago | parent | prev [-] | | You don't need package management by the same token. C is proof of that. Having worked professionally in C, Java, Rust, Ruby, Perl, PHP I strongly prefer lock files. They make it so much nicer to manage dependencies. | | |
| ▲ | potetm 4 days ago | parent [-] | | "There is another tool that does exactly the job of a lockfile, but better." vs "You can use make to ape the job of dependency managers" wat? | | |
| ▲ | jeltz 4 days ago | parent | next [-] | | I have worked with Maven and dependency management is a pain. Not much nicer than vendoting dependencies like you do for C. When I first was introduced to lock files that was amazing. It solved so many problems I had with vendored dependencies, CPAN and Maven. Just because thousands of programmers manage to suffer through your bad system every day does not make it good. | |
| ▲ | aidenn0 4 days ago | parent | prev [-] | | Now you're moving the goalposts; I think lockfiles that are checked-in to version control are superior to Maven's "Let's YOLO it if your transitive dependencies conflict." Version ranges are more expressive than single-versions, and when you add lockfiles you get deterministic builds. | | |
| ▲ | deepsun 4 days ago | parent | next [-] | | I don't understand how Maven's YOLO is different from NPM's range. If you force a transitive dependency in Maven, then yes, some other library may get incompatible with it. But in NPM when people declare dependency as, say, ~1.2.3 the also don't know if they will be compatible with a future 1.2.4 version. They just _assume_ the next patch release won't break anything. Yes npm will try to find a version that satisfies all declarations, but library devs couldn't know the new version would be compatible because it wasn't published at that time. And my point is that it's _exactly_ the same probability that the next patch version is incompatible in both Maven and NPM. That's why NPM users are not afraid to depend on ~x.x or even ^x.x, they basically YOLOing. | | |
| ▲ | eptcyka 4 days ago | parent | next [-] | | Yeah, npm people expect that semantic versioning will be abided by. Obviously, it will not work if a minor version bump introduces a breaking change. Obviously this is better than pinning the same one dependency in literally every package - imagine the churn and the amount of life lost to bumping dependencies in any given ecosystem if every package had to pin a specific version of a dependency. Ultimately, these are imperfect solutions to practical problems, and I know that I much prefer the semantic versioning and lockfile approach to whatever the java people are into. | |
| ▲ | beart 4 days ago | parent | prev | next [-] | | From my experience, this is a self-policing issue in the npm ecosystem. When major packages break semver, the maintainers take a ton of heat. When minor packages do it, they quickly fall out of the ecosystem entirely. It's not "YOLO"ing, but rather following the ecosystem conventions. But anyway.. isn't that exactly the purpose of lock files? If you don't trust the semver range, it shouldn't matter because every `npm ci` results in the same package versions. | |
| ▲ | aidenn0 4 days ago | parent | prev [-] | | > I don't understand how Maven's YOLO is different from NPM's range. The person who wrote the range selected a range that they deem likely to work. I don't use NPM, but in Python it definitely happens that you see e.g.: foo >= 0.3.4, <= 0.5.6
Which can save a lot of headaches early on for packages that use ZeroVer[1]1: https://0ver.org/ | | |
| |
| ▲ | cogman10 4 days ago | parent | prev [-] | | Maven builds are deterministic (so long as you don't have SNAPSHOT dependencies). The version resolution is insane but deterministic. You'll only break that determinism if you change the dependencies. That's precisely because maven doesn't support version ranges. Maven artifacts are also immutable. Maven also supports manual override when the insane resolution strategy fails that's the "dependencymanagement" section. | | |
|
|
|
|
