| ▲ | GitLab discovers widespread NPM supply chain attack(about.gitlab.com) |
| 172 points by OuterVale 18 hours ago | 73 comments |
| |
|
| ▲ | arkh 13 minutes ago | parent | next [-] |
| Most of those attacks do the same kind of things. So I'm surprised to never see something akin to "our AI systems flagged a possible attack" in those posts. Or the fact Github from AI pusher fame Microsoft does not already use their AI to find this kind of attacks before they become a problem. Where is this miracle AI for cybersecurity when you need it? |
|
| ▲ | wonderfuly 22 minutes ago | parent | prev | next [-] |
| I'm a victim of this. In addition to concerns about npm, I'm now hesitant to use the GitHub CLI, which stores a highly privileged OAuth token in plain text in the HOME directory. After the attacker accesses it, they can do almost anything on behalf of me, for example, they turned many of my private repos to public. |
| |
| ▲ | febusravenga a minute ago | parent [-] | | this, this, this All our tokens should be in is protected keychain and there are no proper cross-platform solutions for this. All gclouds, was aww sdks, gh and other tools just store them in dotfile. And worst thing, afaik there is no way do do it correctly in MacOS for example. I'd like to be corrected though. |
|
|
| ▲ | mrklol 19 minutes ago | parent | prev | next [-] |
| Is there any reason to keep using postinstall scripts allowed instead of asking e.g. the user? Are they even needed in most cases? |
|
| ▲ | dawnerd an hour ago | parent | prev | next [-] |
| Everyone is blaming npm but GitHub should be put on blast too for allowing the repos to be created and not quickly flagged. GitHub has a massive malware problem as it is and it doesn’t get enough attention. |
| |
| ▲ | princevegeta89 14 minutes ago | parent | next [-] | | I love! how Github, as a corporate company now owned by Microsoft, is directly tied to GoLang as the main repository of the vast majority of packages/dependencies. Imagine the number of things that can go wrong when they try to regulate or introduce restrictions for build workflows for the purpose of making some extra money... lol The original Java platform is a good example to think about. | |
| ▲ | benatkin 42 minutes ago | parent | prev [-] | | They're part of the same company, but that's a good point. They both have mediocre security. |
|
|
| ▲ | AmbroseBierce 23 minutes ago | parent | prev | next [-] |
| Microsoft should just bite the bullet and make a huge JS standard library and then send GitHub notifications to all the project maintainers who are using anything that could be replaced by something from there suggesting them to do such replacement. This would likely significantly reduce the number of supply chain attacks on the npm ecosystem. |
|
| ▲ | Aeolun 3 hours ago | parent | prev | next [-] |
| I thought this was a really insightful post, until they used it to try and sell me on Gitlab’s security features. |
| |
| ▲ | norman784 an hour ago | parent | next [-] | | You are not the target then, but people using Gitlab might find insightful. | |
| ▲ | jaggirs 3 hours ago | parent | prev [-] | | Why would that make it any less insightfull? | | |
| ▲ | hu3 2 hours ago | parent [-] | | Because bias and incentives matter. There's a reason disclosures are obligatory in academic papers. | | |
| ▲ | baq an hour ago | parent | next [-] | | It’s published on gitlab.com, not arxiv | | |
| ▲ | rockskon 39 minutes ago | parent [-] | | It's almost like the speakers are motivated by advertising a product to solve a problem in their own garden. |
| |
| ▲ | serial_dev 44 minutes ago | parent | prev [-] | | They pulled a little sneaky on ya, mentioning GitLab security features available to GitLab users in a GitLab Security blog post with GitLab logos everywhere. Call me a conspiracy theorist, but I start to think these people might be affiliated with GitLab. |
|
|
|
|
| ▲ | wiradikusuma 4 hours ago | parent | prev | next [-] |
| Does anyone know why NPM seems to be the only attractive target? Python and Java are very popular, but I haven't heard anything in those ecosystems for a while. Is it because something inherently "weak" about NPM, or simply because, like Windows or JavaScript, everyone uses it? |
| |
| ▲ | broeng 23 minutes ago | parent | next [-] | | Compared to the Java ecosystem, I think there's a couple of issues in the NPM ecosystem that makes the situation a lot worse: 1) The availability of the package post-install hook that can run any command after simply resolving and downloading a package[1]. That, combined with: 2) The culture with using version ranges for dependency resolution[2] means that any compromised package can just spread with ridiculous speed (and then use the post-install hook to compromise other packages). You also have version ranges in the Java ecosystem, but it's not the norm to use in my experience, you get new dependencies when you actively bump the dependencies you are directly using because everything depends on specific versions. I'm no NPM expert, but that's the worst offenders from a technical perspective, in my opinion. [1]: I'm sure it can be disabled, and it might even be now by default - I don't know.
[2]: Yes, I know you can use a lock file, but it's definitely not the norm to actively consider each upgraded version when refreshing the lockfile. | |
| ▲ | kace91 3 hours ago | parent | prev | next [-] | | One factor is that node's philosophy is to have a very limited standard library and rely on community software for a ton of stuff. That means that not only the average project has a ton of dependencies, but also any given dependency will in turn have a ton of dependencies as well. there’s multiplicative effects in play. | | |
| ▲ | louiskottmann an hour ago | parent | next [-] | | This is my take as well.
I've never come accross a JS project where the built-in datastructures were exclusively used. One package for lists, one for sorting, and down the rabbit hole you go. | |
| ▲ | rhubarbtree 2 hours ago | parent | prev [-] | | This is the main reason. Pythons ecosystem also has silly trends and package churn, and plenty of untrained developers. It’s the lack of a proper standard library. As bad a language as it may be, Java shows how to get this right. | | |
| ▲ | palata 2 hours ago | parent [-] | | > As bad a language as it may be, Java shows how to get this right. To be fair Java has improved a lot over the last few years. I really have the feeling that Java is getting better, while C++ is getting worse. |
|
| |
| ▲ | parliament32 4 hours ago | parent | prev | next [-] | | Larger attack surface (JS has been the #1 language on GitHub for years now) and more amateur developers (who are more likely to blindly install dependencies, not harden against dev attack vectors, etc). | | |
| ▲ | Sophira 3 hours ago | parent | next [-] | | Unfortunately, blindly installing dependencies at compile-time is something that many projects will do by default nowadays. It's not just "more amateur developers" who are at risk here. I've even seen "setup scripts" for projects that will use root (with your permission) to install software. Such scripts are less common now with containers, but unfortunately containers aren't everything. | |
| ▲ | dboreham 4 hours ago | parent | prev [-] | | Also: a culture of constant churn in libraries which in combination with the potential for security bugs to be fixed in any new release leads to a common practice of ingesting a continual stream of mystery meat. That makes filtering out malware very hard. Too much noise to see the signal. None of the above cultural factors is present in the other ecosystems. |
| |
| ▲ | Balinares an hour ago | parent | prev | next [-] | | As far as I understand, NPM packages are not self-contained like e.g. Python wheels and can (and often need to) run scripts on install. So just installing a package can get you compromised. If the compromised box contains credentials to update your own packages in NPM, then it's an easy vector for a worm to propagate. | |
| ▲ | dtech 3 hours ago | parent | prev | next [-] | | Npm has weak security boundaries. Basically any dependency can (used to?) run any script with the develop permissions on install. JVM and python package managers don't do this. Of course in all ecosystems once you actually run the code it can do whatever with the permissions of the executes program, but this is another hurdle. | | |
| ▲ | lights0123 3 hours ago | parent [-] | | Python absolutely can run scripts in installation. Before pyproject.toml, arbitrary scripts were the only way to install a package. It's the reason PyPi.org doesn't show a dependency graph, as dependencies are declared in the Turing-complete setup.py. | | |
| ▲ | oefrha 3 hours ago | parent [-] | | Wrong. Wheels were available long before pyproject.toml, and you could instruct pip to only install from wheels. setup.py was needed to build the wheels, but the build step wasn’t a necessary part of installation and could be disabled. In that sense its role is similar to that of pre-publish build step of npm packages, unless wheels aren’t available. |
|
| |
| ▲ | Ekaros 2 hours ago | parent | prev [-] | | I feel with Python upgrade cycle is slower. I upgrade dependencies when something is broken or there is known issue. That means any active vulnerabilities propagate slower. Slower propagation means lower risk. And also as there is fewer upstream packages impact of compromised maintainer is more limited. |
|
|
| ▲ | thepasswordapp 4 hours ago | parent | prev | next [-] |
| The credential harvesting aspect is what concerns me most for the average developer. If you've ever run `npm install` on an affected package, your environment variables, .npmrc tokens, and potentially other cached credentials may have been exfiltrated. The action item for anyone potentially affected: rotate your npm tokens, GitHub PATs, and any API keys that were in environment variables. And if you're like most developers and reused any of those passwords elsewhere... rotate those too. This is why periodic credential rotation matters - not just after a breach notification, but proactively. It reduces the window where any stolen credential is useful. |
| |
| ▲ | Ferret7446 14 minutes ago | parent | next [-] | | > if you're like most developers and reused any of those passwords elsewhere Is this true? God I hope not, if developers don't even follow basic security practices then all hope is lost. I'd assume this is stating the obvious, but storing credentials in environment variables or files is a big no-no. Use a security key or at the very least an encrypted file, and never reuse any credential for anything. | |
| ▲ | Towaway69 2 hours ago | parent | prev | next [-] | | > anyone potentially affected How does one know one is affected? What's the point of rotating tokens if I'm not sure that I've been affected - the new tokens will just be ex-filtrated as well. First step would be to identify infection, then clean up and then rotate tokens. | | |
| ▲ | mcintyre1994 an hour ago | parent [-] | | The article has some indicators of compromise, the main one locally would be .truffler-cache/ in the home directory. It’s more obvious for package maintainers with exposed credentials, who will have a wormed version of their own packages deployed. From what I’ve read so far (and this definitely could change), it doesn’t install persistent malware, it relies on a postinstall script. So new tokens wouldn’t be automatically exfiltrated, but if you npm install any of an increasing number of packages then it will happen to you again. |
| |
| ▲ | mcintyre1994 an hour ago | parent | prev | next [-] | | Also the user data destruction if it stops being able to propagate itself. | |
| ▲ | dawnerd an hour ago | parent | prev [-] | | Also a good reminder that you should be storing secrets in some kind of locker, not in plain text via environment variables or config files. Impossible to get everyone on board but if you can you should as much as possible. I hate that high profile services still default to plain text for credential storage. |
|
|
| ▲ | dmitrygr 4 hours ago | parent | prev | next [-] |
| Lucky for us C programmers. Each distro provides its own trusted libc, and my code has no other dependencies. :) |
| |
|
| ▲ | yupyupyups 8 hours ago | parent | prev | next [-] |
| Something helpful here would be to enable developers to optionally identify themselves. Not Discord-style where only the platform knows their real identity, but publically as well. |
| |
| ▲ | gruez 6 hours ago | parent | next [-] | | So, EV code signing certificates? Windows has that, and it'll verify that right in the OS. Git for instance, shows as being signed by CN = Johannes Schindelin
O = Johannes Schindelin
S = Nordrhein-Westfalen
C = DE Downside is the cost. Certificates cost hundreds of dollars per year. There's probably some room to reduce cost, but not by much. You also run into issues of paying some homeless person $50 to use their identity for cyber crimes. | | |
| ▲ | mc32 5 hours ago | parent [-] | | How would the homeless chap have the creds or gravitas for people to trust him or her? | | |
| ▲ | veeti 2 hours ago | parent [-] | | I don't really know who Johannes Schindelin is either but use git quite happily. |
|
| |
| ▲ | dcrazy 6 hours ago | parent | prev | next [-] | | This is what macOS codesigning does. Notarization goes one step further and anchors the signature to an Apple-owned CA to attest that Apple has tied the signature to an Apple developer account. | | |
| ▲ | laserbeam 4 hours ago | parent [-] | | As I understand it, this attack works because the worm looks for improperly stored secrets/keys/credentials. Once it find them it publishes malicious versions of those packages. It hits NPM because it’s an easy target… but I could easily imagine it hitting pip or the repo of some other popular language. In principle, what’s stopping the technique from targeting macos CI runners which improperly store keys used for Notorization signing? Or… is it impossible to automate a publishing step for macos? Does that always require a human to do a manual thing from their account to get a project published? |
| |
| ▲ | morkalork 5 hours ago | parent | prev [-] | | You don't think bad actors don't have access to entire countries worth of stolen identities to use for supply chain attacks? | | |
| ▲ | hirsin 5 hours ago | parent [-] | | This was largely the reason I rejected "real name verification" ideas at GitHub after the xz attack. (Especially if they are state sponsored) it's not that hard for a dedicated actor (which xz certainly was) to get a quality stolen identity. The inevitable evolution of such a feature is a button on your repo saying" block all contributors from China, Russia, and N other countries". I personally think that's the antithesis of OSS and therefore couldn't find the value in such a thing. | | |
| ▲ | morkalork 5 hours ago | parent [-] | | That would be easily defeated by a VPN. The inevitable evolution would be some kind of in-person attestation of identity backed up with some kind of insurance on the contributor's work, and, well you're converging on the employer-employee relationship then. | | |
| ▲ | hirsin 5 hours ago | parent [-] | | Yep, I saw the cat and mouse ending at ever increasingly invasive verifications involving more parties, that could ultimately still be worked around by a state actor. We already get asked for "block access from these country ip ranges please" as a security measure despite it being trivially bypassed, so it is easy to predict a useless but strong demand for blocking users based on their verified country. |
|
|
|
|
|
| ▲ | Incipient 9 hours ago | parent | prev | next [-] |
| Surely in this day and age we can fairly trivially find out these come from the usual suspects - China, Russia, Iran, etc. Being in such a digital age, where our economies are built on this tech...is this not effectively (economic) warfare? Why are so many governments blase about it? |
| |
| ▲ | bhouston 3 hours ago | parent | next [-] | | The US and Israel also have advanced penetration teams. But they wouldn't be this sloppy - they want persistent advanced access. I suspect Iran, Russia and China also wouldn't be this sloppy. This is too wide ranging and easily detectable and scattershot. This feels like opportunistic cyber criminals, or North Korea (which acts like cyber criminals.) | | |
| ▲ | Towaway69 2 hours ago | parent [-] | | Or anti-virus companies selling more of their wares. This kind of large scale attack is perfect advertising for anyone selling protection against such attacks. Spy agencies have no interest in selling protection. |
| |
| ▲ | halJordan 7 hours ago | parent | prev | next [-] | | It shouldn't be a "get the foreigners!" situation. Sure that is a method of solving the symptoms. But what you're really asking for is ... a software bill of materials. Why dont we have that yet? Bc it's cheaper to get ripped off than it is to pay for a bom. Thats the real problem | | |
| ▲ | c0balt 6 hours ago | parent | next [-] | | SBOMs exist. You can get them generated for most software via package managers in standard forms like cyclonedx. It's just not that effective when the SBOM becomes unmanageable. For example, our JS project at $work has 2.3k dependencies just from npm. I can give you that SBOM (and even include the system deps with nix) but that won't really help you. They are only really effective when the size is reasonable. | |
| ▲ | Ekaros 2 hours ago | parent | prev [-] | | SBOM really doesn't do much when compromise happens before or while you are building it. It really is orthogonal to these types of attacks. Best you can do is to find that you were compromise afterwards. |
| |
| ▲ | Nextgrid 8 hours ago | parent | prev | next [-] | | Proving the attack is state-sponsored is difficult (as any attack you attribute to a country can very well be a false-flag operation), and “state sponsorship” is itself a spectrum; for example, you could argue India’s insufficient action against tech-support scammers is effectively state-sanctioned. This can of course be resolved, but here’s the kicker: our own governments equally enjoy this ambiguity to do their own bidding; so no government truly has an incentive to actually improve cross-border identity verification and cybercrime enforcement. Not to mention, even besides government involvement, these malicious actors still “engage” or induce “engagement” which happens to be the de-facto currency of the technology industry, so even businesses don’t actually have any incentive of fighting them. | | |
| ▲ | mc32 5 hours ago | parent [-] | | A one or two off can be a false flag, thousand upon thousands is not going to be a false flag. |
| |
| ▲ | epolanski 6 hours ago | parent | prev | next [-] | | They aren't, in fact the very true happens, that we are bombarded non stop with information that everything is the fault of actors from these companies even when it isn't. We should fight this kind of behavior (and our privacy) regardless of whose involved, yet our governments in the west have nurtured this narrative of always pointing at big tech and foreign actors as scape goats for anything privacy or hacking related. Also, any cyber attack tracker will show you this is a global issue, if you think there aren't millions of attacks carried out from our own countries, you're not looking enough. | |
| ▲ | csomar 5 hours ago | parent | prev | next [-] | | We are still bound to our primal instincts. If you cut the throat of a baby in the middle of Times Square, the outrage will be insane. Yet, lack of financing to hospitals can do that many times over but people are numb to it. Take the Jaguar hack, the economic loss is estimated at 2.5bn. Given an average house price in the UK of $300k, that’s like destroying ~8.000 homes. Do you think the public and international response will be the same if Russia or China leveled a small neighborhood even with no human casualties? | |
| ▲ | kachapopopow 3 hours ago | parent | prev [-] | | majority of these are actually north korea, india and america. the really disappointing ones are usually india and american and ones that lay dormant code are usually north korea. |
|
|
| ▲ | xyzal an hour ago | parent | prev | next [-] |
| Okay ... what best practices should I as a mere dev follow to be protected? Is the "cooldown" approach enough, or should every npm command be run in bubblewrap ... ? |
| |
| ▲ | mcintyre1994 an hour ago | parent [-] | | In this narrow case, using pnpm or something similar that blocks postinstall scripts by default should be sufficient. In general, you probably want to use a container/vm/sandbox of some sort so dev stuff can’t access anything else on your machine. |
|
|
| ▲ | TZubiri 11 hours ago | parent | prev | next [-] |
| Not all the npm packages, but always an npm package |
| |
| ▲ | cyanydeez 10 hours ago | parent [-] | | While you think this is a producer problem, it's simply a userland market. Just like in the 90s when viruses primarily went to windows, it' wasn't some magical property of windows, it was the market of users available. Also, following this logic, it then becomes survivorship bias, in that the more attacks they get, the more researchers spend time looking & documenting. | | |
| ▲ | elwebmaster 5 hours ago | parent | next [-] | | While it can happen to anyone npm does preselect the users most likely to unknowingly amplify such an attack. Just today I was working on a simple JS script while disconnected from the Internet, Qwen Coder suggested I “npm install glob” which I couldn’t because there was no internet, so I asked for an alternative and sure enough the alternative solution was two lines of vanilla JS. This is just one example but it is the modus operandi of the NPM ecosystem. | |
| ▲ | KevinMS 7 hours ago | parent | prev | next [-] | | > it' wasn't some magical property of windows no, it really was windows | | |
| ▲ | foobiekr 5 hours ago | parent [-] | | It really wasn't. MacOS classic was full of vulnerabilities as was OS/2 and Linux up through 2004. Windows dominated because it was the biggest ecosystem. | | |
| |
| ▲ | TZubiri 9 hours ago | parent | prev [-] | | right, npm users. The extreme demand for simple packages and the absent consideration creates an opportunity for attackers to insert "free" solutions. The problem are the 'npm install' happy developers no doubt. |
|
|
|
| ▲ | ChrisArchitect 17 hours ago | parent | prev [-] |
| Discussion: https://news.ycombinator.com/item?id=46032539 |
| |
| ▲ | ares623 10 hours ago | parent [-] | | Phew, thought it was another one. | | |
| ▲ | gchamonlive 10 hours ago | parent [-] | | > Our internal monitoring system has uncovered multiple infected packages containing what appears to be an evolved version of the "Shai-Hulud" malware. Although it's not entirely new, it's something else. | | |
| ▲ | prophesi 5 hours ago | parent [-] | | Gitlab's post and the linked discussion thread are both from November 24th 2025. I may be misreading the parent comment, but I'm personally thankful there isn't a Return of the Return of Shai-Hulud, as I assumed this was a third recent incident. For those concerned about these attacks, Helixguard's post (from the linked discussion) lists out the packages they found to be effected, while Gitlab's post gives more information on how the attack works. Since it's self-propagating though, assume the list of affected packages might be longer as more NPM tokens are compromised. |
|
|
|