| ▲ | eMudhra ignored Mozilla Root investigation deadline; blames social media(bugzilla.mozilla.org) |
| 135 points by IcePenguin 2 hours ago | 65 comments |
| |
|
| ▲ | bradfa 24 minutes ago | parent | next [-] |
| Many HSMs do not force you to use unique labels for keys. So it's somewhat understandable if a human mixed up some keys during a non-standard or deviated process. This doesn't explain how so many different keys were "generated" so poorly or why no one cared about mismatched signatures. And the discussion mentioning wrap/unwrap testing kind of feeds directly into the fear that either: 1. These weak keys were created outside the HSM and then imported. 2. These weak keys were meant to be the wrap keys somehow. Like, if they hadn't paid for some of the wrap/unwrap features on one of their HSMs and so were using easy-for-humans keys to handle the wrap/unwrap operations so they could more easily move keys between HSMs. |
| |
| ▲ | tomrod 5 minutes ago | parent | next [-] | | Makes me wonder where the certs are with private keys 1-255 that weren't in these 8. | |
| ▲ | baud147258 12 minutes ago | parent | prev [-] | | > Like, if they hadn't paid for some of the wrap/unwrap features on one of their HSM Also depending on the FIPS configuration of their HSM, some functionality like wrap/unwrap might not be available |
|
|
| ▲ | jwineinger 34 minutes ago | parent | prev | next [-] |
| Do I, as a US-based English speaking person, need to have CAs from India and other places trusted by default in my browsers/devices? My gut says no and it just represents unnecessary risk, but I'm also not fully aware of how interleaved cert issuers are with cross signing and such. |
| |
| ▲ | tptacek 32 minutes ago | parent | next [-] | | eMudhra is (was?) a global CA and had subsidiaries serving the US market. Your browser also doesn't have a nationalist configuration; the same browser serves people all over the world. The problem isn't foreign companies; it's bad CAs. | | |
| ▲ | seki285 23 minutes ago | parent [-] | | No, the problem is foreign companies. Foreign nations have interests and as a US citizen I want to only trust other US citizens with my security, so should any other citizen anywhere else e.g. India can rely on India. | | |
| ▲ | bunderbunder 8 minutes ago | parent | next [-] | | As a US citizen, I don't love this mindset. The US has relatively lax rules around cybersecurity and consumer protection. Some other countries are even worse. Furthermore, I continue to participate in a relatively borderless global Internet. Unconditionally placing identity politics ahead of those realities is a strange and impractical choice for a great many people. | |
| ▲ | mintplant 5 minutes ago | parent | prev | next [-] | | I just had a horrifying vision of a world where we have to purchase separate certificates from local certificate authorities in each country where we want to make a website available. shudder | |
| ▲ | paaloeye 11 minutes ago | parent | prev | next [-] | | Is it paranoia, stupidity, or ignorance? I really hope it's the last one.
Suggested Google prompt: “How does WebPKI work; explain to a US-citizen ONLY by an US-citizen". | |
| ▲ | cvhc 12 minutes ago | parent | prev | next [-] | | Millions of non-US citizens (foreign workers in the country and outsource workers) build American software. Too bad :doge: | |
| ▲ | whynotmaybe 12 minutes ago | parent | prev | next [-] | | And what about when you, as a US citizen, are in India? | |
| ▲ | mathisfun123 6 minutes ago | parent | prev | next [-] | | > as a US citizen I want to only trust other US citizens with my security man fox news has really done a number on people's brains..... | |
| ▲ | Vaushite 14 minutes ago | parent | prev [-] | | So if you want to visit an Indian website, your browser should say "no! US websites only!"? |
|
| |
| ▲ | paaloeye 23 minutes ago | parent | prev | next [-] | | Very bad argument. WebPKI and the internet are the global infrastructure.
For the sake of argument, what unnecessary risk do you have in mind? | |
| ▲ | throwaway7356 17 minutes ago | parent | prev | next [-] | | If you use the internet, yes. If you want only to connect to the US net, stop connecting to the internet and use a US browser instead of a web browser. | |
| ▲ | maxnoe 19 minutes ago | parent | prev | next [-] | | Many US institutions seem to have used eMudhra as a cert provider, including Cornell, and the U of Michigan, Maryland and Chicago. Check InCommon Certificate Service. | |
| ▲ | qurren 31 minutes ago | parent | prev [-] | | Until some website of some service you're using embeds an Indian-sourced JavaScript required for its support chat and you're unable to get a refund on a defective product you purchased. Or worse, you were billed incorrectly by some US-based service, their support chat loads Indian assets, and they threaten to hit your credit score and send debt collectors after you if you don't pay up. | | |
| ▲ | nofunsir 5 minutes ago | parent [-] | | US consumer protection laws do not and CANNOT require US citizens to trust foreign agents. Period. There are other ways to extract a lawful refund. |
|
|
|
| ▲ | IcePenguin 2 hours ago | parent | prev | next [-] |
| eMudhra's recent response regarding their Intermediate CA private key compromise, citing social media activity and investor relations as blockers to technical investigation and analysis: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c39 > "There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible." Mozilla Root Program's update, stating that eMudhra has not responded to investigatory questions within deadlines (and had provided an extension): https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c42 > "The Mozilla Root Program sent the attached questions to eMudhra on the 20th October at 20:31 UTC. eMudhra was asked to provide an initial response within 24 hours. After the deadline passed on October 21st, eMudhra asked for a further 24 hours to respond. As of the time of writing on Friday 23rd, Mozilla has not received a response to these questions." |
| |
| ▲ | jmspring an hour ago | parent [-] | | I hadn't heard of this Intermidate CA provider. I guess I've been a bit lax on keeping myself updated in the PKI space. That said, given the country it is based, it doesn't surprise me about the general lack of quality and seriousness about what they did and how it's been exposed. I may get downvotes, but this just screams amateur hour. | | |
| ▲ | paaloeye 20 minutes ago | parent | next [-] | | Somewhat liberal use of the verb "may" | |
| ▲ | dsr_ 38 minutes ago | parent | prev [-] | | The downvote is for generalizing to a country. | | |
| ▲ | 15155 28 minutes ago | parent | next [-] | | Is it possible to make a broadly-accurate generalization about work ethic in a given country? (There are no Foxconn-style suicide nets in the United States that I am aware of - so perhaps?) Do some countries possibly value quality or responsibility as a desirable property more than others at a cultural level? (Is "German Engineering" a real thing?) Why do 419-style scams seem to emanate from a few specific countries? Why is piracy on the high seas more prevalent in Somalia than the United Kingdom? | | |
| ▲ | lukan 16 minutes ago | parent | next [-] | | "Is it possible to make a broadly-accurate generalization about work ethic in a given country? " It is, but those generalisations are usually debatable and will then lead to nationalistic debates that are allmost always flame war style, with people shouting over each other who is from the worst nation. Which is why many flag anything, that smells like leading towards this. | |
| ▲ | VirusNewbie 13 minutes ago | parent | prev [-] | | Is it work ethic or is it nepotism/corruption? I mean if I'm going to make really large generalizations about Software Engineering talent by country, Swiss people are the absolute bottom of the barrel maybe followed by the cheap outsourcing countries next, but ymmv. I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here. | | |
| ▲ | 15155 3 minutes ago | parent [-] | | > Swiss people are the absolute bottom of the barrel maybe followed by the cheap outsourcing countries next By virtue of population, a generalization may or may not hold more weight. Anecdotes are also tricky to really apply broadly: it's simply impossible for you to have encountered that many Swiss software engineers because few exist, because few Swiss people exist. In 2019, for instance, there were over 300,000 Indian H-1B petitions, to Switzerland's total of 202. > I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here. Relative quality of life in Germany is objectively better in India by pretty much every metric, so there's an automatic/inherent bias in any immigrants you encounter. (More than 10% of the Indian population doesn't have access to modern toilet facilities, for instance.) The competition for U.S. immigrant visas is far higher amongst Indians simply by the numbers, so there may be a filtering function at play. |
|
| |
| ▲ | icedchai 14 minutes ago | parent | prev | next [-] | | It is poor form. But if you've ever worked with one of the WITCH companies, these statements come as no surprise. | |
| ▲ | moi2388 33 minutes ago | parent | prev [-] | | If you ever dealt with indian software companies you would also know it to be an accurate statement, however | | |
|
|
|
|
| ▲ | IcePenguin 2 hours ago | parent | prev | next [-] |
| Previous HN discussions: eMudhra intermediate CA compromise https://news.ycombinator.com/item?id=50166579 1271 points | galadran | 3 days ago | 504 comments -- Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds https://news.ycombinator.com/item?id=50177916 935 points | hkc | 2 days ago | 460 comments -- eMudhra to exit global SSL/TLS certificate business https://news.ycombinator.com/item?id=50195110 1111 points | bo0tzz | 1 day ago | 713 comments |
|
| ▲ | Plasmoid 42 minutes ago | parent | prev | next [-] |
| What responsibility does the CA that signed eMudhra's intermediate cert bear here? |
| |
| ▲ | IcePenguin 36 minutes ago | parent [-] | | There may be some scrutiny of Sectigo's part in this, however, they did revoke the cross-signed certs within 24 hours of acknowledging the issue: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c13 "Sectigo has completed the revocation of the affected cross-signed CA Certificates. A new CRL and OCSP responses have been published and are making their way to our CDN edge nodes right now. We will continue monitoring and evaluating this bug for further updates from Emudhra." | | |
| ▲ | spockz 22 minutes ago | parent [-] | | Yes that if reactive. But I think GP meant what their responsibility in this is given that they signed and thereby vouched for an intermediate ca that now has become untrustworthy. |
|
|
|
| ▲ | nraynaud 21 minutes ago | parent | prev | next [-] |
| I have to say that if one of the RSA public keys was really 4, I have to tip my hat, stumbling on a perfect square of an even number, that’s really a score. |
| |
|
| ▲ | Tomdarkness 26 minutes ago | parent | prev | next [-] |
| What kind of leverage do these browser CA programs have with eMudhra given they are no longer issuing TLS certificates? If eMudhra just don't comply with Mozilla/Chrome, is there anything they can actually do? |
| |
| ▲ | bauruine 3 minutes ago | parent | next [-] | | Their RSA root certificate is still trusted. They could say they want that removed and the questions would instantly stop. | |
| ▲ | bradfa 23 minutes ago | parent | prev [-] | | The browsers could kick ALL their keys out and break all of eMudhra's customers websites. Those customers could sue. |
|
|
| ▲ | skeledrew an hour ago | parent | prev | next [-] |
| If you can't take the heat... |
| |
| ▲ | jdiff 13 minutes ago | parent [-] | | They are already attempting to flee the kitchen as quickly as they can, unfortunately the rest of the planet that's still in the kitchen isn't keen on letting them escape the grease fire they started. |
|
|
| ▲ | tamimio 41 minutes ago | parent | prev | next [-] |
| When an engineer (or engineering firm) found that it caused a bridge collapse due to negligence, they are taken accountable, sometimes even prison times if harm or death happened. We rarely see the same in tech world for some reason, there’s still this green pass and the idea that tech are auxiliary and all its mistakes are just reset a password. That company should be accountable, the damage is far from over as these certs that made by stupid people could be anywhere from IoT to critical infrastructure. |
| |
| ▲ | iamnothere 5 minutes ago | parent | next [-] | | If you want formal engineering outcomes, you need formal engineering standards. But in the same way that redoing a home bathroom doesn’t require the same multimillion dollar process as building a bridge, not all software projects can fall under the same umbrella. Any scheme like this has to take into account where certification is actually needed and what the goals and costs are. And guess what: this already exists. DO-178C for aviation, ISO 26262 for automotive, IEC 62304 for medical, etc. But these standards are specialized to the industry because, no surprise, there is no appropriate one-size-fits-all software development approach. You can’t just say “software must be bug-free” and leave it at that. That’s been shown to be impossible. Even formally verified software can have spec bugs. | |
| ▲ | bob1029 18 minutes ago | parent | prev | next [-] | | Accountability is rarely something that a person or organization will spontaneously adopt of their own free will. It generally has to be forced from the outside by way of additional consequences. Right now, it would appear that the strongest external consequences for fucking up a CA are strongly-worded posts in a web forum. I would hesitate to call the wind-down of a business unit a "consequence" in this context because it's the default outcome. Everyone was already playing that game. We need an additional layer of pressure to encourage good behavior. | | |
| ▲ | throwaway7356 2 minutes ago | parent | next [-] | | > Right now, it would appear that the strongest external consequences for fucking up a CA are strongly-worded posts in a web forum. No, the strongest consequence is their stock value. It has gone down a lot and if they continue not following their agreed obligations as shown I hope that trend will continue. | |
| ▲ | Vaushite 12 minutes ago | parent | prev [-] | | No, it's not limited to strongly worded posts. The external consequences are usually immediate bankruptcy of the CA after the investigation concludes. An untrusted CA has absolutely no business model whatsoever, and may have to refund its former customers. | | |
| ▲ | bob1029 4 minutes ago | parent [-] | | > The external consequences are usually immediate bankruptcy of the CA after the investigation concludes. It would seem that we have different definitions of "wind-down". Everyone in the business operated with this consequence already on the table and it did not seem to provide the desired outcome. |
|
| |
| ▲ | Vaushite 13 minutes ago | parent | prev | next [-] | | That's because a bridge didn't collapse. | | |
| ▲ | ktm5j 2 minutes ago | parent [-] | | You don't know that. No one knows what's potentially happened out in the wild because of these faulty certificates yet. |
| |
| ▲ | ForHackernews 33 minutes ago | parent | prev [-] | | No one at OpenAI, Anthropic or Google will be held accountable for their various felony bots[0], Facebook encouraged an actual genocide[1], but HN commenters are baying for blood of some random Indian contracting firm that royally goofed up their SSL certs. [0] https://www.felonybench.com/ [1] https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb... |
|
|
| ▲ | patjensen 2 hours ago | parent | prev | next [-] |
| Revoked. |
|
| ▲ | ForHackernews an hour ago | parent | prev [-] |
| This seems like a fair response to me: > There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible. The event happened on April 30 2025 and was done by set of people who have worked in the PKI industry for several years and worked with us for a while. There is a lot of digging we need to do to understand the circumstances, why, what, how to share a meaningful report that can hopefully benefit the community. I do not know why some community members think we wont be getting back on this bug. When we signed up for the browser program trust, we also signed up for disclosure obligations to the community which we will uphold. https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c39 ETA- HN rate-limiting won't let me reply, but to the commenter asking how all the pressure could delay a sober analysis: I've never been involved with a mess like this, but I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire. |
| |
| ▲ | InsidiousWombat an hour ago | parent | next [-] | | If some social media nonsense makes it hard for you to do internal investigation on the problem that shows that you're organisationally incompetent in addition to whatever else in the process failed. | |
| ▲ | roblabla an hour ago | parent | prev | next [-] | | > I've never been involved with a mess like this, but I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire. In any sane business, the people handling the screaming investors are different from the people handling the social media mess, and they are also different from the people doing the RCA. That said, given the mess, I wouldn't be surprised if eMudhra only had one employee responsible for all those jobs... | |
| ▲ | wolvoleo an hour ago | parent | prev | next [-] | | No it does not. How does social media upheaval make internal analysis impossible? | | |
| ▲ | thumbsup-_- an hour ago | parent | next [-] | | because they are more interested in saving the image on social media than doing the right thing. | | |
| ▲ | Vaushite 11 minutes ago | parent [-] | | Which is their job. The "right thing" if that is to shut down the CA will be managed by the CAB, there is no need for eMudhra to shut themselves down when that will be done for them. |
| |
| ▲ | ghostbrainalpha an hour ago | parent | prev [-] | | I was involved in the an incident like this, and the intense pressure for public immediate action led to the (firing or resignation) of multiple people who were actually needed for the investigation. Something that could have been fully analyzed with those people in a couple weeks ending up taking two years with an independent outside auditor. The independent auditor would have been needed either way, but if we gave the original crew the opportunity to respond, THEN fired them, the outside auditor could have just verified the story much more quickly. | | |
| ▲ | bauruine 33 minutes ago | parent | next [-] | | You where involved in the biggest fuckup by orders of magnitude your whole industry ever did in it's existance? I'm interested to hear the story. | | | |
| ▲ | mcmcmc an hour ago | parent | prev [-] | | > the intense pressure for public immediate action led to the (firing or resignation) of multiple people who were actually needed for the investigation. That just sounds like a management failure. You don’t actually have to cave. |
|
| |
| ▲ | altairprime 35 minutes ago | parent | prev | next [-] | | > I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire. Such a business will not be long operating a CA, then: these sorts of requests are common and CA operations are done openly in public with all accidents openly admitted in plain view. If management can’t deliver an environment that prioritizes compliance over rage and blame, then termination of CA acceptance for the company they manage with such irrationality is a successful outcome for the process. I respect that there may be opposing corporate and/or cultural pressures, and I imagine that’s why they’re voluntarily exiting the CA business, but they’re at risk of having all issuances distrusted retroactively if they think that gives them the leeway to reply late. | |
| ▲ | criddell an hour ago | parent | prev | next [-] | | That's what managers are for. They shield the people doing the work from all the external distractions. | |
| ▲ | andor 25 minutes ago | parent | prev | next [-] | | Some of these questions shouldn't be that difficult to answer, and they were asked to provide an initial response anyways. For example, further up in the bug tracker, they stated that there was an error in a script. If they were able to find an error, why can't they provide the script? | |
| ▲ | bauruine an hour ago | parent | prev [-] | | Those are just more reasons to remove every cert they ever touched. |
|