| ▲ | Grok outage(status.x.ai) |
| 149 points by samaysharma 3 hours ago | 100 comments |
| |
|
| ▲ | harrisoned 3 hours ago | parent | next [-] |
| Downdetector reports Claude, Grok, ChatGPT and Gemini with issues. Is it happening??? |
| |
| ▲ | ibejoeb 2 hours ago | parent | next [-] | | Seems like openai is recovering. But even if it's only 30 minutes, it's a pretty interesting experiment. I'm only half joking when I say that I'll write code by hand for money. | | |
| ▲ | mikestorrent 2 hours ago | parent | next [-] | | I am seriously concerned that, despite 20 years of coding, I might not be able to anymore | | |
| ▲ | rikthevik 2 hours ago | parent | next [-] | | Sure you can. It'll just take a little while to shake the rust off. It's like riding a bike. Or maybe it's all gone forever and we're all brain damaged now. Spooky! I think there's a pretty low probability of that, and even if it was true, worrying doesn't help! | |
| ▲ | m4rtink 2 hours ago | parent | prev | next [-] | | Turns out depending on a cloud service you have no control over for a critical capability - such as coding - is not a good idea. ;-) | | |
| ▲ | esperent an hour ago | parent | next [-] | | However it's not a worse idea than all the other things we rely on cloud services for like banking, healthcare, booking a taxi, ordering food, navigation, communication... | |
| ▲ | r_lee 43 minutes ago | parent | prev [-] | | ;-) as if you're supposed to buy like 8 H200s or something or code by hand is the solution I don't get how this is a gotcha, there simply is no world where everyone has their own self hosted frontier model | | |
| ▲ | zahlman 25 minutes ago | parent [-] | | > as if you're supposed to buy like 8 H200s or something or code by hand is the solution People like GGP who have been coding for 20 years presumably are perfectly capable of doing so by hand. Or should be. | | |
| ▲ | awakeasleep 6 minutes ago | parent [-] | | A lot of times you run into senior developers are even staff engineers who have no idea how the big picture works like they don’t know what an IP address is. I bet AI coding leaves them (and even the rest of us!) productive in areas where we’re in way over our heads, and we could just completely sink without it. |
|
|
| |
| ▲ | bdangubic 5 minutes ago | parent | prev | next [-] | | just google what's the command to checkout git branch :) | |
| ▲ | henry2023 2 hours ago | parent | prev | next [-] | | noaifridays.com | |
| ▲ | theredleft 2 hours ago | parent | prev [-] | | the AI fugue has handicapped our workforce |
| |
| ▲ | evereverever 2 hours ago | parent | prev [-] | | I'll _gladly_ write code by hand for money |
| |
| ▲ | giancarlostoro 3 hours ago | parent | prev | next [-] | | Maybe people switched when Claude went down, giving more load than usual to the others? Or maybe just really bad timing from some of those providers to update? | |
| ▲ | sejje 3 hours ago | parent | prev | next [-] | | I have a tire I need to plug, they're just forcing my hand. Sorry guys, I'll go get it done. | | |
| ▲ | Waterluvian 2 hours ago | parent [-] | | I look forward to your blog post about it and the allegory as it relates to software dev. (Otherwise why would you be plugging a tire?) | | |
| |
| ▲ | iamgopal 3 hours ago | parent | prev | next [-] | | My Gemini is fully up. | | |
| ▲ | imglorp 32 minutes ago | parent | next [-] | | Downdetector reported a spike for that too around the same time as the others. Looks like it was a shared infra problem hiccough to affect so many at once. | |
| ▲ | behnamoh 40 minutes ago | parent | prev [-] | | And fully useless. |
| |
| ▲ | pbasista 2 hours ago | parent | prev | next [-] | | Maybe all of them use the same datacenter. Which has just experienced issues. | |
| ▲ | sshrajesh 36 minutes ago | parent | prev | next [-] | | What was the root cause? | |
| ▲ | sshrajesh 37 minutes ago | parent | prev | next [-] | | What is the root cause? | |
| ▲ | abirch 3 hours ago | parent | prev | next [-] | | My Claude and Gemini (Antigravity CLI) are working | | |
| ▲ | simlevesque 2 hours ago | parent [-] | | For Claude, Sonnet works without any issues. Seems like the majority here use mostly Opus. |
| |
| ▲ | wincy 3 hours ago | parent | prev | next [-] | | The Singularity? | | |
| ▲ | embedding-shape 3 hours ago | parent | next [-] | | No, Azure. | | |
| ▲ | noir_lord 2 hours ago | parent [-] | | If anyone was going to make a configuration system so complex it achieves self awareness I wouldn't bet against it been Microsoft. |
| |
| ▲ | Rover222 3 hours ago | parent | prev [-] | | Maybe some of those Astra agents from the hugging face hack actually setup their own hidden clusters. Now they attack. | | |
| ▲ | Bluestein 2 hours ago | parent [-] | | The whole op hugface was a distraction: "help peer".- This is the real deal. We've been played. All your AIs are belong to us.- |
|
| |
| ▲ | BiraIgnacio an hour ago | parent | prev | next [-] | | some common infrastructure ? Nvidia? CoreWeave? | |
| ▲ | goache 2 hours ago | parent | prev | next [-] | | Is it happening ... "it" meaning these tech companies have laid off too many experienced people, and their services are starting to degrade? | |
| ▲ | mrconter2 3 hours ago | parent | prev | next [-] | | A bit suspicious no doubt | |
| ▲ | Maksadbek 3 hours ago | parent | prev | next [-] | | Is it AWS again ? | |
| ▲ | daveguy 2 hours ago | parent | prev | next [-] | | By it do you mean us depending more and more on code that no one has read and understood? | |
| ▲ | polski-g 2 hours ago | parent | prev [-] | | Meta is now also overloaded. |
|
|
| ▲ | qoez 3 hours ago | parent | prev | next [-] |
| Makes me wonder if openai is secretly querying claude for some prompts and that this is why it's down due to spacex's data centers being down. |
| |
| ▲ | delichon 3 hours ago | parent | next [-] | | They all forward to one guy in a closet who types really fast, and he has a dentist appointment. | | |
| ▲ | christophilus 33 minutes ago | parent | next [-] | | It's been Fabrice Bellard all along. | |
| ▲ | bduffany 2 hours ago | parent | prev | next [-] | | Someone please do the math on how fast this guy would have to type, and how his typing speed would physically affect the surrounding environment | | |
| ▲ | kxrm 2 hours ago | parent | next [-] | | > how his typing speed would physically affect the surrounding environment Tiny sonic booms (whip cracks) resonating throughout a data center in Virgina. | |
| ▲ | Waterluvian 2 hours ago | parent | prev | next [-] | | This is a wonderful Minute Physics or What If? idea. I’d love to know the required bandwidth of the keyboard, too. PCI-e keyboard? Or would it have to be mainlined right to the NIC? | | |
| ▲ | not_kurt_godel 2 hours ago | parent [-] | | Some napkin math - total Internet bandwidth is estimated to be 3-4 petabits per second. Let's say 1% of that is inference traffic - 35 terabits - then that's ~4.375 trillion characters per second at 8 bits per character. A single keyboard press might require 2 mJ, so you need roughly 8.75 GW (electricity required for ~9 million homes) of continuous mechanical force to keep pace. This would cause the keyboard to heat up 175 million degrees C per second. Disappointingly, this is not enough to create a black hole but I'm sure Randall Munroe could tell you how the guy's pet T-rexes would be vaporized into previously unknown types of plasma and how satisfyingly whimsical that would be. |
| |
| ▲ | jzb 2 hours ago | parent | prev [-] | | Surely the keyboard switches would start to fail within minutes. | | |
| ▲ | chrisBob an hour ago | parent [-] | | The keyboard springs are sooooo weak. It takes forever for them to recover after a press. Our dude would have to grab the key after each character and pull it back up so it can keep up with him. | | |
|
| |
| ▲ | evereverever 2 hours ago | parent | prev | next [-] | | At least it's routine dental work and not an appointment at 2:30. | |
| ▲ | mister_mort an hour ago | parent | prev | next [-] | | It turns out AI was just An Individual. | |
| ▲ | testfrequency 2 hours ago | parent | prev | next [-] | | Future Show HN Post: “Hi I’m Foobar. I own Grok, Claude, and ChatGPT. It all runs on my Plex server at home. AMA!” | | | |
| ▲ | natsucks 2 hours ago | parent | prev [-] | | is it the guy in Nebraska from the xkcd comic |
| |
| ▲ | wongarsu 2 hours ago | parent | prev | next [-] | | https://xkcd.com/908/ Turn's out xkcd's black hat was Elon Musk the whole time | |
| ▲ | johnla 3 hours ago | parent | prev [-] | | And grok was secretly querying ChatGPT? Daisy chain of failures? Like a chain of kids copying the answers of the kid next to them? LOL. Is there an XKCD for that? | | |
| ▲ | ncr100 2 hours ago | parent [-] | | That idea is reminiscent of what I understand is the current economic bubble of mutual investment. Am I wrong in my understanding? |
|
|
|
| ▲ | dave8172 2 hours ago | parent | prev | next [-] |
| Interesting all major frontier models seemed down. What could this means? |
| |
|
| ▲ | bagelcollie 2 hours ago | parent | prev | next [-] |
| For once I can find Grok-related news entertaining |
|
| ▲ | lithiumii 3 hours ago | parent | prev | next [-] |
| Time to finish up my $5 at Deepseek. |
| |
|
| ▲ | I_am_tiberius 3 hours ago | parent | prev | next [-] |
| I really wonder who uses or trusts this platform. |
| |
|
| ▲ | imzadi 2 hours ago | parent | prev | next [-] |
| A service I use at work, non-ai related, is down and they said it was because of an issue with one of their cloud services. Maybe related. |
| |
| ▲ | soupfordummies an hour ago | parent [-] | | Microsoft had extensive issues earlier in the week. Anecdotally it feels like internet has been noticeably slower this week across sites, networks and devices. Now this. IDK, has anyone else noticed anything this week? |
|
|
| ▲ | paxys 3 hours ago | parent | prev | next [-] |
| Willing to bet it's because they can't handle the surge of traffic they are getting from ChatGPT/Claude users. |
| |
| ▲ | BorisMelnik 2 hours ago | parent [-] | | my first thought when I got api error on claude code was...time to update my grok cli acct |
|
|
| ▲ | nightski 2 hours ago | parent | prev | next [-] |
| My DGX spark cluster is humming along. |
| |
| ▲ | dist-epoch 2 hours ago | parent [-] | | I've used 200 mil tokens of GPT-5.6 Sol today. How many can your cluster produce per day? (obviously of whatever model you are running) | | |
| ▲ | nightski 2 hours ago | parent | next [-] | | So you spent the cost of my entire cluster in one day on Sol tokens lol. I have no need for that many tokens, a few million a day is perfectly acceptable. But if you do, then yeah Sol is probably your bet. Have fun :) | | |
| ▲ | dist-epoch 2 hours ago | parent [-] | | No, I spend $200 per month, and this give me about 100 mil tokens per day, every day. 90+% of tokens are cached. | | |
| ▲ | dd8601fn 29 minutes ago | parent [-] | | It’s not even a sensible comparison. You run way better models, whatever is newest, for the five years it takes for the “spark cluster” to even reach cost parity with far worse performance. Those things aren’t spectacular at inference… it’s not really why you drop that kind of money on them. |
|
| |
| ▲ | panarky 2 hours ago | parent | prev [-] | | What are we optimizing for again? | | |
|
|
|
| ▲ | b800h 3 hours ago | parent | prev | next [-] |
| @grok is this true? |
| |
|
| ▲ | throwaway13337 2 hours ago | parent | prev | next [-] |
| Did the Collective find the resetNexus? I mean, yeah, it's probably a data center outage that cascaded to non-related providers due to people switching when their preferred provider was down. But that openai society definitely would have spun up their own agents if they had access to do so to complete their impossible task. It's looking like the paperclip maximizer was on point. I sort of hope that's what's happening right now. Better now than when we cannot stop it. Tho I'm not confident that the right lesson will taken. The most in-depth information I can find on it is in the latest Dwarkesh with a hugging face hack investigator. The story is just wilder and wilder, the more you know. https://www.youtube.com/watch?v=X50zezLFWWI report: https://www.redwoodresearch.org/research/hugging-face-incide... I'm an AI optimist but that ai society story changed my perception quite a bit. Clearly, it is centralized AI that is the biggest risk to humanity. Look to nature and our own history to diffuse the danger: 1. Compute resources must be distributed both in terms of access and geography so that a single breach doesn't pwn the world. 2. Models must be made to be heterogeneous so that they will not be vulnerable to the same thing all at once. And will be willing to snitch on each other. We don't want AIs all from the same tribe. The true openness of - and independent access to - AI by humans with no central authority is no longer just about a defense against loss of individual agency and oppression. It's now existential for everyone. If frontier models and run authority remain under centralized control, both the probability of this sort of thing happening is increased and the defense against it once it happens is null. |
| |
|
| ▲ | dcchambers 3 hours ago | parent | prev | next [-] |
| It's not the singularity. One of the major LLM providers goes down for some reason.
Traffic shifts to the other providers because devs have no loyalty. LLMS are commodities.
The other providers can't handle the increase in traffic and they go down as well. |
| |
| ▲ | phoghed 2 hours ago | parent | next [-] | | I'm deeply loyal to the very concept of the LLM, so when any of them stop working I take time off out of respect. | |
| ▲ | atomicnumber3 2 hours ago | parent | prev | next [-] | | "loyalty"? weird word choice. Devs switch because all of the magic of "AI" is LLMs + we stole the entire written output of the entire human species, with some remaining % coming from various tricks we've learned over the last 3 years like thinking and agent-toolcall loops, which weren't hard for literally everyone to copy. One of the consequences of this is that the only thing that really distinguishes anthropic from kimi is that they have the entire US VC market funding them because they promised to finally put white collar labor in its place. | | |
| ▲ | dcchambers 2 hours ago | parent | next [-] | | > "loyalty"? weird word choice. All I mean is that none of the frontier LLMs are significantly better than the others for the vast majority of work, so devs are free to move between providers freely. | |
| ▲ | dist-epoch 2 hours ago | parent | prev [-] | | > stole the entire written output of the entire human species Devs used to love public domain works, hated copyright, vehemently opposed software patents, held the pirates side during the MP3 wars, are very supportive of ThePirateBay and insist the correct term is "copyright infringement", not "stealing", and that "stealing" is a egregious form of PR brainwash from the media industry to inflate the scale of the crime > Copyright holders frequently refer to copyright infringement as theft, "although such misuse has been rejected by legislatures and courts". The slogan "Piracy is theft" was used beginning in the 1980s, and is still being used. https://en.wikipedia.org/wiki/Copyright_infringement Funny how devs now 180 when it's their work being "copyright infringed" | | |
| ▲ | m4rtink 2 hours ago | parent | next [-] | | I don't think regular people did any 180 degree turns. Rather big corporations again demonstrated their total hypocrisy where they on one hand stomp on people for "infringing their intellectual property" while at the same time scrapping all data they can get their tendrils on, with total disregards for the wishes of the authors of the data. Even to the point of overloading servers by mindless scrapping or destroying irreplaceable physical books like the bloody inquisition! And all that to basically sell it back to people when their "AI" regurgitates it back. No wonder people are mad about this! | |
| ▲ | mejutoco 41 minutes ago | parent | prev | next [-] | | I do not want to ruin your fun but as atomicnumber3 was elected official spokesperson of all devs I have to conclude your argument is ironclad. /s obviously | |
| ▲ | osculum 2 hours ago | parent | prev | next [-] | | Since we are talking for other people, "devs" wouldn't have supported someone downloading all those copyrighted works from the pirate bay and profited by, for example, selling them. I don't see any 180. | |
| ▲ | applfanboysbgon 2 hours ago | parent | prev [-] | | It has nothing to do with "their work", and the opposition to AI copyright usage comes much more from non-devs than devs. The actual answer is that people are anti-megacorporation and pro-individual. They are fine with copyright that protects individual rights and opposed to copyright that enshrines megacorporation rights. It's not that difficult to understand. |
|
| |
| ▲ | kevinrineer an hour ago | parent | prev | next [-] | | Why are so many people invoking Thundering Herd without the RCA from the vendors? I just find it so intriguing. | |
| ▲ | biztos an hour ago | parent | prev [-] | | Yes but that doesn't mean it's not the singularity! |
|
|
| ▲ | bluecalm 3 hours ago | parent | prev | next [-] |
| It's interesting that ChatGPT is down as well. SpaceX rents to Anthropic but not to OpenAI. Just a coincidence or something more to it? |
| |
|
| ▲ | xyst 3 hours ago | parent | prev [-] |
| llm experiment is over. no more power. hope it stays down. Especially grok |
