| ▲ | A macOS bug that causes TCP networking to stop working after 49.7 days(photon.codes) |
| 139 points by RyanZhuuuu 10 hours ago | 88 comments |
| |
|
| ▲ | EdNutting 9 hours ago | parent | next [-] |
| I got tired of the AI writing before finding out if they even attempted to contact Apple about this issue? Does anyone know? Also, massively over-dramatised. Yes, a bug worth finding and knowing about, but it’s not a time bomb - very few users are likely to be affected by this. Knowing the nature of OS kernels, I’m guessing even just putting a Mac laptop to sleep would be enough to avoid this issue as it would reset the TCP stack - which may be why some people are reporting much longer uptimes without hitting this problem, since (iirc) uptime doesn’t reset on Macs just for a sleep? Only for a full reboot? Anyway, all in all, yeah hopefully Apple fix this but it’s not something anyone needs to panic about. |
| |
| ▲ | bigiain 8 hours ago | parent | next [-] | | > very few users are likely to be affected by this I have a reasonably strong suspicion that I experienced this a week or two back, on a MacBook that doesn't go into sleep automatically and quite likely had 50-ish days of uptime. It had all the symptoms described - tcp connections not working while I could still ping everywhere just fine, and all the other devices on the same network were fine. Switching WiFi networks and plugging in to ethernet didn't help. A reboot "fixed" it. | | |
| ▲ | EdNutting 8 hours ago | parent | next [-] | | I would not be surprised if people on HN were more likely to hit this issue than Apple's average users. We're a weird bunch ;) | | | |
| ▲ | castillar76 7 hours ago | parent | prev [-] | | Yep, I concur: this explains a bizarre behavior I’ve noted in my Mac laptops for ages now. I have a tendency to just suspend them without rebooting for ages, especially the work one that doesn’t leave my office as frequently. Periodically, I’d come in to find the system bizarrely frozen just as they describe: TCP stack blocked up, but everything else on it behaving normally. (Well, mostly: some apps would block starting and bounce eternally, but I suspect that’s because they’re trying to make a network call while starting up and it’s blocking.) The only fix was a reboot. It’s not a disaster, but very annoying. At least now I can just schedule a reboot every 30 days at minimum to keep things running. |
| |
| ▲ | RyanZhuuuu 9 hours ago | parent | prev | next [-] | | yes we have reported to Apple and they have filed it in their internal system. | | |
| ▲ | otterley 8 hours ago | parent | next [-] | | Did you need to make this blog post 20 pages long and have AI write it? Especially in such dramatic style? Remember the golden rule: if you can't be bothered to write it yourself, why should your audience be bothered to read it ourselves? | |
| ▲ | Aloisius 4 hours ago | parent | prev [-] | | Might want to update it if you used the blog post explanation because it's incorrect as justinfrankel noted below. From the post: tcp_now = 4,294,960,000 (frozen at pre-overflow value)
The mistake in the blog post is timer isn't wrapped, even though it notes it should be: timer = 4,294,960,000 + 30,000 = 4,294,990,000 - MAX_INT = 22,704
Therefore: TSTMP_GEQ(4294960000, 22704)
= 4294960000 - 22704
= 4294937296
= 4294937296 >= 0 ? → true! (not false)
This is a bug of course, but it would cause sockets in TCP_WAIT state to be reaped anytime tcp_gc() is called, regardless of whether 2*MSL has passed or not. This only happens though if tcp_now gets stuck after 4,294,937,296 ms from boot.A bug similar to what the blog described can happen however if tcp_now gets stuck at least 30 seconds before it it would have wrapped. Since tcp_now is only updated if there is TCP traffic, this can happen if there is no TCP traffic for at least 30 seconds before before it would roll over (MAX_INT ms from boot). It's should be easy to prevent the latter from happening with some TCP traffic, though reaping TCP_WAIT connections early isn't great either. |
| |
| ▲ | delusional 9 hours ago | parent | prev [-] | | Apparently no. They'll be fixing it themselves? It really reads like Claude run amok on the blog. > We are actively working on a fix that is better than rebooting — a targeted workaround that addresses the frozen tcp_now without requiring a full system restart. Until then, schedule your reboots before the clock runs out. |
|
|
| ▲ | tjohns 9 hours ago | parent | prev | next [-] |
| Does anybody else find these AI-authored blog posts difficult to read? Something about the writing style and structure just feels unnatural, it's hard put my finger on it. At the very least, the writing takes way too long to get to a point. |
| |
| ▲ | dawnerd 7 hours ago | parent | next [-] | | Same, AI written anything is really difficult for me to read and pretty exhausting. | |
| ▲ | gowld 9 hours ago | parent | prev [-] | | AI does a good job of condensing the blog post to 2 paragraphs -- Mac refuses to let the tcp_now clock rollover when it exceeds the max value in its data type. | | |
| ▲ | coldtea 9 hours ago | parent | next [-] | | Can it summarize it down to a non-post? | | | |
| ▲ | nslsm 8 hours ago | parent | prev [-] | | Use AI to expand your thoughts into a long-winded post, use AI to compress the long-winded post into something that can be digested by a human. | | |
| ▲ | EdNutting 7 hours ago | parent | next [-] | | This but Gemini and Email - literally marketed as "write bullet points and Gemini will draft your email", followed by "received a long email? Let Gemini summarise it for you." The world's most effective _de_compression technology for email - total waste of time and compute when combined, but each product would make sense in isolation if human-generated mail was the majority of email sent/received (except sadly it isn't). We're using AI to spam people, AI to detect spam, AI to write non-spam and AI to summarise non-spam. AI inefficiency at every level and no way back. | |
| ▲ | bigiain 8 hours ago | parent | prev [-] | | Step 3) Sam Altman profits. |
|
|
|
|
| ▲ | mcculley 10 hours ago | parent | prev | next [-] |
| > It will not be caught in development testing — who runs a test for 50 days? You don't have to run the system for 50 days. You can simulate the environment and tick the clock faster. Many high reliability systems are tested this way. |
| |
| ▲ | dezgeg 8 hours ago | parent | next [-] | | IIRC the initial value for the jiffies time counter in Linux kernel is initialized at boot time to something like five minutes before the wraparound point, precisely to catch this kind of issues. | | | |
| ▲ | hombre_fatal 9 hours ago | parent | prev | next [-] | | It uses a hardware clock, one that pauses during sleep. There is no tick. If you wanted to see how time impacts the program, you'd prob change fns like calculate_tcp_clock to take uptime as an argument so that you could sanity check it. | | |
| ▲ | mcculley 7 hours ago | parent | next [-] | | Yes. I do mean designing software to make it testable. The code that uses that value can be run in an environment where that value can be controlled. I have written code that does this same thing and built a test harness for it. | |
| ▲ | adamtulinius 8 hours ago | parent | prev [-] | | We're talking about a company that produces the hardware their OS is running on. I'm sure they can find a way to make the hardware clock run faster. |
| |
| ▲ | sho_hn 9 hours ago | parent | prev [-] | | Heck, many video games are tested this way. |
|
|
| ▲ | otterley 10 hours ago | parent | prev | next [-] |
| Sounds like it affects every open TCP connection, not just OpenClaw. (It's pretty rare for a TCP connection to live that long, though.) |
| |
| ▲ | josephcsible 10 hours ago | parent | next [-] | | Individual TCP connections don't need to live that long. Once a macOS system reaches 49.7 days of uptime, this bug starts affecting all TCP connections. | | |
| ▲ | throw0101d 10 hours ago | parent | next [-] | | > Once a macOS system reaches 49.7 days of uptime, this bug starts affecting all TCP connections. Current `uptime` on my work MacBook (macOS 15.7.4): 17:14 up 50 days, 22 mins, 16 users, load averages: 2.06 1.95 1.94
Am I supposed to be having issues with TCP connections right now? (I'm not.)My personal iMac is at 279 days of uptime. | | |
| ▲ | Aloisius 9 hours ago | parent | next [-] | | According to the post: $ netstat -an | grep -c TIME_WAIT If the count it returns keeps growing, you're seeing a slow leak. At some point, new connections will start failing. How soon depends entirely on how quickly your machine closes new connections. Since a lot of client traffic involves the server closing connections instead, I imagine it could take a while. It's unclear if it'll leak whenever your mac closes or only when it fails to get a (FIN, ACK) back from the peer so the TCP_WAIT garbage collector runs. If it's the latter, then it could take substantially longer, depending on connection quality. | | |
| ▲ | throw0101d 9 hours ago | parent [-] | | % netstat -an | grep -c TIME_WAIT | wc -l
1
| | |
| ▲ | Aloisius 9 hours ago | parent [-] | | You want to drop the wc -l. Mac `grep -c` counts lines that match, so it always prints 1 line, so piping to wc -l will always return 1. Or just open up and do netstat -an |grep TCP_WAIT and just watch it. If any don't disappear after a few minutes, then you're seeing the issue. | | |
|
| |
| ▲ | 0x457 9 hours ago | parent | prev | next [-] | | You can run `sysctl kern.boottime` to get when it was booted and do the math from there. I also can't reproduce. I want to say I have encountered this issue at least once, yesterday I before rebooted my uptime was 60 days. But it's not instant, it just never releases connections. So you can have uptime of 3 years and not run out of connections or run out shortly after hitting that issue. | |
| ▲ | spogbiper 9 hours ago | parent | prev | next [-] | | I'm just going from the bug description in the article, but it seems that depending on your network activity, the exact time you will actually notice an impact could vary quite a bit | |
| ▲ | driftcoder 9 hours ago | parent | prev | next [-] | | if it's in keepalive or retransmission timers, desktop use would mask it completely. browsers reconnect on failure, short-lived requests don't care about keepalives. you'd only notice in things that rely on the OS detecting a dead peer — persistent db connections, ssh tunnels, long-running streams. | |
| ▲ | bigiain 7 hours ago | parent | prev [-] | | > 17:14 up 50 days, 22 mins, 16 users, load averages: 2.06 1.95 1.94 > Am I supposed to be having issues with TCP connections right now? (I'm not.) If my skim read of the slop post is correct, you'll only have issues on that machine if it hasn't spent any of that time asleep. (I have one Macbook that never sleeps, and I'm pretty sure it hit this bug a week or two back.) |
| |
| ▲ | CamperBob2 10 hours ago | parent | prev [-] | | Sure they do. They need to live until torn down. They almost never do live that long, for whatever reason, but they should. | | |
| ▲ | josephcsible 10 hours ago | parent [-] | | I meant that having a connection live that long isn't necessary to trigger this bug. I know that for some workloads, it can be important for connections to live that long. |
|
| |
| ▲ | gpvos 10 hours ago | parent | prev [-] | | Obviously, OpenClaw is now more important than anything else. | | |
|
|
| ▲ | justinfrankel 9 hours ago | parent | prev | next [-] |
| have multiple macOS machines with 600-1000+ day uptimes, which do TCP connections every minute or so at a minimum, they are still expiring their TIME_WAIT connections as normal. these kernel versions: Darwin Kernel Version 20.6.0: Thu Jul 6 22:12:47 PDT 2023; root:xnu-7195.141.49.702.12~1/RELEASE_ARM64_T8101 arm64 Darwin Kernel Version 17.7.0: Wed Apr 24 21:17:24 PDT 2019; root:xnu-4570.71.45~1/RELEASE_X86_64 x86_64 so... wonder what that's about? |
| |
| ▲ | justinfrankel 9 hours ago | parent | next [-] | | ah reading their analysis, there are errors that explain this. Particularly this: tcp_now = 4,294,960,000 (frozen at pre-overflow value)
timer = 4,294,960,000 + 30,000 = 4,294,990,000
(exceeds uint32 max → wraps to a small number)
timer wraps to a small number, they say TSTMP_GEQ(4294960000, 4294990000)
they forgot to wrap it there, it should be TSTMP_GEQ(4294960000, small_number) = (int)(4294960000 - 4294990000)
= (int)(-30000)
= -30000 >= 0 ? → false!
wrong!There may be a short time period where this bug occurs, and if you get enough TCP connections to TIME_WAIT in that period, they could stick around, maybe. But I think the original post is completely overreacting and was probably written by a LLM, lol. | | |
| ▲ | Aloisius 7 hours ago | parent [-] | | There does appear to be a bug, but it's not what the blog describes. If tcp_now stops updating at <= 2^32 - 30000 milliseconds, then TSTMP_GEQ(tcp_now, timer) will always fail since timer is tcp_now + 30000 which won't wrap. This does look like it is possible since calculate_tcp_clock() which updates tcp_now only runs when there's TCP traffic. So if at 49 days uptime you halted all TCP traffic and waited about a day, tcp_now would be stuck at the value before you halted TCP traffic. In cases where tcp_now gets stuck at > 2^32 - 30000, it looks like TCP sockets in the TIME_WAIT will end up being closed immediately instead of waiting 30 seconds, which isn't great either. | | |
| |
| ▲ | comex 9 hours ago | parent | prev [-] | | The bug was introduced only last year in macOS 26: https://github.com/apple-oss-distributions/xnu/blame/f6217f8... | | |
| ▲ | plorkyeran 9 hours ago | parent | next [-] | | > Apple Community #250867747: macOS Catalina — "New TCP connections can not establish." New connections enter SYN_SENT then immediately close. Existing connections unaffected. Only a reboot fixes it. This is a weird thing to cite if it's a macOS 26 bug. I quite regularly go over 50 days of uptime without issues so it makes sense for it to be a new bug, and maybe they had different bugs in the past with similar symptoms. | |
| ▲ | Aloisius 9 hours ago | parent | prev [-] | | Interesting. The article mentions complaints on the forums running Catalina, so that must be something else. | | |
| ▲ | js2 9 hours ago | parent | next [-] | | As someone who also operates fleets of Macs, for years now, there is no possible way this bug predates macOS 26. If the bug description is correct, it must be a new one. | |
| ▲ | groby_b 9 hours ago | parent | prev [-] | | The article is written using AI, so unless you verified the complaints, the safe default assumption is that they don't exist. | | |
|
|
|
|
| ▲ | netcoyote 7 hours ago | parent | prev | next [-] |
| This type of problem plagues all sorts of software. Having experienced this type of problem before, for Guild Wars game servers -- which run deterministic game instances that live for long periods of time -- we initialized a per-game-context variable that gets added to Windows GetTickCount() to a value such that the result was either 5 seconds before 0x7fff_ffff ticks, or 5 seconds before 0xffff_ffff ticks, so that any weird time-computation overflow errors would be likely to show up immediately. |
| |
| ▲ | toast0 3 hours ago | parent [-] | | Yep, everything that relies on overflow needs to overflow soon after start, so that it's well tested. |
|
|
| ▲ | JensRantil 8 hours ago | parent | prev | next [-] |
| This reminds me of the Linux kernel scheduler bug that kicked in after 208 days: https://www.claudiokuenzler.com/blog/247/linux-virtual-serve... |
| |
|
| ▲ | loloquwowndueo 10 hours ago | parent | prev | next [-] |
| lol reminds me of the windows 95 crash bug after 49.7 days. Have we learned nothing. https://pipiscrew.github.io/posts/why-window/ |
| |
| ▲ | aranelsurion 10 hours ago | parent | next [-] | | I was just trying to remember where did I last see this magic number of days. | | |
| ▲ | loloquwowndueo 10 hours ago | parent | next [-] | | The article does mention a few instances found over the years, including the windows one. That’s the one I remember though because we used to joke it was not a big deal - the only way for a windows 95 computer to reach 49 days of uptime is if it’s literally not doing anything or being used in any way. Windows 95 would crash if you looked at it funny. | | |
| ▲ | StilesCrisis 9 hours ago | parent | next [-] | | And throws in a Pac-man 8-bit level counter overflow just to remind us that AI cannot be trusted! | |
| ▲ | flomo 9 hours ago | parent | prev [-] | | OS/2 had a similar bug, and people used that as a server, so I'm sure it bit some people. |
| |
| ▲ | larodi 10 hours ago | parent | prev [-] | | 49-7=42
it is all clear |
| |
| ▲ | auspiv 10 hours ago | parent | prev | next [-] | | probably same thing for boeing 787 jets - https://www.theregister.com/2020/04/02/boeing_787_power_cycl... says 51 days, which would be an interesting number of (milli)seconds | | |
| ▲ | otherme123 9 hours ago | parent [-] | | It could be an overflow but related with the frequency at which the register was increasing, rather than the max value of te register. E.g. +1 this uint16 (65535) once every 500,000 cycles on this 32 Mhz chip, that previously was a 1 Mhz chip and never had a problem. |
| |
| ▲ | ok123456 10 hours ago | parent | prev | next [-] | | Quite literally "the new old thing." | |
| ▲ | znpy 9 hours ago | parent | prev [-] | | that's why the 49.7 days sounded familiar! |
|
|
| ▲ | gghootch 10 hours ago | parent | prev | next [-] |
| What does this have to do with OpenClaw exactly? |
|
| ▲ | beanjuiceII 10 hours ago | parent | prev | next [-] |
| i'm on sequoia M1 laptop with
uptime
16:38 up 228 days, 21:03, 1 user, load averages: 6.14 5.93 5.64 guess i'm marked safe! |
|
| ▲ | dvh 10 hours ago | parent | prev | next [-] |
| Exactly like arduino |
|
| ▲ | bawolff 8 hours ago | parent | prev | next [-] |
| Wasn't windows 95 famous for having an issue like this? |
| |
| ▲ | guywithahat 8 hours ago | parent [-] | | Arduino too; I assume they all have to do with storing milliseconds in a uint32_t, and then getting unpredictable behavior when it rolls over |
|
|
| ▲ | MatMercer 10 hours ago | parent | prev | next [-] |
| This made me remember some folks that are "I never reboot my MacOS and it's fine!". Yeah probably it is but I'll never trust any computer without periodic reboots lol. |
| |
| ▲ | QuantumNomad_ 9 hours ago | parent | next [-] | | I’m still at where when I connect external hard drive or SSD via USB, use it and then eject it, I shut down the MacBook Pro completely before I unplug the USB cable. Just in case. The longest uptime I have had on any of my recent laptops is probably around 90 days but that’s because that laptop was sitting in my garage with wall power connected (probably bad for the battery) and some external storage connected and I’d remote into that machine over WireGuard now and then. When I did reboot that machine it was only out of habit that I accidentally clicked on reboot via a remote graphical session. Most of the time my remote use of the laptop in the garage would be ssh sessions, but occasionally I’d use Remote Desktop. Right after I clicked reboot in the Remote Desktop session I realized what mistake I had just done - I have WireGuard set up to start after login. So after the reboot, I was temporarily unable to get back in. As I was in another country I couldn’t just walk over to the garage. But I do have family that could, so I instructed one of them over the phone on how to log in for me so that WireGuard would automatically start back up. You’d think this would happen only once, but I probably had to send family to the garage on my behalf maybe three or four times after me having made the same mistake again. For the laptops that I actually carry around and plug and unplug things to etc, normal amount of time between reboots for me is somewhere between every 1 and 3 days. Cold boot is plenty fast anyway, so shutting it down after a day of work or when ejecting an external HDD or SSD doesn’t really cost me any noticeable amount of time. | | |
| ▲ | Delk 9 hours ago | parent [-] | | > I’m still at where when I connect external hard drive or SSD via USB, use it and then eject it, I shut down the MacBook Pro completely before I unplug the cable. Just in case. That sounds... a bit paranoid? At least on Linux (Gnome), if I click to "safely remove drive" it actually powers off the drive and stops external mechanical drives from spinning. No useful syncing is going to happen anyway once a hard drive no longer spins. A modern OS should definitely be reliable enough that it can be trusted to properly unmount a drive. > For the laptops that I actually carry around and plug and unplug things to etc, normal amount of time between reboots for me is somewhere between every 1 and 3 days. Cold boot is plenty fast anyway, so shutting it down after a day of work or when ejecting an external HDD or SSD doesn’t really cost me any noticeable amount of time. I personally don't reboot my laptop that often, but it's not because of a boot taking too much time. It's because I like to keep state: open applications, open files, terminal emulator sessions, windows on particular virtual desktops, etc. | | |
| |
| ▲ | exe34 9 hours ago | parent | prev [-] | | $ uptime 22:22:45 up 3748 days 21:20, 2 users, load average: 1.42, 1.36, 1.02 It's very funny, I think it's because my laptop battery died and when I replaced it, it had to update the time from 10 years ago? I'm not sure why, as the laptop is from mid-2012. | | |
| ▲ | jasonjayr 9 hours ago | parent [-] | | > 17:27:20 up 1112 days, 10:36, 50 users, load average: 0.20, 0.19, 0.18 I thought I had a record going here with my Dell laptop, but I guess you win. After a certain point, I just decided to see how long I can make it go. |
|
|
|
| ▲ | daveorzach 10 hours ago | parent | prev | next [-] |
| If you want to see exactly when your machine will hit this, I threw together a fish shell function that calculates the precise timestamp, mostly vibe coded. calc_tcp_overflow_time.fish: https://gist.github.com/daveorzach/64538f82a89fa24e5d134557c... monitor_tcp_time_wait.fish: https://gist.github.com/daveorzach/0964a7a67c08c50043ff707cf... |
|
| ▲ | Philpax 10 hours ago | parent | prev | next [-] |
| Ctrl+F "OpenClaw". No results. Que? |
|
| ▲ | apatheticonion 9 hours ago | parent | prev | next [-] |
| Ignoring the AI article contents. God I wish Apple offered first party support for Linux on Mac computers. |
|
| ▲ | throw03172019 10 hours ago | parent | prev | next [-] |
| I only have 11 days left until my machine crashes and I lose all of my tabs. |
|
| ▲ | fortran77 9 hours ago | parent | prev | next [-] |
| Nobody keeps their Macs running for more than 49.7 days? We have Windows Servers here (with long-term TCP/IP connections) that are only rebooted every 6 months to apply patches. |
|
| ▲ | poppafuze 9 hours ago | parent | prev | next [-] |
| https://news.ycombinator.com/item?id=41939318 |
|
| ▲ | nalekberov 9 hours ago | parent | prev | next [-] |
| I rarely restart my Mac mini, and I have never had such an issue beyond my internet provider suddenly stopping properly working in the middle of the night. |
|
| ▲ | revv00 8 hours ago | parent | prev | next [-] |
| Orz! A kindly reminder for rebooting. |
|
| ▲ | WesolyKubeczek 10 hours ago | parent | prev | next [-] |
| In case of OpenClaw, this is a feature. |
| |
| ▲ | 4fterd4rk 10 hours ago | parent [-] | | When some Russians do a prompt injection and OpenClaw is threatening to send your NSFW pics to Grandma unless you give it some Bitcoin all you have to do is drag out the negotiations for 49 days! |
|
|
| ▲ | cute_boi 10 hours ago | parent | prev | next [-] |
| too much words and text for simple thing..... probably written by openclaw |
|
| ▲ | jijji 10 hours ago | parent | prev | next [-] |
| I thought Alan Cox fixed all the TCP IP bugs in the early 1990s lol |
| |
|
| ▲ | awithrow 10 hours ago | parent | prev [-] |
| A ticking time bomb? What an overly dramatic way to talk about a bug that requires a reboot. Its not even a hard crash. |