| ▲ | yardstick 3 days ago |
| It’s been decades, why doesn’t getaddrinfo have a standardised way to specify a timeout? Set a timeout to 10 seconds and life becomes a lot easier. Yes I know in Linux you can set the timeout in a config file. But really the dns setting should be configurable by the calling code. Some code requires fast lookups and doesn’t mind failing which, while others won’t mind waiting longer. It’s not a one size fits all thing. |
|
| ▲ | eklitzke 3 days ago | parent | next [-] |
| A few reasons, I think. The first is that getaddrinfo is specified by POSIX, and the POSIX evolve very conservatively and at a glacial pace. The second reason is that specifying a timeout breaks symmetry with a lot of other functions in Unix/C, both system calls and libc calls. For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. There are ways to do these things in a non-blocking manner with timeouts using aio or io_uring, but those are already relatively complicated APIs for just using simple system calls, and getaddrinfo is much more complicated. The last reason is that if you use the sockets APIs directly it's not that hard to write a non-blocking DNS resolver (c-ares is one example). The thing is though that if you write your own resolver you have to consider how to do caching, it won't work with NSS on Linux, etc. You can implement these things (systemd-resolved does it, and works with NSS) but they are a lot of work to do properly. |
| |
| ▲ | jstimpfle 3 days ago | parent [-] | | > For example, you can't specify a timeout when opening a file, reading from a file, or closing a file, which are all potentially blocking operations. No they're not. Not really, unless you consider disk access and interacting with the page cache/inode cache inside the kernel to be blocking. But if you do that, you should probably also consider scheduling and really any CPU instruction to be blocking. (If the system is too loaded, anything can be slow). To be fair, network requests can be considered non-blocking in a similar way, but they're depending on other systems that you generally can't control or inspect. In practice you'll see network timeouts. Note that you (at least normally -- there might be tricky exceptions) won't see EINTR from read() to a filesystem file. But you can see EINTR for network sockets. The difference is that, in Unix terminology, disks are not considered "slow devices". | | |
| ▲ | jcalvinowens a day ago | parent | next [-] | | > No they're not. Not really, unless you consider disk access and interacting with the page cache/inode cache inside the kernel to be blocking. The important point is that the kernel takes locks during all those operations, and will wait an unbounded amount of time if those locks are contended. So really and truly, yes, any synchronous syscall can schedule out for an arbitrary amount of time, no matter what you do. It's sort of semantic. The word "block" isn't a synonym for "sleep", it has a specific meaning in POSIX. In that meaning, you're correct, they never "block". But in the generic way most people use the term "block", they absolutely do. | |
| ▲ | jcelerier 3 days ago | parent | prev | next [-] | | I'd consider "blocking" anything that given same inputs, state and cpu frequency, may take variable time. That means pretty much every system call and entering the system scheduler, doing something that leads to a page fault, etc. Pretty much only pure math in total functions and function calls to paged functions are acceptable. | | |
| ▲ | Joker_vD 3 days ago | parent [-] | | Yeah... the sudden paging in also has been noted as a source of latency in the network-oriented software. But that's the problem with our current state of the APIs and their implementations: ideally, you'd have as many independent threads of executions as you want/need, and every time one of them initiates some "blocking" operation, it is quickly end efficiently scheduled, and another ready-to-run thread is switched in. Native threads don't give you that context-switching efficiency, and user-space threads can accidentally cause an underlying native thread block even on "read a non-local variable". |
| |
| ▲ | surajrmal 2 days ago | parent | prev | next [-] | | In a data center, networks can have lower latency than disk (even ssd). Now the real place this all falls on its head is page faults. That there are definitely places where you need to have granular control over what can and cannot stall a thread from making progress. | |
| ▲ | Joker_vD 3 days ago | parent | prev [-] | | > disks are not considered "slow devices". And neither are the tapes. But the pipes, apparently, are. Well, unfortunately, disk^H^H^H^H large persistent storage I/O is actually slow, or people wouldn't have been writing thread-pools to make it look asynchrnous, or sometimes even process-pools to convert disk I/O to pipe I/O, for the last two decades. | | |
| ▲ | jstimpfle 2 days ago | parent | next [-] | | There is a misunderstanding. "Slow device" in the POSIX sense is about unpredictability, not maximal possible bandwidth. Reading from a spinning disk might be comparably slow in the bandwidth sense, but it's actually quite deterministic how much data you can shovel to or from it. A pipe on the other hand might easily stall for an hour. The kernel generally can't know how long it will have to wait for more data. That's why pipe reads (as well as writes) are interruptible. The absolute bandwidth number of a harddisk doesn't matter --- in principle you can overload any system such that it fails to schedule and complete all processes in time. Putting aside possible system overload, the "slow device" terminology makes a lot of sense. | | |
| ▲ | Joker_vD 2 days ago | parent [-] | | Seeking a tape also takes an unpredictable amount of time; and so is seeking a disk, for that matter (IIRC, historically it was actually quite difficult for UNIX systems to saturate disk's througput with random reads). | | |
| ▲ | jstimpfle 2 days ago | parent [-] | | According to ChatGPT, a tape device is actually considered a "slow device". Even though I'm not sure it's that unpredictable. Maybe for most common use cases it is. I was under the impression that seeking a disk you can generally calculate well with 10ms? Again, it depends on the file system abstractions built on top, and then the cache and the current system load -- how many seeks will be required? |
|
| |
| ▲ | jcalvinowens 21 hours ago | parent | prev [-] | | >> disks are not considered "slow devices". > And neither are the tapes. But the pipes, apparently, are. The "slow vs fast" language is really unfortunate, I realize it's traditional but it's unnecessarily confusing. A better way to conceptualize it IMHO is bounded vs unbounded: a file or a tape contains a fixed amount of data known a priori, a socket or a pipe does not. |
|
|
|
|
| ▲ | o11c 3 days ago | parent | prev | next [-] |
| On Linux you can do what you're asking with `getaddrinfo_a` + `gai_suspend`. As always, on non-Linux Unixen the answer is "screw you!" |
| |
| ▲ | throwawayoogux 3 days ago | parent | next [-] | | OpenBSD has getaddrinfo_async since 5.6 (March 2014). | |
| ▲ | sedatk 3 days ago | parent | prev | next [-] | | Just wanted to note that Windows doesn't have that problem either. Even Windows NT had async getaddrinfo() variants. | |
| ▲ | surajrmal 2 days ago | parent | prev | next [-] | | The fact that a library as polific as libcurl doesn't use it and tried to resort to pthread_cancel, known to have serious problems, points to a larger problem. Either posix needs to incorporate an async getaddrinfo or there is an education problem that needs to be addressed. | |
| ▲ | BobbyTables2 3 days ago | parent | prev [-] | | Wow, TIL ! Thanks! |
|
|
| ▲ | asveikau 3 days ago | parent | prev | next [-] |
| I think getaddrinfo_a is cancellable, including the ability to block with a timeout. It is a glibc extension. |
|
| ▲ | okanat 3 days ago | parent | prev | next [-] |
| Just leave DNS out, are there any POSIX standard async functionality for networking or even normal IO? All I know by reading some libraries is epoll or io_uring used on Linux, kevent on BSDs. |
| |
| ▲ | comex 3 days ago | parent | next [-] | | Yes for networking. You set your sockets into O_NONBLOCK mode and use poll() or select(). These APIs are in POSIX and also have direct equivalents in Winsock. There is also POSIX AIO for async I/O on any file descriptor, but at least historically speaking it doesn't work properly on Linux. | | |
| ▲ | epcoa 3 days ago | parent [-] | | POSIX AIO on “Linux” is implemented in glibc with userland threads using regular blocking syscalls behind the scenes. It basically works properly, it just doesn’t gain any potential efficiency benefits, it adds avoidable overhead, prone to priority inversion, etc. The linux kernel has no provision for POSIX AIO. Until io_uring the only asynchronous disk IO interface was the io_* syscalls, which were confusingly referred to as Asynchronous IO, though these have nothing to do with POSIX AIO and can only be used bypassing the page cache, and suck for general purpose use. |
| |
| ▲ | mort96 3 days ago | parent | prev [-] | | POSIX has poll for that. |
|
|
| ▲ | ComputerGuru 3 days ago | parent | prev [-] |
| I disagree, there are too many variables and ultimately the end user would be th one that knows best. The proper solution isn’t having the library or application dev, who has no idea what kind of network connection the user is running, the type of dns server (caching or not, lan or remote, etc) or the name servers of the target domain and their performance or availability. This is all really the domain of the sysadmin. The solution is to make it a properly non-blocking api. |
