| ▲ | mdasen 3 days ago |
| I'd say that you care about it being UNIX-like, not UNIX®. You don't care that Linux isn't UNIX. You don't care that GNU versions of things like ed and awk are slightly off-spec. In some ways, Apple's adherence to UNIX specifications probably makes macOS less useful for you. For example, I wish that grep on macOS was closer to GNU grep. When I look up commands online, I often find answers based on the GNU implementations. Those often work on macOS, but sometimes don't (or have subtly different behavior) because macOS is adhering to the UNIX specification rather than to what those utilities do on the vast majority of systems out there. I don't think Apple would be removing UNIX-like tools from macOS even without certification. They know how valuable it is that most developers use their systems. Even Microsoft went so far as to implement the Windows Subsystem for Linux for developers. At this point, I think that UNIX certification makes macOS less compatible with the tools and help out there which generally targets Linux. Usually the differences are small, but they certainly can be meaningful. |
|
| ▲ | lucideer 3 days ago | parent | next [-] |
| > I don't think Apple would be removing UNIX-like tools from macOS even without certification. They know how valuable it is that most developers use their systems. I hope you're right but I'm not as confident. Corporations - Apple included - have been guilty of some surprising ignorance when it comes to things like this. I'm thankful for this certification circus to continue so that we don't need to test your theory. |
| |
| ▲ | selimnairb 3 days ago | parent [-] | | Yeah, except Apple dogfoods macOS to build most of the software for Macs and everything else they make. Presumably they rely on UNIX-like tools and would have to retool as well. So why mess with what’s working for them and others? |
|
|
| ▲ | skissane 2 days ago | parent | prev | next [-] |
| > In some ways, Apple's adherence to UNIX specifications probably makes macOS less useful for you. For example, I wish that grep on macOS was closer to GNU grep. When I look up commands online, I often find answers based on the GNU implementations. Those often work on macOS, but sometimes don't (or have subtly different behavior) because macOS is adhering to the UNIX specification rather than to what those utilities do on the vast majority of systems out there. UNIX certification is not the reason why macOS utilities are missing options compared to GNU - UNIX standards say you have to have certain options which work a certain way, they don’t prohibit adding additional options as vendor extensions. The reason is that Apple’s investment in improving these tools is minimal because it is a low priority for them, and because people who get annoyed by this often just end up installing the GNU tools anyway (using Homebrew or MacPorts) In fact, GNU/Linux systems have been certified as UNIX in the past, by a couple of different Chinese vendors (Inspur K-UX, Huawei EulerOS)-which shows use of the GNU tools is no inherent obstacle to certification. The reason these vendors stopped, I suspect, is the money it was making them was smaller than the certification costs and UNIX trademark license fee |
| |
| ▲ | jchw 2 days ago | parent | next [-] | | Pretty sure GNU coreutils really does intentionally deviate from POSIX compliance in a handful of places, otherwise POSIXLY_CORRECT wouldn't exist. That said you're probably right, though I also suspect dealing with GPL licensing is another major reason they don't bother with things like GNU coreutils. (Obviously they definitely wouldn't have done it after coreutils switched to GPLv3, but I'm sure even before then they would've greatly preferred permissively-licensed software.) | | |
| ▲ | chasil 2 days ago | parent | next [-] | | There is some subtlety that you are missing here. Outside of coreutils, let's consider bash and ksh88. The two have differing behavior in several areas (coprocesses, alias handling, final pipeline fork, etc.), but this divergence in behavior happened before POSIX.2 and the standardization of the POSIX shell, which is largely a subset of ksh88. The gist is that activating a mode for POSIX compliance will generally remove functionality, because the standardization happened a decade after development began, and the standards themselves were excessively conservative in adherence to System V. I've seen that useful GNU extensions are generally adopted by BSD, but much more slowly by POSIX. That does not serve UNIX well. Someone should challenge the Austin Group for effective control of UNIX standardization. | | |
| ▲ | jchw 2 days ago | parent [-] | | AFAIK, enabling POSIXLY_CORRECT doesn't get rid of any functionality. It changes some very subtle behaviors, such as the way certain argument parsing edge cases would be handled. Anyway, I think this is somewhat a non-issue: even if bash doesn't fully comply with POSIX standards by default, it should still be possible to be POSIX compliant by delivering a compliant shell in the right place. Though this does make me wonder if there's anything in POSIX that would require the user's default login shell to be POSIX-compliant, Bourne shell compatible. Probably not, right? After all, macOS had been using bash for ages with no issues complying. | | |
| ▲ | chasil 2 days ago | parent [-] | | Nope nope nope. You can see this in a script by defining: alias p=printf
Then try to use it with bash. If bash is running as #!/bin/sh, then it will work, because bash is forced into POSIX mode.However, if the script is running as #!/bin/bash, then you will be in the '80s behavior, and it will fail. Try it. | | |
| ▲ | jchw 2 days ago | parent [-] | | Bash isn't part of GNU coreutils. | | |
| ▲ | chasil 2 days ago | parent [-] | | I realize that, but I'm illustrating that POSIX.2 required a retrofit to bash, and probably required similar adjustments to the rest of userland, including coreutils. | | |
| ▲ | jchw 2 days ago | parent [-] | | I knew about the fact that bash behaves quite differently in POSIX mode, but that isn't much of a problem in most cases since nobody is forcing you to use a POSIX-compatible bourne shell as your login shell or for scripting, it's just the shell that you can guarantee will exist if something is POSIX compliant, right? If I were addressing bash, I would've said set -o posix instead of POSIXLY_CORRECT. (I didn't even realize POSIXLY_CORRECT did anything to bash.) The GNU bash documentation covers the differences pretty well: https://www.gnu.org/software/bash/manual/html_node/Bash-POSI... GNU coreutils however, the behavior differences seem rather minor, and I couldn't find exhaustive documentation. However, I may as well try to back this up with more than conjecture since we're already this deep in the thread. Let's dig into GNU coreutils and see what POSIXLY_CORRECT appears to do as of current git HEAD: - cp: Allow the destination to be a dangling symlink when POSIXLY_CORRECT is set. - dd: Does not trap SIGINFO if it's equal to SIGUSR1 (default) and POSIXLY_CORRECT is set. I guess this means that POSIXLY_CORRECT makes the `pkill -USR1 dd` thing not work? - df: Use 512-byte block size if POSIXLY_CORRECT is set, otherwise 1024. - echo: POSIXLY_CORRECT disallows parsing options unless the first option is `-n`, and enables parsing "v9"-style interpretation of backslash escapes. Demonstration: `$(which echo) -e \\n` - id: Will not print SELinux context even when --context is passed. Not sure why. This is the only thing I've seen that explicitly disables functionality. - nohup: The exit code for internal failures is 127 instead of 125 when POSIXLY_CORRECT is set. - pr: Changes default date format when POSIXLY_CORRECT is set. - printf: POSIXLY_CORRECT disables a warning about ignored characters following a character constant. Demonstration: `$(which printf) %x "'xx"` - same output in both modes, but in POSIXLY_CORRECT you are not warned about the second x being ignored. - pwd: Defaults to using -L ("logical" mode, uses $PWD value as long as it refers to the CWD) instead of -P. - readlink: Defaults to --verbose if POSIXLY_CORRECT is set. - sort: Allow operands to be parsed after files if POSIXLY_CORRECT is not set. - touch: Seems to disable some kind of warning when an invalid date is passed. - uniq: Seems to be the same as sort. - wc: Treats non breaking space characters as word delimiters, if POSIXLY_CORRECT is unset. I believe this is an exhaustive list as of GNU coreutils f4dcc2a495c390296296ad262b5a71996d0f6a86. | | |
| ▲ | chasil 2 days ago | parent [-] | | I still run some rhel5, and there were quite a few standard options that were not implemented by GNU. Looking now is good, but looking in the past is also illuminating. I generally trust busybox to give me both a uniform and compliant userland, certainly more than rhel5 coreutils. | | |
| ▲ | jchw 2 days ago | parent [-] | | I only chose the latest version because I figured it would have the most POSIXLY_CORRECT effects. Documentation seems to confirm this: the NEWS file documents added effects over time, but not removed ones, it seems. I wouldn't necessarily be surprised if GNU coreutils from RHEL5 is old enough to be missing some options needed to comply with POSIX, or if it complied with older POSIX standards, but I think we're losing track here. GNU coreutils maintains essentially all of its functionality and options when in POSIXLY_CORRECT mode; there's really just a handful of differences and they are mostly around edge cases that few people care about, except insofar as it breaks their scripts, especially since in most cases they'll be using shell builtins instead which don't follow the POSIX behavior anyway. I am not really arguing for or against GNU coreutils. I currently use GNU coreutils and would prefer GNU-compatible coreutils on my systems purely as a matter of muscle memory, regardless of whether it is in POSIX compatibility mode or not. That said, I don't think GNU coreutils are necessarily anything special, and the utilities that macOS and BusyBox provide are almost always perfectly fine with me, with some minor exceptions. I'm sure the same will be true if I ever try a uutils-based system. The only point to be made here is that at least as far as coreutils go, it doesn't really seem like POSIX compliance is a hindrance. If anything stopped macOS from using GNU coreutils, I suggest it's probably to avoid having more GPL software in macOS (especially post-GPLv3.) Though there could be multiple factors at play. | | |
| ▲ | chasil 2 days ago | parent [-] | | And I'll get back to my original point. When Debian demoted bash and migrated to the Almquist shell, there was great anguish from Ubuntu users of all tiers (Adobe was notable, IIRC). That anguish was due to a decade that predated POSIX.2. That was a substantial hindrance. | | |
| ▲ | jchw 2 days ago | parent [-] | | I absolutely remember Debian switching to Almquist shell, but that was about more than just POSIX compliance. IIRC a big deal at the time was that the cost of starting and using GNU bash all over the place was actually a measurable performance impact, and switching to dash improved on this. Also, bashisms became pervasive in scripts with /bin/sh hashbangs, which is definitely wrong no matter how you feel about POSIX. And anyway, this tangent doesn't feel terribly connected to this discussion thread since macOS never had this issue in the first place and this particular discussion thread was never really about UNIX shells... |
|
|
|
|
|
|
|
|
| |
| ▲ | skissane 2 days ago | parent | prev [-] | | > Pretty sure GNU coreutils really does intentionally deviate from POSIX compliance in a handful of places, otherwise POSIXLY_CORRECT wouldn't exist. To get UNIX certification, you can just patch it to make POSIXLY_CORRECT=1 the default. Or even don’t patch the utilities, and just patch /etc/profile to set POSIXLY_CORRECT=1 globally. UNIX certification requires that the system have a mode of operation available which passes the test suite; the existence of config settings which if changed from their defaults produce standards violations is not in itself a standards violation. | | |
| ▲ | jchw 2 days ago | parent [-] | | The point is that the default build of GNU coreutils in the default configuration is not POSIX compliant, not that it can't be made to be POSIX compliant. Obviously it can be done, otherwise that environment variable would not exist. |
|
| |
| ▲ | pornel 2 days ago | parent | prev [-] | | Apple got spooked by GPL v3 anti-tivoization clauses and stopped updating GNU tools in 2007. macOS still has a bunch of GNU tools, but they appear to be incompatible with GNU tools used everywhere else, because they're so outdated. | | |
| ▲ | wkat4242 2 days ago | parent [-] | | And Apple is doing a lot of Tivoization these days. They're not yet actually stopping apps that they haven't "notarized" but they're not making it easier. One of the many reasons I left the Mac platform, both private and at work. The other reason was more and more reliance on the iCloud platform for new features (many of its services don't work on other OSes like Windows and Linux - I use all those too) The problem with the old tools is that I don't have admin rights at work so it's not easy to install coreutils. Or even homebrew. I can understand why they did it though. Too many tools these days advocate just piping some curl into a root shell which is pretty insane. Homebrew does this too. | | |
| ▲ | flocked 2 days ago | parent [-] | | Couldn't you simply use macOS without the iCloud features? Which features require iCloud to work? | | |
| ▲ | wkat4242 2 days ago | parent [-] | | You can but there's just not much point anymore. I don't remember all the specifics but every time there was a new macos I could cross most of the new features off. Nope this one requires iCloud or an apple ID. Nope this one only works with other macs or iPhones. Stuff like that. The Mac didn't use to be a walled garden. You can still go outside of their ecosystem (unlike on iOS) but then there's not much point. You're putting a square peg in a round hole. Now, Apple isn't the only one doing this. Microsoft is making it ever harder to use windows without a Microsoft account. That's why I'm gravitating more and more to foss OSes. But there are new problems now, like with Firefox on Linux I constantly get captcha'd. M365 (work) blocks random features or keeps signing me out. My bank complains my system is not 'trusted'. Euh what about trusting your actual customers instead of a mega corp? I don't want my data locked in or monitored by a commercial party. |
|
|
|
|
|
| ▲ | kelnos 2 days ago | parent | prev | next [-] |
| > I'd say that you care about it being UNIX-like, not UNIX®. Right, but I think GP's point is that if Apple didn't feel pressured to get macOS UNIX-certified, then they wouldn't even bother to be UNIX-like. That is, all sorts of UNIX-required command-line tools would start to disappear from the default install, and things like POSIX conformance would take a back-burner, etc. Not sure if I agree with that, but that's what GP seems to be suggesting. > For example, I wish that grep on macOS was closer to GNU grep. This has nothing to do with UNIX conformance; this just comes from macOS's BSD background, which does not use the GNU core utils. If the Linux folks wanted to go through UNIX conformance testing, they wouldn't have to switch away from GNU tools. And macOS could swap out the BSD-sourced tools for GNU tools and still get their UNIX certification. |
|
| ▲ | Avamander 3 days ago | parent | prev | next [-] |
| Built-in grep is thankfully not as odd as the builtin find is. Might be the first one I replace on my systems. |
| |
| ▲ | pseudocomposer 3 days ago | parent [-] | | Given that both grep and find are weird/inconsistent between BSD/GNU versions, and I typically use them piped together for the same things anyway, I’ve found that ripgrep is a nice/faster/universal alternative that is pretty unproblematic to install in whatever environment I want: https://github.com/BurntSushi/ripgrep | | |
|
|
| ▲ | zeckalpha 3 days ago | parent | prev | next [-] |
| Some Linux distros have gotten certified. I assume they have released necessary patches as required by GPL. |
| |
| ▲ | pxc 3 days ago | parent [-] | | I don't think any patches are required. It's literally just a matter of going through the certification dance for the average distro. | | |
| ▲ | mdasen 3 days ago | parent | next [-] | | There are patches required. Many GNU utilities are very close to the UNIX spec, but not quite the UNIX spec - including glibc. But making a Linux distro that is UNIX certified would likely make it a worse Linux distro for most people since it would be less compatible with what everyone is assuming for a Linux distro. A lot of the differences are subtle edge cases, but do you really want that in your distro? It's not just about going through a song-and-dance. It's about making an OS that has different behavior - often very tiny differences, but differences that would make the distro worse for most users. | | |
| ▲ | gsnedders 3 days ago | parent | next [-] | | How close are the GNU coreutils with the POSIXLY_CORRECT environment variable set? I wouldn't be surprised if there were still patches required, but at least on the face of it that should get you most of the way there. | |
| ▲ | skissane 2 days ago | parent | prev [-] | | > It's not just about going through a song-and-dance. It's about making an OS that has different behavior - often very tiny differences, but differences that would make the distro worse for most users. Is this actually true? Can you give a real example of where Huawei EulerOS or Inspur K-UX are inferior to their non-certified base (yet another Red Hat Enterprise Linux clone) due to their UNIX certification? |
| |
| ▲ | swiftcoder 3 days ago | parent | prev [-] | | Don't you have to ship the BSD-compatible versions of grep et al? | | |
| ▲ | pxc 3 days ago | parent [-] | | I'm not sure why you would. I don't think POSIX generally specifies the behavior of command line tools in such a level of detail. FWIW, the regex type used by default by GNU Grep is already POSIX's Basic Regular Expressions. (It also supports POSIX Extended Regular Expressions and PCRE2.) Afaik, EulerOS and other Unix-certified Linux distros just ship the usual GNU userland. | | |
| ▲ | mdasen 3 days ago | parent | next [-] | | The Single UNIX Specification does specify the behavior of many command line tools like ed, grep, awk, etc. OpenBSD sometimes notes where their tools vary from the UNIX spec. It's usually very small ways that don't matter to most people, but it does put them outside of the UNIX spec. | |
| ▲ | swiftcoder 2 days ago | parent | prev [-] | | POSIX != Unix. Unix certification does require POSIX, but it also comes with a bunch more stringent requirements | | |
| ▲ | pxc a day ago | parent [-] | | My mistake! Thanks for informing me :) |
|
|
|
|
|
|
| ▲ | cesaref 3 days ago | parent | prev | next [-] |
| brew install grep? |
| |
| ▲ | latchkey 3 days ago | parent [-] | | brew is such a treasure. | | |
| ▲ | eric_h 3 days ago | parent [-] | | and kind of a counter point to the GGGP's "Unix compliancy isn't what's keeping me on macOS, the Unix tools it has under the hood still is." I certainly replace a large chunk of the "unix tools under the hood" with tools installed via homebrew. |
|
|
|
| ▲ | mechanicum 3 days ago | parent | prev [-] |
| > because macOS is adhering to the UNIX specification Isn’t it rather that Darwin was based on BSD 4.4? I’d imagine GPL 3.0 is a bigger impediment to them ever migrating to GNU tools than any desire to be UNIX certified. |
| |
| ▲ | p_ing 3 days ago | parent [-] | | https://developer.apple.com/library/archive/documentation/Da... It derives from 4.4 BSD but it's more than that. macOS includes a woefully outdated bash 3.2 due to GPL 3.0; they switched to zsh long ago. | | |
| ▲ | mechanicum 2 days ago | parent [-] | | I know. My question is, isn’t the reason the command line tools work the way they do simply that they’re essentially the BSD programs (give or take an Apple patch), with BSD options, not because they needed to work that way for Apple to get the OS certified? Even if macOS wasn’t UNIX-certified, Apple would still be unwilling/unable to include the GNU software due to the license. I can’t see the Apple of today implementing a full suite of non-GNU software but with GNU-style options either. So, POSIX compliant or not, there’s probably no world where `grep -P` works out of the box on a Mac. | | |
| ▲ | p_ing 2 days ago | parent [-] | | Sure, much like ping and tracert are BSD programs on Windows, just with a few different options that are OS-specific. There's no reason to include GNU anywhere. I don't know why Apple would. |
|
|
|
