Ex-Apple engineer here. This is, for better or worse, just the way Apple approaches this type of problem. From Apple's perspective, this is the way to preserve Finder / Gatekeeper / metadata semantics. It avoids silent data loss when round-tripping archives between Macs. This behavior also maintains consistency with copyfile(3) (as well as the Archive Utility behavior).

Apple treats tar less like “portable Unix interchange” and more like “archive this filesystem object faithfully.” That is very Apple, and very libarchive. ;-)

This is probably going to get worse (as Apple continues to add macOS-specific metadata), so your workaround is very helpful.

I haven't tested it in a while, but at one point, setting the COPYFILE_DISABLE=1 env variable would disable the inclusion of macOS-specific metadata.

▲

Terretta a day ago | parent | next [-]

Arguably, principle of least surprise is very Apple.

If I point "tape archive" at a file system, I want that file system archived to tape. And so, tar does.

If I don't, well, that's a fine option, and there's a fine option for that.

So it's less of a "workaround" or something that "gets worse", than, "No, I don't really want a tape archive of this filesystem, only of some of it." And that's supported.

That said, never seeing another .DS_Store should be a system-wide option!

▲

JoshTriplett 4 hours ago | parent | next [-]

> Arguably, principle of least surprise is very Apple.

Principle of least surprise is good engineering practice. The question is always whose surprise. Someone who expects tar to behave like other UNIX systems is going to be surprised by this. Someone who expects tar on Apple to have perfect fidelity would be surprised by not-this.

I increasingly feel like build systems should never be relying on any "native" utilities from the host system, and should instead be bringing them in via dependencies. You can't have this problem if your packaging system pulls in a specific portable `tar` library.

▲

crazygringo an hour ago | parent | next [-]

> The question is always whose surprise.

I think that the surprise of more data than expected is more desirable than the surprise of data loss. So in this case, it seems like the safe choice.

▲

amarant 3 hours ago | parent | prev | next [-]

Nixos has a pretty solid solution to this issue: key your dependencies with checksums of the content. That way you get the best of both worlds: you always get the exact version you want, and you can share a copy of that exact version with other software that wants to use that exact version too!

	▲	altairprime an hour ago \| parent \| next [-]
		Are the xattr / chattr / umask checksums rolled into the main data fork content or are they hashed separately (or not at all)?
	▲	JoshTriplett 3 hours ago \| parent \| prev [-]
		Yeah, Nix-like distributions (e.g. guix, lix) do for Linux systems what some language package managers (e.g. cargo) do for individual projects.

▲

Joker_vD 3 hours ago | parent | prev [-]

> I increasingly feel like build systems should never be relying on any "native" utilities from the host system, and should instead be bringing them in via dependencies.

Well, you see, while this, frankly, applies not just to build systems but to most of software, the consensus in the community of distro-maintainers is that it's actually wrong: you should use your system's package manager, and tools it can install, and let it fiddle with the ambient environment and give you that delicious "path dependency". And if your distro's packaging environment doesn't allow to do the things you need (e.g. being able to install both mongodb 3.8 and mongodb 5.0, ideally at the same time, but okay, I can keep running apt remove/install over and over, but I do need to check if my app correctly handled the wire protocol changes), well, that's your problem for desiring strange things.

▲

taftster 5 hours ago | parent | prev [-]

> That said, never seeing another .DS_Store should be a system-wide option!

Yes please.

▲

ryandrake 4 hours ago | parent [-]

.DS_Store, .fseventsd, .Spotlight-V100, .Trashes, and ._this and ._that

These can all die in a fire too, as far as I am concerned. macOS loves to treat the user's filesystem as its own personal garbage dump.

▲

gerdesj 4 hours ago | parent | next [-]

thumbs.db and those weird MS alternative stream files for recording origination.

filesystem attributes are for decorating files with meaning. Anything else that attempts to use filesystems in "interesting" ways is silly.

Apple and MS really ought to consider why they do this sort of fragile, idiosyncratic nonsense.

▲

Joker_vD 3 hours ago | parent [-]

But... thumbs.db is precisely not an "attempt to use filesystems in "interesting" ways" — it's literally a just hidden file with previews stored in it. Storing the preview in the alternative stream of the file with the picture itself would be "an interesting way".

	▲	mook 22 minutes ago \| parent \| next [-]
		In the particular case of thumbs.db, storing them in NTFS alternate data streams would have been a good idea; they're essentially caches for the main data stream, so if they fail to copy to different filesystems it's totally fine. Of course, that wasn't viable because 1) IIRC that was before the widespread adoption of NTFS, and 2) they probably still need the cache somewhere for vFAT USB drives.
	▲	kstrauser 3 hours ago \| parent \| prev [-]
		Agreed. Where else would you put that stuff? It’s gotta go somewhere, and this is the least surprising place IMO. Anywhere else would have to be a parallel store that follows filesystem mounts and unmounts, renaming directories, etc so that it alway perfectly mirrors the thing it’s configuring.

▲

emmelaich 3 hours ago | parent | prev [-]

OTOH, If you want the information contained in those files, where else would you save it?

	▲	ajxs 2 hours ago \| parent [-]
		To me it seems more sensible to store information relevant only to this OS in a specific cache somewhere within that OS. It would even make cache-like functionality such as evicting old entries super easy.

▲

matheusmoreira 4 hours ago | parent | prev | next [-]

It's a good attitude to have, in my opinion. Portability is overrated. Linux developers should be doing a lot more of this. We should be making everything work better for us without caring how it's going to impact other irrelevant platforms. Let the people who actually care about those platforms worry about such things.

▲

cozzyd 4 hours ago | parent | next [-]

It would at least be nice if there was a way to keep apple users from shitting all over the filesystem with remote mounts and ds_store files. Perhaps by automatically unmounting if one is detected.

	▲	bombcar 3 hours ago \| parent [-]
		At least if you're using ZFS as the backing store and Samba, you can set vfs objects = catia fruit streams_xattr and similar config options to use extended attributes.

▲

messe 33 minutes ago | parent | prev [-]

> Linux developers should be doing a lot more of this. We should be making everything work better for us without caring how it's going to impact other irrelevant platforms

Linux developers already do. Using a BSD can already be a pain in the arse, thanks to (often poorly thought out) Linux-isms cropping up everywhere.

▲

jmclnx 4 hours ago | parent | prev [-]

To me, the big question is why Apple needs all these file attribute ? If the files are extracted OK, just ignore the errors :)

	▲	bombcar 3 hours ago \| parent [-]
		Apple has had multiple streams per file since the very beginning, and it can store useful and necessary information (the latter is quite rare now, as most things have sane defaults, but losing the extended attributes can lose things that can be annoying).