I really really want to do version control for everything, but most of my data is binary so it really doesn't agree with git; I tried using LFS as well but it didn't work for my specific workflow. Hopefully something less text oriented comes along in the future.

Lore looked interesting for this purpose. https://github.com/EpicGames/lore

▲

catlifeonmars 37 minutes ago | parent | next [-]

Hash the files and commit the hashes. Store the large files somewhere convenient, keyed by the hash. Voila, you’ve invented an LFS.

	▲	globular-toast 11 minutes ago \| parent [-]
		That's just how plain git works anyway. Commits are just a tree of hashes which refer to the blobs in the objects store. Diffing is done on demand. If you can implement diff for binary files, then you've got git.

▲

Borg3 3 hours ago | parent | prev | next [-]

But versionning everything is not really great idea. Probably 90% of your files are static. Archives, Media, LIBs. Not worth versioning. But for that 10%, having any (D)VFS is handy. Thats why I wrote DOT for myself, to handle that niche need to version every document I work with.

Its private for now, but here is printout of help: http://borg.uu3.net/~borg/?dot

▲

denkmoon 3 hours ago | parent | next [-]

why not version control those things? if they're static, there's no changes so version controlling them has little cost. maybe existing tools aren't adequate for this conception of the world, but I don't think that's a good reason not to work towards "version control everything"

	▲	Borg3 7 minutes ago \| parent [-]
		Cost of keeping all data (up to +100%). If they dont change, no point of really versioning them. Backup of those is simple either, whatever syncing software (incremental). Docs that often change, its completly different story. I love VCS like features where I can slam 'dot status' to see whats going on, commit when I feel I need snapshot, sync to whatever device I want. And because I run DVFS I dont even bother with backups because I have multiple copies of repo with entire history.

▲

XorNot 2 hours ago | parent | prev [-]

The bigger issue is that most of the files you're likely to capture in "version everything" are just noise.

Disk space is far from cheap when the value of what's being stored is basically zero.

You can version everything today with something like ZFS snapshoting - and very quickly realize hundreds of gigabytes of noise aren't worth keeping.

▲

2001zhaozhao 2 hours ago | parent | prev | next [-]

It's interesting you brought up Lore. I also wonder whether importing VCS tech from the games industry for versioning general agentic work would be a good idea.

▲

xyzsparetimexyz 2 hours ago | parent | prev [-]

Why not just fork git and make it better at binary files?

▲

eru an hour ago | parent | next [-]

You don't even need to fork it: git has something like a plugin system for exactly this use-cases. (I'm not exactly sure what their terminology is. But they thought about this use-case before.)

▲

DanielHB 2 hours ago | parent | prev [-]

Problem with dealing with binary files with git (or source control in general) is that you want to be able to see diffs, which is very dependent on the type of binary file you have.

I think you can create git plugins to show diffs in different formats for certain file-types (or even open a 3rd party tool to let you visualize it), but it is a lot of work. Most productivity tools don't have anything of the sort.

If you just care about storing the file git LFS + manual text-changelog per binary file works well, but it is annoying to get everyone on your team to work in this workflow (heck, getting everyone and all automation scripts to install LFS is already a pain).

▲

gmueckl an hour ago | parent [-]

The problem is really starting g with a coders mindset that reducing a change to an explicit visual representation of a minimal delta is such a useful thing for everybody. It isn't. For example, for images, an A/B comparison is enough in a lot of cases. There are also no meaningful ways to "merge" changes for a lot of data outside source code, so the insistence that version control can't exist without merging is shortsighted.

Meaningful deltas and meaningful merges are actually a problem set that is totally and utterly disjointed from keeping the underlying change history. Deltas and merges can always be computed independently of the underlying storage format for versioned data. Git could have indicriminately used binary deltas for file storage and would still work the same on every higher layer.

	▲	eru an hour ago \| parent [-]
		In Git the deltas (or diffs) are derived data. They aren't central to what Git is. Natively and conceptually, Git works with repository snapshots.