I feel like submodules are one of Git's most misused features. They're intended as a method of pinning read-only upstream Git dependencies. And when used for that purpose, they're good at what they do.

I think that people mostly get a bad taste in their mouths because they try to use submodules for building multi-repo workspaces where a developer might need to commit in some/all of the repos. They're a bad fit for that problem, but it's mostly because that's not what they were designed to do.

I'd love to see the jj team tackle case #2, personally. I bet they'd do a pretty good job of it.

▲

IshKebab 3 days ago | parent | next [-]

> They're intended as a method of pinning read-only upstream Git dependencies. And when used for that purpose, they're good at what they do.

No they are not. In theory they could be good, but the actual implementation falls down in ... let me count the ways:

1. Transitive dependencies. I sure do love that my company's submodule-based repo has 12 copies of one of our dependencies.

2. Checkouts don't work any more. You can't simply `git switch <branch>`, especially if that other branch has a different set of submodules. And good fucking luck if one branch has a submodule and another branch has a not-submodule in the same location.

3. They don't work with worktrees. In theory... maybe. In practice, the documentation says not to try and in my experience it is right!

4. The submodule URLs are now baked into the git repo. This means you can't mirror the repo anymore easily. I've even had cases where I couldn't even clone the repo because the authors had used `ssh://` URLs which required permissions I didn't have. It's insane that the authentication method gets baked into the repo. I have no idea why they implemented it like this.

5. The tooling experience is just way worse. Want to see a diff of everything you've changed? Well you can't. If you've changed anything in a submodule you just get a hash difference, or at best a list of commits (which is better but it's not even the default!).

Before you instinctively reach for the "well obviously it must work like that" part of your brain, take a moment to think if it should work like this. I can think of several ways to do it better (beyond just making the implementation less buggy).

▲

JoshTriplett 3 days ago | parent | prev | next [-]

> They're intended as a method of pinning read-only upstream Git dependencies.

Except, dependencies are rarely read-only.

	▲	0x6c6f6c a day ago \| parent [-]
		I read that "read-only" as essentially a "lockfile". Which, this is exactly how pinning dependencies works. However if you are mutating your dependencies frequently and want the reference to them to change at the same time, this is the big pain with submodules- gotta do both yourselves. Not to mention there are now logistical problems to answer as this cannot happen atomically in all scenarios, let alone automatically.

▲

hju22_-3 3 days ago | parent | prev [-]

Do you know of something that's good for what people tend to misuse Git submodules for? A multi-repo workspace thingamajig.

▲

gpm 3 days ago | parent [-]

I think what I'd like for that is a mono-repo with better support for subsetting instead of a multi-repo with better support for unioning.

I understand that Google (and may some of the other fangs) have tooling like that internally - but I haven't had the pleasure of using it.

▲

JoshTriplett 3 days ago | parent [-]

That doesn't help if you actually need those repositories to exist separately.

For instance, consider the problem of having an external Open Source project your company maintains or heavily contributes to, and which also has common libraries shared with internal non-public work. Or, the same problem applies if you have a downstream Open Source project that needs to incorporate and vendor an upstream one but easily contribute changes upstream.

Some folks do this by generating the external repo as a filtered version of an internal monorepo, but that's awful in so many ways, and breaks the ability to properly treat the external repo as a first-class thing that can merge PRs normally. It leads to workflows where people submitting PRs feel like their work gets absorbed internally and maybe eventually spit back out the other end, rather than just being merged.

▲

gpm 3 days ago | parent [-]

Is the pain with publishing a subsetted version of the internal monorepo anything but a tooling limitation where things like pushing into that that subsetted version, and merging changes made on the subsetted version, aren't automatic because our tools don't natively understand subsets?

It would require forge integration, but I'd like a world where I could make a PR to `company/open-source-subdir` and the company could merge that PR and that was that without any extra steps because open-source-subdir is just a publicly published subset of the `company` repo.

	▲	JoshTriplett 3 days ago \| parent [-]
		No, it's not just a tooling limitation. Or, at least, not one solvable just by having forges expose public subsets of private repos. That might partially solve the simplest case of `company/open-source-subdir`, if the company trusts the forge tooling to handle subsetting and to not expose more than they want, but it doesn't solve the more general problem. Consider the case where the repositories are owned by different entities, for instance, or have different governance. For instance, Project X needs to vendor Project Y, and have a few downstream patches that they're working on upstreaming. Right now, the two major options include: - Use a submodule. Experience all the pain of submodules. - Use some tooling to fold the repo into your own repo. Extract commits from it when you want to upstream. Experience all the pain of not having the original commits and the ability to easily pull/merge/cherry-pick/rebase, plus pain when you make changes across that repo and other code.