> Does this somehow prevent you from vendoring everything?

Yes. Because in these environment soon or later you will be shipping libraries and not executable.

Shipping libraries means that your software will need to be integrated in other stacks where you do not control the full dependency tree nor the versions there.

Vendoring dependencies in this situation is the guarantee that you will make the life of your customer miserable by throwing the diamond dependency problem right in their face.

▲

alexvitkov 4 days ago | parent [-]

You're making your customer's life miserable by having dependencies. You're a library, your customer is using you to solve a specific problem. Write the code to solve that and be done with it.

In the game development sphere, there's plenty of giant middleware packages for audio playback, physics engines, renderers, and other problems that are 1000x more complex and more useful than any given npm package, and yet I somehow don't have to "manage a dependency tree" and "resolve peer dependency conflicts" when using them.

▲

zahlman 4 days ago | parent | next [-]

When you're a library, your customer is another developer. By vendoring needlessly, you potentially cause unavoidable bloat in someone else's product. If you interoperate with standard interfaces, your downstream should be able to choose what's on the other end of that interface.

▲

adev_ 4 days ago | parent | prev [-]

> You're making your customer's life miserable by having dependencies. You're a library, your customer is using you to solve a specific problem. Write the code to solve that and be done with it.

And you just don't know what you are talking about.

If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.

This is not up to me to fix or choose the library and the driver version that the customer will use. He will choose the certified version he will ship, he will test my software on it and integrate it.

Vendoring dependency for anything which is not a final product (product as executable) is plain stupid.

It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.

If you want to vendor, do vendor, but stick to executables with well-defined IPC systems.

▲

alexvitkov 4 days ago | parent [-]

> If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.

If you're writing an ADAS system, and you have a "dependency tree" that needs to be "resolved" by a package manager, you should be fired immediately.

Any software that has lives riding on it, if it has dependencies, must be certified against a specific version of them, that should 100% of the time, without exceptions, must be vendored with the software.

> It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.

The exact opposite. Vendoring is the ONLY way to prevent the ABI madness of "v1.3.1 of libfoo exports libfoo_a but not libfoo_b, and v1.3.2 exports libfoo_b but not libfoo_c, and in 1.3.2 libfoo_b takes in a pointer to a struct that has a different layout."

If you MUST have libfoo (which you don't), you link your version of libfoo into your blob and you never expose any libfoo symbols in your library's blob.

▲

seba_dos1 4 days ago | parent [-]

You keep confirming that you don't know what you are talking about.

The vendoring step happens at something like Yocto or equivalent and that's what ends up being certified, not random library repos.

▲

adev_ 4 days ago | parent | next [-]

Yes exactly.

And in addition: Yocto (or equivalent) will also be the one providing you the traceability required to guarantee that what you ship is currently what you certified and not some random garbage compiled in a laptop user directory.

▲

BobbyTables2 4 days ago | parent [-]

Did Yocto ever clean up how they manage the sysroot?

It used to have a really bad design flaw. Example: - building package X explicitly depends on A to be in the sysroot - building package Y explicitly depends on B in the sysroot, but implicitly will use A if present (thanks autoconf!)

In such a situation, building X before Y will result in Y effectively using A&B — perhaps enabling unintended features. Building Y then X would produce a different Y.

Coupled with the parallel build environment, it’s a recipe for highly non deterministic binaries — without even considering reproducibility.

▲

adev_ 4 days ago | parent | next [-]

> Did Yocto ever clean up how they manage the sysroot?

It's better than before but you still need to sandbox manually if you want good reproducibility.

Honestly, for reproducibility alone. There is better than Yocto nowadays. It is hard to beat Nix at this game. Even Bazel based build flows are somewhat better.

But in the embedded world, Yocto is pretty widespread and almost the de-facto norm for Linux embedded.

▲

1718627440 4 days ago | parent | prev [-]

> but implicitly will use A if present (thanks autoconf!)

When you want reproducibility, you need to specify what you want, not let the computer guess. Why can't you use Y/configure --without-A ? In the extreme case you can also version config.status.

	▲	BobbyTables2 3 days ago \| parent [-]
		One certainly can, but such is not default. Things using autotools evolved to be “manual user friendly” in the sense that application features are automatically enabled based on auto detected libraries. But for automated builds, all those smarts get in the way when the build environment is subject to variation. In theory, the Yocto recipe will fully specify the application configuration regardless of how the environment varies… Of course, in theory the most Byzantine build process will always function correctly too!

▲

alexvitkov 4 days ago | parent | prev [-]

"Vendoring step" You cannot make this shit up.

You're providing a library. That library has dependencies (although it shouldn't). You've written that library to work against a specific version of those dependencies. Vendoring these dependencies means shipping them with your library, and not relying on your user, or even worse, their package manager to provide said dependencies.

I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, but if they're not certifying the "random library repos" that are part of your code, I pray I never have to interact with your code.

	▲	seba_dos1 4 days ago \| parent \| next [-]
		> I don't know what industry you work in I dabbled my fingers in enough of them to tame my hubris a bit and learn that various fields have specific needs that end up represented in their processes (and this includes gamedev as well). Highly recommended before commenting any further.
	▲	adev_ 4 days ago \| parent \| prev [-]
		> I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, [..], I pray I never have to interact with your code. You illustrate perfectly the attitude problem of the average "gamedev" here. You do not know shit about the realities and the development practice of an entire domain (here the safety critical domain). But still you brag confidently about how 'My dev practices are better' and affirm without any shame that everybody else in this field that disagree is an idiot. Just to let you know: In the safety critical field, the responsibility of the final certification is on the integrator. That is why we do not want intermediate dependency to randomly vendor and bundle crap we do not have control of. Additionally, it is often that the entire dependency tree (including proprietary third party components like AUTOSAR) are shipped as source available and compiled / assemblied from sources during the integration. Thats why the usage of package manager like Yocto (or equivalent) is widespread in the domain: It allows to precisely track and version what is used an how for analysis and traceability back to the requirements. Additionally again, when the usage of binary dependencies is the only solution available (like for Neutrino QNX and its associated compilers). Any serious certification organism (like the TUV) will mandate to have the exact checksum of each certified binary that you use in your application and a process to track them back to the certification document. This is not something you do by dumping random fu*ng blob in a git repository like you are proposing. You generally do that, again, by using a proper* set of processes and generally a package manager like Yocto or similar. Finally, your comment on "v1.3.1 of libfoo" is completely moronic. You seem to have no idea of the consequence of duplicated symbols in multiples static libraries with vendored dependencies you do not control nor the consequences it can have on functional safety.