Unfortunately, Glibc and loader are linked together intrinsically in the Linux ecosystem. So, if you want to be able to launch a program reliably, shipping your own loader and libc might actually be the only way.

Glibc is backward compatible though, they even have symbol versioning to provide multiple versions of the same symbol. So as long as you have the same or a newer version of glibc (than what was built against) you should be good to go. And I don't remember hearing about breakages for this.

Other libraries on the system is far more hit and miss, but glibc is quite compatible.

The other way around is harder though, you can't take a program built against a newer glibc and run it on an older version. So you generally need to spin up a container with some LTS distro and build your binary in it if you want it to be maximally compatible. (However, zig apparently is able to deal with this by shipping a mapping between glibc versions and symbol versions and doing the linking themselves. You can even use zig to link rust code using cargo-zigbuild and get that benefit.)

But if you want to be maximally portable: static linking against musl. Though beware that many things are slower in musl, such as the allocator.