So do Gentoo and Nix, yet they have packaging separate from the source code. The source is fetched, but the build is sandboxed from the network during the configure, build and install phases. So it's technically possible.