> We have a private repo with pre-compiled binaries, and a simple Homebrew formula that downloads the utilities and installs them.

Perfectly doable with Nix. Ignore the purists and do the hackiest way that works. It's too bad that tutorials get lost on concepts (which are useful to know but a real turn down) instead of focusing on some hands-on practical how-to.

This should about do it and is really not that different nor difficult than formulas or brew install:

    git init mychannel
    cd mychannel
    
    cat > default.nix <<'NIX'
    {
      pkgs ? import <nixpkgs> { },
    }:
    
    {
      foo = pkgs.callPackage ./pkgs/foo { };
    }
    NIX
    
    mkdir -p pkgs/foo
    cat > pkgs/foo/default.nix <<'NIX'
    { pkgs, stdenv, lib }:
    
    stdenv.mkDerivation {
      pname = "foo";
      version = "1.0";
    
      # if you have something to fetch
      # src = fetchurl {
      #   url = http://example.org/foo-1.2.3.tar.bz2;
      #   # if you don't know the hash, put some lib.fakeSha256 there
      #   sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m";
      # };

      buildInputs = [
        # add any deps
      ];
    
      # this example just builds in place, so skip unpack
      unpackPhase = "true"; # no src attribute
    
      # optional if you just want to copy from your source above
      # build trivial example script in place
      buildPhase = ''
        cat > foo <<'SHELL'
        #!/bin/bash
        echo 'foo!'
        SHELL
        chmod +x foo
      '';
    
      # just copy whatever
      installPhase = ''
        mkdir -p $out/bin
        cp foo $out/bin
      '';
    }
    NIX

    nix-build -A foo -o out/foo  # you should have your build in '/out/foo'
    ./out/foo/bin/foo  # => foo!

    git add .
    git commit -a -m 'init channel'
    git add origin git@github.com:OWNER/mychannel
    git push origin main
    
    nix-channel --add https://github.com/OWNER/mychannel/archive/main.tar.gz mychannel
    nix-channel --update
    
    nix-env -iA mychannel.foo
    foo  # => foo!

Note: if it's a private repo then in /etc/nix/netrc (or ~/.config/nix/netrc for single user installs):

    machine github.com
        password ghp_YOurToKEn

Instantly sounds like a whole reason to use nix and capture those tools as part of the dependency set.