You can allow-list certain IP address ranges to not need the local network permission and I believe that should stop the popup from appearing (but I'm not sure). See https://developer.apple.com/documentation/technotes/tn3179-u... for more information, TL;DR:

  sudo defaults write com.apple.network.local-network AllowedEthernetLocalNetworkAddresses -array "169.254.0.0/16"
  sudo defaults write com.apple.network.local-network AllowedWiFiLocalNetworkAddresses -array "169.254.0.0/16"