I’m not an expert in BNNs but the prior does not need to be justified in terms of each parameter. Bayesian analysis frequently uses hyperparameters to set the overall tightness or looseness of the parameters (a la Minnesota priors in the econometric literature for example). This would be a similar regularisation intuition as, eg, L1 and L2 regularisation in traditional NN training. This is of course just one example.