I like Bayes, but I thought the "surprising" result is that double descent is supposed to prevent nns from overfitting?

Good point. We wrote this pre-double descent, and a massively overparameterized model would make a nice addition to the tutorial as a baseline. However, if you want a rich predictive distribution, it might still make sense to use a Bayesian NN.