A little while ago, I chucked together a comparison game on image file formats (incl a stab at av2 images), encoders, parameters to back out preferences relative to ssimulacra2 and butteraugli: https://aadhar.sh/pixel-peeper

It does seem like AVIFs win on small images, and JXL gives you progressive rendering, easy upgradability on JPEGs, and better lossless and high bitrate images.

Note that progressive rendering is now supported by avif, and the comparison in the OPs post makes is appear quite a lot better.

It does sound like the remaining areas jpegxl is better at are almost all non-web use cases.