404s were designed for performance, as OP says here, but also to be as simple as possible.

Part of the difficulty was info leakage; early on there was a lot of concern that a 403 would imply the existence of a repo that the org might otherwise not want known publicly. This was a little trickier in practice, really: early versions of the page took slightly different pathways as it went through auth, so based on the duration of the page to render you could make some assumptions about whether the underlying repo existed or not. It was annoying to sort out, so it wasn't touched very often.

Certainly a lot could have been done with 404s over the years; they've only gotten worse. I put ultrathin font weights on the error pages move than a decade ago, when they were en vogue, and it kills me every time to see them still there. And, of course, the parallax effect has been removed, so now it's just sort of a dorky Star Wars or Looney Tunes reference without a lot behind it. Weird.