It’s more the old TTY layout which differs somewhat from the modified typewriter layout that’s become standard for computer keyboards. The old Apple ][ keyboard had 1–9 corresponding to the next row in ASCII, shift-0 was @, I think other characters were ±16 based on shift. Early ASCII implementations were often slightly inconsistent but codings were often based on keyboard layouts.

The order of the punctuation descends from the very first typewriters, in the late 19th century:

https://en.wikipedia.org/wiki/File:Remington_2_typewriter_ke...