The order of the punctuation descends from the very first typewriters, in the late 19th century:

https://en.wikipedia.org/wiki/File:Remington_2_typewriter_ke...