The 1878 one is fascinating:

When the plug is inserted, the jack "breaks its normal connection." Like they didn't want to leave the audio output like a floating pin to reduce stray voltage?

Scribner calls the switch "spring-jack" after "jack-knife" where the "jack" part of it comes from the name Jack and in the 1300s meant a mechanical device. So the "female" component of the connection was thereby given a "male" name.

  Charles E. Scribner filed a patent in 1878 to facilitate switchboard 
  operation using his spring-jack switch. In it, a conductive lever pushed by a 
  spring is normally connected to one contact. But when a cable with a 
  conductive plug is inserted into a hole and makes contact with that lever, 
  the lever pivots and breaks its normal connection. The receptacle was called 
  a jack-knife because of its resemblance to a pocket clasp-knife. This is said 
  to be the origin of calling the receptacle a jack. Scribner filed a patent in 
  1880 which removes the lever and resembles the modern connector and made 
  improvements to switchboard design in subsequent patents filed in 1882.

  late 14c., jakke "a mechanical device," from the masc. name Jack. The proper 
  name was used in Middle English for "any common fellow," and thereafter 
  extended to various appliances which do the work of common servants (1570s). 

I don't think it was about not wanting to leave the audio input floating. Rather the "normal connection" is that the telephone subscriber is connected directly to the switchboard operator's annunciator (a display panel) so that the subscriber can light up a bulb on the annunciator when that subscriber wishes to ask the operator to reroute that subscriber's connection to another subscriber (instead of to the switchboard operator). This is why the switch ought to act like a double-throw, not just a single-throw switch. I think something along those lines is the reason...

> In a telephone-exchange system the wires of the several subscribers are run into a cen tral office, where, upon request, any wire may be connected with that of any other subscriber.

> In Fig. 4 is shown the cut-out connected with subscriber's wire in and the relay and annunciator P and O, and also, with the operator's telephone J, by means of the plug A, which is provided with a metallic point, and conducting-cord d. The connections are formed as follows: The subscriber S, by throwing on his local battery, sends a current along the wire in through the relay P, which, closing, the annunciator number of S is indicated at O, and the current passes along the Wire H, and thence through the switch to the ground Wire G.