long s and thorn would like to have a word with you, but they can't because they were removed from the keyboard

In Unicode, that's ſ and þ. Both historical English letters that are no longer used.

"Ye Olde Mill" or whatever archaic silliness you'll find at fairs and whatnot was the result of the printing press dropping þ (as in þe, þ is just th-) and was never supposed to be pronounced with a "y" sound.

"Ye Olde" ye was not the same word as "Hear ye, hear ye!", that ye is a plural 'you' basically the same word as "y'all" and never had a thorn.