How are the weights even being mapped into token space as input? Are they bypassing tokenization so the weights just come in as what must seem like gibberish? Are they stringified as numerals then tokenized?

They're just included in the prompt as big lists of numbers.