This is why it's better to call the underlying technology transformers rather than 'language model'. It can make a model from anything you can digitize and is informational (random noise would not be informational for example). It's the relations between the bits of data that matters.