The default transport should have accommodated binary data. Whether it’s tensors of image data, audio waveforms, or pre-tokenized NLP workloads it’s just going to hit a wall where JSON-RPC can’t express it uniquely and efficiently.

This is a really, really, good point.

Devil's advocating for conversation's sake: at the end of the day, the user and client app want very little persistent data coming from the server - if nothing else than the client is expecting to store chats as text, with external links or Potemkin placeholders for assets like files.