are there any affordances for prediction or replay? you could try to help network latency by having the server resimulate a small period of time roughly equivalent to the client's network delay - it's not perfect without client-side prediction but it could help

It's possible, but harder than traditional client/server paradigm since the client here is generic so the predictablity should be based on something other than heuristics

I'm thinking about simple ML to predict inputs and feedbacks. Since the amount of data generated in the streaming is massive and well structured, it looks like a possible approach