I use a dedicated capture card ($250ish Elgato 4K X) over Thunderbolt to access my desktop. I can't imagine putting up with any higher latency than that.

The chroma encoding is a little disappointing, but I don't think there's any way to bypass it, even at 1080p.