Saved the NVIDIA paper below for reading material, but if I had to guess just as a SWE how I would do it: use hierarchal thinking.

At the top sits a SOTA model capable of seeing snapshots of what's happening and directing intent. Then smaller, more reactive models translate the intent down into actions, before coming to a purely deterministic layer that handles inverse kinematics.

Sorta like how I control my body. I usually don't think about my fingers going to every key as I type - I know what I want to type, then that signal goes to some autonomous "using a keyboard" system, which invokes my muscle memory, which actually moves my muscles.

And if I get burned while I do something, my actual thinking brain doesn't really have time to process it before I'm already moving away; because those smaller systems have much faster - but more rudimentary - feedback loops.