Hey, author here! Thanks so much for reading :) I totally agree, the latent captures details useful for prediction, but not necessarily for control, making planning noisy. Still, I was surprised by how well it understood horizontal position after less than two hours of training on one A100!