Remix.run Logo
imtringued 3 hours ago

I'm honestly more concerned with your lack of understanding of these topics.

There are two main ways to accomplish what the kung-fu robot does.

First you train a reinforcement learning policy for balancing and walking and a bunch of dynamic movements, then you record the movement you want to perform using motion capture, then you play back the recorded trajectory.

Second, you train a reinforcement learning policy for balancing and walking, but also bake in the recorded movement into the policy directly.

Okay, I lied. There is also a third way. You can use model predictive control and build a balancing objective by hand and then replay the recorded trajectory, but I think this method won't be as successful for the shown choreography however it's what Boston dynamics did for a long time.

In both cases you will still be limited to a pre-recorded task description. Is this really that hard to understand? Do you really think someone taught the robot in Chinese language and by performing the movement in front of the camera of the humanoid how to perform the choreography like a real human or that the robot came up with the choreography on its own? Because that's the conclusion you have to draw if you deny the two methods I described above.

throwawayffffas 2 hours ago | parent [-]

As far as I understand the state of the art as of 2-3 years ago, there is no reinforcement learning at all at any point. At least not in the dynamics.

What you do is you map the dynamics of your system, and solve them, that solution is a program that can produce torque inputs in joints to move the system in the way that you want.

You then create a sequence of desirable intermediate and end states. The program then does it best to achieve these.

The difference between atlas and the kawasaki robots, is that to achieve those states, the kawasaki robots use a program that attempts to stop all inertial rotations and movements in order to maintain full control of it's movements at all times.

While atlas and the chinese robots leverage the inertia and gravity to achieve their movements, again you do that by solving a large set of equations, no ML required.

The GP described a system of prerecorded motions, like a video game animation, if you try to do that, and have no controller to adjust to the real time environment, you are just going to tip over and continue doing the prerecorded motions. We saw that with the Russian robot last year.

You can use a real human that does the choreography as a way for capturing the desired intermediate states that is the step that might require ML.

scotty79 2 hours ago | parent [-]

> As far as I understand the state of the art as of 2-3 years ago, there is no reinforcement learning at all at any point. At least not in the dynamics.

I think this might no longer be true. I don't think this years dance routine would have been possible without RL given how crappy robots were 2-3 years ago.