| ▲ | spwa4 a day ago | |
Please keep in mind that "uncut footage of ..." is the same as those instagram videos where people throw a basketball backwards 20 meters while drinking beer, straight into the goal. It doesn't prove this is average performance. In fact, if it was, that's not what they'd be showing. | ||
| ▲ | porridgeraisin a day ago | parent [-] | |
From my skim of their paper, they have gotten robodojo 14% versus previous sota 9%. So its an improvement, but doesnt seem like a reliable model yet, which is fine. I also notice the model becomes worse with more data on the printer refilling task, so thats either a bad run or its a data point against the robustness. The transfer learning results are meaningful. 75% on new complex tasks from <10 hours of demo is really good. I believe this is a 2x improvement over the SOTA. 10 trials per task is standard for these papers but is pretty weak statistically. Memory tasks its weaker at but they have acknowledged it, and I cant see why this arch would have problems w that in the future. | ||