I fail to understand why we would lack data. Sure, there is limited (historical) text, but if we just open up all available video, and send out interactive robots into the world, we'll drown in data. Then there is simulated data, and tons of sensors that can capture vast amounts of even more data.

Edit: from the source [1], this quote pretty much sums it all up: "Our 2022 paper predicted that high-quality text data would be fully used by 2024, whereas our new results indicate that might not happen until 2028."

[1] https://epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-...

>send out interactive robots into the world

Easier said than done.

Robotics tends to be even more data-constrained than NLP. The real world only runs at 1x speed, and if your robot breaks something it costs real money. Simulators are simplistic compared to reality and take a lot of manual effort to build.

You will always need to make efficient use of the data you have.

Robotics data isn't labeled and if you build a robot, there ain't anyone who has collected data for your particular robot.

There is also the problem that on-device learning is not yet practical.