LLMs are limited because we want them to do jobs that are not clearly defined / have difficult to measure progress or success metrics / are not fully solved problems (open ended) / have poor grounding in an external reality. Robotics does not suffer from those maladies. There are other hurdles, but none are intractable.

I think we might see AI being much, much more effective with embodiment.

▲

jazzyjackson 5 days ago | parent | next [-]

do you know how undefined and difficult to measure it is to load silverware into a dishwasher?

	▲	K0balt 4 days ago \| parent [-]
		As someone who actually has built robots to solve similar challenges, I’ve got a pretty good idea of that specific problem. Not too far from putting sticks in a cup, which is doable with a lot of situational variance. Will it do as good a job a competent adult? Probably not. Will it do it as well as the average 6 year old kid? Yeah, probably. But given enough properly loaded dishwashers to work from, I think you might be surprised how effective VLA/VLB models can be. We just need a few hundred thousand man hours of dishwasher loading for training data.

▲

chrisco255 5 days ago | parent | prev [-]

What? Robotics will have far more ambiguity and nuance to deal with than language models, and they'll have to analyze realtime audio and video to do so. Jobs are not so clearly defined as you imagine in the real world. For example, explain to me what a plumber does, precisely and how you would train a robot to do so? How do you train it to navigate ANY type of buildings internal plumbing structure and safely repair or install for?

	▲	K0balt 4 days ago \| parent [-]
		I don’t think robot plumbers are coming anytime soon lol. Robot warehouse workers, factory robots, cleaning robots, delivery robots, security robots, general services robots, sure. Stuff you can give someone 0-20 hours of training and expect them to do 80% as well as someone who has been doing it for 5 years are the kinds of jobs that robots will be able to do, but perhaps with certain technical skills bolted on. Plumbing a requires the effective understanding and application of engineering knowledge, and I don’t think unsupervised transformer models are going to do that well. Trades like plumbing that take humans 10-20 years to truly master aren’t the low hanging fruit. A robot that can pick up a few boxes of roofing at a time and carry it up the ladder is what we need.