| ▲ | mlmonkey 7 hours ago |
| In the clip with a marble bust dropped on a watermelon, at this point https://imgur.com/a/IHLnkUs the bust should've toppled over, if the physics were properly modeled. |
|
| ▲ | Bjorkbat 4 hours ago | parent | next [-] |
| This is a reflection of a complaint I frequently make about so called "world models". It's more accurate to call them video game models. They don't simulate the world, they simulate video games. For that reason a common issue with them is that the physics are video game physics rather than real world physics. |
| |
| ▲ | rcoveson an hour ago | parent [-] | | I'm sure you're probably right about video games being used in training and, as a result, the models reflecting video game physics: No air resistance, no static/dynamic friction curve, etc. However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine. I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal. |
|
|
| ▲ | extraextra 22 minutes ago | parent | prev | next [-] |
| People are arguing if the marble bust should've toppled over or not, completely missing the fact that marble is brittle and the bust would have surely cracked |
|
| ▲ | sobellian 6 hours ago | parent | prev | next [-] |
| It also failed to properly model the physics of knights riding wyverns into battle. |
| |
| ▲ | modeless 5 hours ago | parent [-] | | Which is fine for a video game. Not at all fine if you want to train robots with it. Trying to combine both use cases into a single model is doomed to failure IMO. | | |
| ▲ | sobellian 4 hours ago | parent | next [-] | | I would not be so opinionated, modern coding agents derive from chatbot technology. Very different problem domain. They share language but language is universal so that doesn't say much. Then they figured out that the pretraining is a good starting point for RLVR. Who's to say a more cinematic imagination can't be a good starting point for a world model? It doesn't need to be the whole solution. | |
| ▲ | drewbeck 4 hours ago | parent | prev [-] | | If they don’t get ahead of this our robots are going to be sitting ducks against knights riding wyverns smh |
|
|
|
| ▲ | StilesCrisis 3 hours ago | parent | prev | next [-] |
| I agree but I feel like the exaggerated physics were intentional and fitting with the video theme. If they had wanted it to topple, they wouldn't have posted the video. |
|
| ▲ | peddling-brink 6 hours ago | parent | prev | next [-] |
| How often do things in the real world not behave exactly as you expected them to? I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events. We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff. |
| |
| ▲ | bigwheels 5 hours ago | parent | next [-] | | Busts rarely hang in midair at a 45° angle. As an aside, why fall into the role of being an apologist for google-hype and Big Tech? | |
| ▲ | mlmonkey 3 hours ago | parent | prev [-] | | > How often do things in the real world not behave exactly as you expected them to? They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works. |
|
|
| ▲ | dsissitka 6 hours ago | parent | prev | next [-] |
| They were demonstrating steering. |
| |
| ▲ | stevemk14ebr 5 hours ago | parent [-] | | So what, the world model failed to model physics. The thing they said it did elsewhere. It's a valid critique |
|
|
| ▲ | spwa4 6 hours ago | parent | prev | next [-] |
| That's the thing about machine learning over time, and the big hole in the whole world model idea. Time and time again we make simulations and train models and we think that loop will work. It never does. And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation. It's where (ex-)"AI luminaries" go to die. It's a way to make glacial, slow progress in the correct direction while never actually getting anywhere. So we get to a point and people keep making complicated simulations of one kind or another ... and then someone (some model) who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point. |
| |
| ▲ | Closi 4 hours ago | parent [-] | | It depends if the goal is a more accurate simulation, or if the goal is a tool for creatives to build games or content. If it's the latter, I don't see a 'hole'. | | |
| ▲ | spwa4 4 hours ago | parent [-] | | My point was that there is also the possibility of AI intelligence improving to the point that there isn't a difference anymore. That whatever beats the next set of benchmarks (which of course won't be the simulator or world model) is so smart it is both a better solution to the problem the world simulator poses and a better world simulator all in one and a better tool for creatives to build games or content. This has already happened to other ML fields. For example, there's not much point anymore to ASR models (just feed audio into LLMs) and the space that TTS models occupy is fast disappearing. Oh and it's not just that Qwen 3.6 is a good ASR in addition to being good at coding mario. Doing things this way, not using an ASR model is fundamentally more expressive. A whole bunch of companies started up on ASR + TTS "towers" (lots of models, Pipecat style) as they're called, and they're dead companies walking. Their tech has been surpassed by just bog-standard generic LLMs in a way that they will never get the upper hand again. A chain of models approach, a separate ASR, will never do this: "you said you agree but I can hear you're sarcastic, so I'll explain again". Also getting a conversational tower to switch languages when the user does that (or when it makes sense in a language course) is not quite impossible but very, very hard. And yet feeding audio directly into Qwen will almost magically solve these problems better than I will ever do it (lightly finetuned). It's fundamentally better in every way (except perhaps speed, of course. Although arguably such a tower needs an LLM too, it'll just have way less tokens) This point has long been passed for image recognition. Feeding a photo or screenshot into a full LLM along with a request on what to do beats the crap out of any OCR, image recognition, ... model every time these days. And now there's also hints of world models just being bypassed. Just giving a finetuned model the last 50 video frames (surprisingly, that fits in context, especially at reduced resolution), movement statistics, motor power, status, ... of a robot and asking what to do actually works (with heavy finetuning, but I think it'll start working without). Well ... AND then distilling that knowledge into a much smaller model because B200 racks are more than a bit to heavy for a logistics robot to carry around. But we'll get over that. | | |
| ▲ | Closi 2 hours ago | parent [-] | | Its easy to see what works in hindsight, but in reality with this sort of research you have to find an approach that’s working and keep pushing it and see where it goes. The results so far from genie have been stunning, and its not obvious to me that another approach is guaranteed to end up better for all use cases. |
|
|
|
|
| ▲ | pcurve 4 hours ago | parent | prev [-] |
| yes. I feel Google's video physics has always been bad. slow, aritifical, unnatural. |