With research like this you need to start somewhere. The fact we can get 3d information helps. There are people looking into making splats capture collision information [1].

I have worked on simulation and in my day job do a lot of simulation. While physics is oftem hard and expensive you only need to write the code once.

Assets? You need to comission 3d artists and then spend hours wrangling file formats. Its extremely tedious. If we could take a photo and extract meshes Im sure we'd have a much easier time.

[1] https://trianglesplatting.github.io/