You don't replicate all the problems of raw pointers. You can't have type confusion or undefined behaviour. It's totally memory safe. That's a pretty huge difference.

But I agree, it does give up some of the benefits of using native references.

True. In some ways, that's worse. Instead of crashes, you get something working on the wrong data.

I had one bug in a renderer where I'd see object shadows moving across a 3D scene, but not the object casting the shadow. Didn't crash. That was hard to find. A dead index was still in use, and it pointed to something valid enough to be drawn.