Physical possession isn't enough to prove someone is legitimately in control of the vehicle, though. If a physical connection under the dash will hand out the key, you can bust the window, and get the key.

Part of the utility of the baked-in manufacturer key is that it is unable to be extracted by thieves.

Clearly it isn't "unable to be extracted" as the other comments here have remarked.

Having to break into the vehicle already raises the bar significantly and makes the security equivalent to a physical lock.