My understanding is that they only operate in areas they’ve had approved and pre-mapped out for them though. Practically I don’t think the scenario you’ve described would ever happen, but I too would like to see it (I suspect they wouldn’t handle it well).

I live East of DTLA and twice I’ve seen people driving them around, presumably to map those areas out. It’s pretty annoying that I can’t use them for my late night commutes yet, but I’m eagerly looking forward to when I can!