I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.

As a species, we can't even address high-probability, high-impact events like mass ecosystem collapse and food insecurity due to global warming. This is just another kind of doom on the pile.

You can imagine whatever you want, but I don’t see why that means we have to take it seriously.

I think for now the probability is limited by their requirement for high resource clouds. They have a limited number of targets. They tend to be targets with comparatively high security monitoring and people will notice if their resource usage suddenly jumps.

...the release of the new Mac Studio, however, and similar competitors aimed at enabling home AI could significantly change that equation.