i've always been under the assumption that "AI Safety" is baked into the training of the models and not a parameter that can be turned up or down. So if someone breaks into Anthropic one night and makes a full copy of Mythos or whatever then that model they copied is fully capable and not lobotomized? That raises questions because, if you believe all the PR, that's equivalent to breaking into a research university and stealing an entire bio/chem weapons research department.

edit: if the above is the case then we should just assume it's already happened because of the value to both goodguys(tm) and badguys(tm).