This may be a side conversation, but wondered how far away we are to a small LLM being prompted to turn itself into a worm that uses all exploits it knows, and prompts other LLM's when it has access.

I've seen this discussed on HN a few times under articles, and is a risk I've seen communicated in AI safety.

I guess it's being called an AI prompt worm. It's not quite to the stage of sovereign AI where the entire model weights are moved with the 'virus', but still presents a pretty big risk.