Why wouldn't the system prompt be controlled on the server side of the API? I agree with https://news.ycombinator.com/item?id=47010577 ; I think results like this more likely come from "roleplaying" (lightweight jailbreaking).