I hadn’t thought about testing the bounds of model safety on comparatively benign requests compared to the type of thing described in frontier model cards.

Capture the flag with AI is more fun than ever, in my opinion anyway. Rarely have we created a technology where sheer perverse enough mentality could break it, but today that door is open. Truly we are wizards whose incantations can cause superhuman intelligence conniption fits. Whether that's a wise idea...

Edit: also I've officially had AI damage hardware with that "wrong format wav" trick. $0.70 speaker was kaput.