If you had made a tool that gave gpt-3 the ability to run arbitrary commands on your production systems you could have seen things go badly.

Good news! Today's SOTA models can also make things go badly.

	▲	jrflowers a day ago \| parent [-]
		Yep. I don’t see how that metric indicates how… strong(?) a language model is.