but that's exactly the point, this law would be like suing the car manufacturer because the driver hit a human.

Sue the operator. In this case, it seems like OpenAI was testing its own models. The operator and the manufacturer are the same.

In cases where the operator is not the manufacturer, the operator can decide if they should in turn sue to manufacturer because they built faulty machinery.

No? It would be suing the driver rather than blaming the car.

if the car manufacturer tells you you don't have to think while driving anymore then yeah...

more like suing the driver because the (self-driving) car hit a human.

I mean you only need at add the stipulation that there was clear negligence or malice in your instructions to the agent. like we already do with a bunch of other crimes.