People are held accountable for accidents (things they did but did not intend to do) all the time. That's why we have different crimes depending on whether or not there was: intent to do cause harm, intent to do a thing that was likely to cause harm, reckless disregard for safety, negligence, etc.

I think someone could argue (and many do) that inserting a piece of computer software (AI) in the middle lowers the level of intent and thus the level of responsibility, but having a particular type of software in the middle absolve one of responsibility seems unworkable and poor public policy.

Also, when people operate equipment know to cause harm when accidents happen, such as cars, they're required to undergo training and testing before being granted a license to operate them, and to have insurance to ensure other people can be made whole in case of damage or injury. And there is an organization and profession that monitors people's use of that equipment, handing out fines or license suspensions based on actual or even just potentially dangerous operation.

Sadly, software "engineers" are not held accountable for their profession like everybody else with a real engineer title. And AI companies are all building Ford Pintos as fast as they can, and competing with each other about who's product makes the biggest explosion.