What regulations would you suggest would be the software equivalent of a fire code?

What kind of penalties would apply for not meeting these regulations?

Who would be responsible for enforcement? Do you propose this should apply internationally? Or just to software written in a specific region? Or is the location of where software is hosted (or the headquarters of the company operating the hardware) a better target for legislation?

You're right, there's absolutely no balance you could strike!

Hmm, I wonder how the FDA approves software in a medical devices context. Or if the EU AI act is in any way a precedent.

Oh well, we'll never know.