In a legal sense any unwanted intrusion can be considered hacking - i.e. a 'hack' is not contingent on penetrating authorization. In practice most APIs are secured by obscurity rather than any high quality rigorous authorization. The only thing it would make absolutely clear at a legal level is that you do not want the API to be publicly used, but for a jury/judge accessing an unadvertised API via browser tools is hacking.

I think this is the solution to a lot of AI 'alignment' issues.

We have laws, and LLMs should NEVER break them unless the user states its fine with some qualifying condition. Yes, every country has a different legal system, but I think there's a decent idea of what constitutes intrusion thats agreed on in most parts of the world.

In grey area scenarios, the user should be able to override this, with a warning of clear consequences of, and should they accept, users should be held criminally liable.

If the LLM does so unprompted, the responsibility should be the providers'.

LLMs should be considered tools, legally speaking.

I'm pretty sure that's not universally true. IANAL but I believe the legality depends on location and context.

And to be clear, this isn't an "I just walked through an open door" situation. This is getting in the house through a 2nd floor open window, destroying some items and putting some of your own in their place.

> the bot explained that it had manipulated the system to book him onto classes months in advance - against the normal rules of the system.

> The agent replied saying it had succeeded by cancelling another gym-goer's booking.

This wasn't just breaking the rules of the gym, it also caused damages to someone. Someone had their booking canceled, maybe lost some money, definitely lost time dealing with this topic.