What do you think an LLM is "supposed" to do?

At the end of the day it's just a big weighted graph traversal. Its output is a result of many combined probabilities. It's not deterministic and even if it was the input range is so massive that it would be impossible to comprehensively test.

You cannot possibly know an LLM will do what you command it to. It's impossible by design. LLMs are inherently unpredictable. They can still be useful, but that unpredictability needs to be accounted for to use them safely.

> LLMs are inherently unpredictable.

Exactly my point.

If the tool is inherently unpredictable AI companies should either be held accountable for any mistakes or should not sell/market their services as if they were infallible.