The ship's computer reacts like an (incredibly good) expert system. It refuses to say anything that isn't verifiably correct, and if you get it into a logical inconsistency it essentially throws and error and doesn't elaborate further.

LLMs don't do have those responses. LLMs are like humans: humans respond, correct or not (with humans hopefully if they only know incorrect stuff the response is to say so, but that is not a guarantee, but they will respond). And if you give a human, or an LLM, a logical inconsistency they will simply proceed, whether they detect the inconsistency or not.

This is how LLMs are designed ... and if you look long enough at the human (or animal) nervous system you will eventually realize that this is also the design of our nervous system: if something goes in, something comes out, guaranteed (in fact that's close to the only guarantee). Very different from expert system's "either something correct (according to the programmed axioms) comes out, or nothing".

Hell, if you then look at insect nervous systems, they are also designed that way. The key is to respond to everything. And while reasonable responses are certainly preferred, an idiotic response is still seen as a lot better than not responding at all by God/Darwin. Exactly like LLMs.

Of course, for humans/insects our bodies are what is called "active stable", like a plane. Meaning our bodies damage themselves, and just outright die without constant neural control. Heartbeat. Breathing. Blood flow regulation. Temperature, probably even the immune system. All need constant neural feedback to stay stable, and if that neural feedback totally disappears, we're dead in 2-10 seconds (heart failure), 2 minutes (breathing), a few hours, maybe a day (temperature regulation). Now we have a distributed nervous system, meaning lots of parts can fail semi-independently, for a short while, but even without your cortex operational you die in a few weeks.

The consequences of this are even explored with "I, Borg" (5x23) and the Datalore episodes after that, where the underlying problem is that the Borg try (and fail) to adapt to and to process a logical inconsistency but Soong's androids have no issue with it (with Data trying to help and his brother Lore trying to control them and Data with it)

> It refuses to say anything that isn't verifiably correct, and if you get it into a logical inconsistency it essentially throws and error and doesn't elaborate further… And if you give a human, or an LLM, a logical inconsistency they will simply proceed, whether they detect the inconsistency or not.

Claude has a `End conversation` tool for that. There was a bit of a fuss over it.

And it'll happily give (apparently) wrong (or bafflingly incomplete/confusing) info.

TNG, S4E5:

Beverly: Computer, what is the nature of the universe?

Computer: The universe is a spheroid region 705 meters in diameter.