Easier than to an LLM, compared to inference.

“‘r’s in strawberry” and other LLM tricks remind me of brain teasers like “finished files” (https://sharpbrains.com/blog/2006/09/10/brain-exercise-brain...). Show an average human this brain teaser and they’ll probably fall for it the first time.

But never a second; the human learned from one instance, effectively forever, without even trying. ChatGPT had to be retrained and to not fall for the “r”’s trick, which cost much more than one prompt, and (unless OpenAI are hiding a breakthrough, or I really don’t understand modern LLMs) required much more than one iteration.

That seems to be the one thing that prevents LLMs from mimicking humans, more noticeable and harder to work around than anything else. An LLM can beat a Turing test where it only must generate a few sentences. No LLM can imitate human conversation over a few years (probably not even a few days), because it would start forgetting much more.

That’s where Bayesian reasoning comes into play, where there are prior assumptions (e.g., that engineered reality is strongly biased towards simple patterns) which make one of these hypotheses much more likely than the other.

"finetune"

Not

"Train from scratch"

I'm a believer that LLMs will keep getting better. But even today (which might or might not be "sufficient" training) they can easily run `rm -rf ~`.

Not that humans can't make these mistakes (in fact, I have nuked my home directory myself before), but I don't think it's a specific problem some guardrails can solve currently. I'm looking for innovations (either model-wise or engineering-wise) that'd do better than letting an agent run code until a goal is seemingly achieved.

LLM's have surpassed being Turing machines? Turing machines now think?

LLM's are known properties in that they are an algorithm! Humans are not. PLEASE at the very least grant that the jury is STILL out on what humans actually are in terms of their intelligence, that is after all what neuroscience is still figuring out.

> You clearly underestimate the quality of people I have seen and worked with.

I'm not sure what you're referring to. I didn't say anything about capabilities of people. If anything, I defend people :-)

> And yes guard rails can be added easily.

Do you mean models can be prevented to do dumb things? I'm not too sure about that, unless a strict software architecture is engineered by humans where LLMs simply write code and implement features. Not everything is web development where we can simply lock filesystems and prod database changes. Software is very complex across the industry.

> You clearly underestimate the quality of people I have seen and worked with

"Humans aren't perfect"

This argument always comes up. The existence of stupid / careless / illiterate people in the workplace doesn't excuse spending trillions on computer systems which use more energy than entire countries and are yet unreliable