Remix.run Logo
tarxzvf 3 hours ago

Models hallucinate plausible answers to why they did things. It might be true and it might be complete fiction.

taeric 3 hours ago | parent | next [-]

I'm growing increasingly confident that this is how people often work, as well.

lathiat 2 hours ago | parent | next [-]

“Post Hoc Rationalisation”

https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...

supern0va an hour ago | parent | next [-]

Exactly. I am becoming increasingly convinced that this is actually just a part of how intelligence/cognition works.

astrobe_ 8 minutes ago | parent [-]

But is it really what we want, machines with the same defects as humans? I don't want a pocket calculator that make mistakes "sometimes" so I have to double-check the results, I want a pocket calculator that works (to those who want to argue that pocket calculators don't give the correct result for (1/3)*3: STFU).

BurningFrog 2 hours ago | parent | prev [-]

I learned this from "The Elephant in the Brain", which I strongly recommend: https://amzn.to/4iSyLX8

bmacho 2 hours ago | parent | prev | next [-]

No. People have an inner monologue, partial results and ideas and they remember that.

If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random".

None of these cases are similar to how AI works.

satvikpendem 27 minutes ago | parent | next [-]

> People have an inner monologue

Perhaps up to 50% of people actually don't have an inner monologue, much like many have aphantasia where they can't actually see anything in their mind either.

archagon 9 minutes ago | parent [-]

That’s not how that works. I don’t usually have an inner monologue either, but I do have an abstract stream of thought. It’s not as if I am always acting on instinct.

supern0va an hour ago | parent | prev | next [-]

>No. People have an inner monologue, partial results and ideas and they remember that.

Yes, but the vast, vast majority of decisions you make either don't take place via an inner monologue, or include details that were not actively/consciously "thought" and reasoned with in your inner monologue.

And yet, when asked why you did something, you're not likely to respond "sorry, that decision was made subconsciously". Instead, you use your inner monologue to try to backfill in a reason why. That reason may be correct, or it may not be. You don't actually know, since you have new data that may be updating your own internal state as you try to rationalize it after the fact.

an hour ago | parent [-]
[deleted]
yawnxyz 2 hours ago | parent | prev | next [-]

I think this is true with some people, but I don't think this holds true for some (or even most) people across the US (at least not all the folks I've worked with)

croo 2 hours ago | parent | prev [-]

Actually split brain experiments tells a different story. The left hemisphere actively confabulates, inventing plausible explanations for actions it didn’t initiate, suggesting that much of human self-narrative may be post-hoc storytelling.

bmacho 13 minutes ago | parent | next [-]

> The left hemisphere actively confabulates, inventing plausible explanations for actions it didn’t initiate, suggesting that much of human self-narrative may be post-hoc storytelling.

Sounds like the left hemisphere usually uses something from the right hemisphere to answer those questions and it can't do that if it's been cut off?

We know that our brains are capable of hallucinating due to substances (drugs), being asleep, brain damage (including split brain), hypnosis, etc. Just as RAM damage make your computer do weird shit. That doesn't mean it operates that way normally.

You can't just remove a huge part of a system and then assume that the whole system behaves the same.

satvikpendem 27 minutes ago | parent | prev | next [-]

Indeed. People literally make stuff up when their corpus callosum is severed.

bmacho 6 minutes ago | parent [-]

Aand what if their corpus callosum isn't severed?

25 minutes ago | parent | prev [-]
[deleted]
nemomarx 2 hours ago | parent | prev | next [-]

I kind of want my computer systems to be more reliable and predictable than paying an intern to manage something and asking why they messed up

jaggederest 2 hours ago | parent [-]

At this point it very dramatically is more reliable and predictable than any human I've worked with.

Do you know anyone who actually reads and adheres closely to all of the documentation every time it's changed?

wccrawford 2 hours ago | parent | next [-]

That was my experience with Claude when my vibe-coded project was small.

But now that I've been working on it a month and there's a lot of documentation, it's pretty clearly ignoring parts of the documentation and parts of the code. It will come up with some ridiculous statement about how something works, and I'll challenge it, and it'll admit I'm right.

It definitely reads more documentation than any programmer I've ever worked with (myself included) but because it doesn't have a memory other than the documentation, it still makes mistakes like that.

I haven't turned on "memory" or tried it with Codex, so I don't know how that'll change soon, though.

jaggederest an hour ago | parent [-]

Yeah the biggest task these days that I do manually is curating the documentation. AGENTS.md in every major directory, and a variety of reference docs that are explicitly referenced in those files.

    # See DOC-ITEM-NAME

    DOC-ITEM-NAME.md
    When referencing documents, always use the exact syntax See <TAG> - this is enforced by a lint on precommit
And those doc items are basically all of the values, architectural, strategic, and tactical items. It's a poor man's in-repo RAG but it's shockingly effective, especially if you keep them small. I may migrate some/all of them to skills over time, but I usually update them biweekly, and I only allow agents to make small edits or propose new notes. And typically I go through and delete or curate any agent edits before merge.

Depending on language I've seen this scale past multiple millions of lines of code, as long as you pair it with all of the linting and tooling that you can possibly build.

bluefirebrand 2 hours ago | parent | prev [-]

I don't know anyone who has that kind of time, no

skinfaxi 2 hours ago | parent | prev | next [-]

People don't make rational decisions that make rationalized decisions. Is there any thought to pulling your hand off a hot surface?

voakbasda 2 hours ago | parent | prev [-]

Every time I hear someone complain about hallucinations, I laugh at the total lack of self awareness about our species. Humans are just as bad (now, probably worse) at telling the truth, whether due to intention or poor memory.

theluketaylor 3 hours ago | parent | prev | next [-]

True, but even a hallucinated explanation of where things went wrong added to the context can force the model down a better path over the next few inputs.

Kiro 2 hours ago | parent | prev | next [-]

The point of the parent post is that the explanation shows they made the error themselves, so it's immediately validated.

embedding-shape 2 hours ago | parent | prev [-]

You can also literally tell them: "Here is your session ID: $ID, lookup the .jsonl session, trace exactly why this decision was being made, present evidence and concrete proof, no guessing or assumptions" and you'll get an evidence-based report without guesses.

zamadatix 2 hours ago | parent [-]

It can always hallucinate said report results/evidence/proof just the same. This approach tends to help reduce the hallucination rate though.

You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.

bmacho 2 hours ago | parent [-]

Ask it to build you a simple and deterministic citation checking extension to your IDE that puts source in meta to citations. E.g. color citations green/red depending if they are valid.

zamadatix 2 hours ago | parent [-]

Sure, you can always validate what it's saying yourself at any point & you can have it try to make manual verification an easier process to complete via methods such as the above.