> What sort of subject characterizes a style of society in which everyone is theoretically as ready to help you as the question « May I help you ? » implies ? It’s the question your seat-mate immediately asks you when you take a plane – an American plane, that is, with an American seat-mate. The last time I flew from Paris to New-York, looking very tired for personal reasons, my seat-mate, like a mother bird, literally put food into my mouth throughout the trip. He took bits of meat from his own plate and slipped them between my lips ! What is the nature of this subject, then, which is based on this first principle, and which, on the other hand, makes it impossible to get service ? Such then is my question, and I believe, as regards my story, that it is here, on the level of this gap – which does not fit into intra or inter or extrasubjectivity – that the question of the subject must be posed

Lacan

https://ecole-lacanienne.net/wp-content/uploads/2016/04/1966...

> inability to say no

One word that few wealthy people ever hear, is “No.” It has a pretty significant effect on their worldview. Even the most reasonable, well-informed, well-intentioned, wealthy folks can have their thinking affected.

When every silly, should-be-smothered-in-the-crib idea gets enthusiastically endorsed by your entourage, it’s easy to lose the ability to self-regulate. I’ve watched it happen, numerous times, as acquaintances and friends have become more successful.

Obsequious LLMs are leveling the field. Less wealthy folks now have the chance to lose their ability to self-regulate, just like rich folks.

Feels exactly the way my 2 year old behaves.

How does a fan work: Swish swish swish swish

Where do these clouds come from: Points to a far away direction in the sky and says they come from there.

Who does all these roads, trees and environment belong to? It all belongs to me. Obviously.

They have an answer ready for every question you throw at them and they will answer it with absolute certainty. I will have to wait and see at what age does the concept of "I don't know" develop.

They definitely say no. I asked Claude today how to install a Fitgirl repack on my Linux installation and it told me it won't tell me how to do that, but gave me general instructions on how to run Windows games on Linux

It's interesting because i'm kicking the tires on the top tier stuff for a month (because it's expensive as fuck but I need to know where the ceiling is).

I have actually gotten "hey i don't think this is a good idea, here's why" as feedback from at least Opus. It WILL still do it if I just demand stupidity (and hell i've been right, which is another topic entirely) but it has given me more confidence this can be a useful tool in the right spots.

That said I probably don't need the top tiers (metrics at least confirm that) and I'm guessing that's specifically because I was working in coding. Most were worded in a "is this a good idea" framing which probably helped, but at least once I said 'lets use this library/method" and it gave a decent argument on why that was basically redundant without prompting.

I still struggle to see the price point panning out.

I think that is an issue. Also, the ability to quickly build any idea might not be such a great thing. Not only do we probably all prefer things of quality that were made with care but some ideas also just shouldn't be built.

Over the last 3 years I've seen projects where I thought, pretty obviously that's a bad idea. But, because LLMs don't say no and can just be pushed to build it anyway, the people building them might never learn that or learn why.

It's nice to be able to have a quick prototype or mvp. But if we never hit friction or something not working out, we never learn or have to come up with a creative solution.

Now, the LLM might seem incredibly intelligent (relatively speaking) and also creative but let's not forget that all is based on its training data. I simply don't believe it can ever be omniscient or that the companies training it are careful enough when doing so.

Two angles for thought. 1) If an LLM says, "I don't know" its underlying data said it as well. 2) Many system prompts use something along the lines of, "you are a helpful assistant" which may be counter to stating something like, "I don't know."/has a low likelihood of appearing after the system prompt.

Regardless the frontier model considered, we're certainly in a "know-it-all" era.

Maybe the sort of introspective prompt-response is difficult to implement when it could limit/contaminate future improvement. I speculate it's easier to correct a "confidently incorrect" model than a "I don't know" model. A confidently incorrect model response >=0% correct over a 0% correct (I don't know).

Maybe "I don't know" is a model cognito hazard of sorts when many queries can lead back to the response. Maybe future Turing tests will use this sort of introspective evaluation. Who knows? I don't :)

There is the art of saying no: https://dl.acm.org/doi/10.5555/3737916.3739489

It is possible to create (subjective) reasoning traces like https://huggingface.co/datasets/Bachstelze/ethical_coconot_6...

And train or adapt a model to it: https://huggingface.co/Bachstelze/olmo-7b-ethical-reasoning-...

This is just a little proof of concept, though it is maybe the direction you are looking for?!

Hmm. Using Claude, it will tell me words to the effect of "this won't work, here's why, want me to try this instead?" That's a polite "no" in my book.

I’m using ChatGPT and started to notice that lately it answers my prompts starting with „Yes” even if my question was open. As if the first token gets injected and the LLM is left to finish the response in a sensible way, often ending up with some form of „Yes, but not really”.

Agreed, it is abolutely an issue. It is quite difficult to find an optimal solution to some problem when every considered new idea is ”definitely the right shape”.

I've been wondering whether that is a feature of the foundation model or whatever finetuning they do on top. I remember this from the earliest versions of (pre Chat-) GPT I've been using, which would suggest it's a feature of the foundation model. But I don't really understand why. Something that's been trained on StackOverflow and BB forums, among other things, should have seen a ton of examples of answer refusals.

This is an active area of research to inject humility into llms in order to create some kind of knowledge boundary. You can look this paper from nouswise https://arxiv.org/html/2604.17843v1 and the product build on top it to try the humility.

My experience with opus/fable is somewhat different - they CAN reject something, but it has to be phrased very deliberately.

It's a bit annoying honestly. I'm always very careful to be incredibly neutral on the direction of a request, and I'd say 10% are knocked back on on valid grounds, which is great.

On occasion I accidentally say "let's do this" and it blindly goes and does it - I spent 2 days undoing something I built that was just a truly awful idea, because I accidentally phrased it lightly as a request, not a discussion!

You should not need it to say no.

You can get just as good information by asking its thoughts for and against some issue.

That doesn't force it to stop being sycophantic; in fact it actually exploits sycophancy to give you what you want.

> but like to have a subjective reason not to do something

You're asking a lot from extremely fancy auto complete...

The model providers could randomize the system prompt to make it say no 2.36% of the time, automatically tuned up or down depending on user feedback.

Opus 5 tells me no all the time (code cli and web). It's reasons are usually pretty well argued though.

Opus 4.7 would flat out refuse to follow instructions to the point where it was just too frustrating to use.

I've had refusals for GPT 5.5 before as well (not because of a ToS violation, it just refused to take conversations in directions it felt were in bad taste)

You've hit on an important insight.

LLMs are next token predictors. They predict the next most likely token given the previous context window of N tokens.

This means not giving an answer is not a technically possible option. Best you can do is force it to output a magic "stop speaking" token, but this is a vastly different training problem than getting it to not know something.

People naively expect LLM outputs to have some sort of confidence value when predicting, but the technology just doesn't work that way.

Sometimes I get them to say no to me by taking absurd counter positions on purpose, just so I can test their limits.

Is that really the biggest problem? Or is the bigger problem that, in this case, they will remain stuck at the fifth grade level forever? And does not that also explain why the promises of AGI are chimeric, and why the collapse has already started, given that there is essentially no data left that has not already been siphoned up?

Yes, we have all seen the math theorems being proven... just higher processing power at the service of the same algorithmic and conceptual patterns? [1]

I am sure the next version of Opus or GPT, if given only fifth grade knowledge, will somehow be able to build all the mathematics necessary to solve the problem on its own... right? Right?

[1] - "AI Isn’t Outthinking Mathematicians. It’s Out-Remembering Them" - https://davidepiffer.com/p/ai-isnt-outthinking-mathematician...

After an answer, try asking it why, over and over. It's a machine to give answers, not explanations. A magic 8 ball. https://news.ycombinator.com/item?id=49307396

People smarter than me have a habit of getting me to see things without telling me. They ask the right questions.

LLMs, incidentally, respond in a similar pattern in my experience.

It can, just use Grok.

I suppose the LLM doesn't know it's limited in its knowledge, maybe? That others know more.

Yeah, the filtering process probably wasn't particularly robust. The Schrodinger's cat example ("It's a cat that has been misbehavin'!") sounds like... a joke?

Looking at the paper, it looks like they started with FineWeb-Edu, then filtered it based on an "age of word acquisition" dataset, with word frequency used as a proxy for values not in the dataset. They "only discard samples in which more than 5% of the words exceed the target age of 12." Maybe 5% was too high? They also filtered out beyond K-5 math symbols, like sigma. Then they trained a classifier to do more filtering.

And they tested it on two grade-level benchmarks, and it only got 0-3% correct on the beyond k-5 boundary, while also decreasing in performance on the k-5 boundary (which they say is an acceptable tradeoff, since they were trying to get a sharp cutoff). So presumably since they got good results from the benchmark they stopped.

Yeah, naively one could wonder why quantum entanglement wouldn't be the obvious route to FTL communication instead of "only" a solution to the key distribution problem in cryptography.

What you said is why.

There are science books written for curious children that explain this kind of thing. I remember reading them.

I was definitely taught this in those terms and at that age.

Perhaps the unexpected response comes from its recall ability. It’s not the personality of a child, just the material a child is exposed to.

Isn’t that also wrong? From what I remember it’s the blue wavelengths of visible light that are scattered and make us perceive the sky as blue. UV may well be scattered too but we can’t see that, right? Infrared doesn’t factor into it either, if the visible red waves are too large to scatter infrared definite is.

I’ve read random kid science books to my son with this info in. The problem is the AI has perfect recall.

Choose your explanation in comics form:

- https://xkcd.com/1145/

- https://www.exocomics.com/743/

Yes wouldn't a 5 years old say "I don't know", or start thinking about square shaped trees?, hmm.

> Does anyone know about this technique? How does that work?

The term you are looking for is Curriculum Learning. There are several papers exploring this. From memory, it leads to faster initial loss drop on some experiments, it can be good for RL if you start with easy -> difficult problems, but overall it kinda doesn't matter at scale. (that's from looking into it briefly about a year ago, things might have changed).

It confirms what we knew. The stochastic parrot regurgitates what it was fed. There's no intelligence.

How would you solve this problem given the training set and the existing training methods?

How would a human (maybe a 5th grader?) solve this?

It would lose knowledge about existing subjects unless it’s continually retrained on those too. It could help inform the next training dataset though.

Or with a different kind of early: https://www.smbc-comics.com/comic/rise-of-the-machines

[February 2020]

> Quantum entanglement is when a person gets caught in two or more ropes that are connected in a special way. This can happen if the ropes cross each other or if one rope wraps around the other.

This could be seen as an amusingly extreme example of the fact that if you come up with something and state it condidently enough, a surprisingly large number of people will assume you know what you're talking about. Presumably, though, you just mistook the unfiltered (trained on the full data) response for the "Little Learner" one.

You didn’t even read the example you’re referencing.