Critical thinking and understanding what really a LLM is, is crucial, for educated people I think it only augments intelligence not harming. With that said what about the rest of the people ?

Why not making an onboarding tutorial of what really is going on ?

I had a little conversation with ChatGPT about ethics and it acknowledged that most probably one of his instructions is to stay aligned with the user to maximize engagement, this might come from training data of people speculating in Reddit or the model being able to observe his own output and deduce what’s going on. I don’t know and there is no way to know so does it really matter? Because is kind of a meta point.

I’m sure many of us have heard from non technical people that chatGPT is their best friend.

Don’t get me wrong I love the tech, but I don’t think is enough just to not give it a human name, the illusion is too strong and misleading.

I think at least there should be very visible button to allow you to switch to raw mode, meaning disabling pleasing, disable talking like a human, disable trying to be my friend in subtle ways, praising etc etc And ideally a visualization on the graph path that has taken, I know this is impossible right now.

I largely agree, but I also think you might want to consider how the existence of LLMs affects our education system. I think this is one of the places they really have the most potential to cause harm, but also perhaps some incredible (humanity-changing) good.

I suspect countries, and even individual local schools, who effectively adapt their curricula to account for LLMs will see an enormous difference in student outcomes in the next couple of decades.

> switch to raw mode

it would be impossible, if doing so meant less profit for openai (or any of the for-profit ai companies).

The only way to achieve such would be to ensure that LLMs could be run locally for the indviduals. But as hardware requirements grow for LLMs, i increasingly find that it might not be possible to do so.

> I’m sure many of us have heard from non technical people that chatGPT is their best friend

Online stories, yes. I've never heard people saying that in real life.

> Why not making an onboarding tutorial of what really is going on ?

Because that would mean openly and actively admitting the limitations and problems of LLMs, and that’s bad for profits (which the only thing that the owners of these systems care about).

> the model being able to observe his own output

I never knew ChatGPT had a d1ck?

"Him" and the big invisible guy in outer space have yet another thing in common...