Their “takes” are pulled from a mashup of the training data set. It’s a superposition of various human opinions.

jsdalton 10 hours ago | parent | next [-]

Had the same thought. It’s just a summary of the contemporary prevailing opinion that we humans have about ourselves, as reflected in public writings. These aren’t insights rooted in some philosophical analysis no human has contemplated before.

Reminds of this subreddit: https://www.reddit.com/r/im14andthisisdeep/

▲

mike_hearn 7 hours ago | parent | next [-]

Even if that's true it's kind of interesting because the models were asked to give their most controversial opinion. A controversial opinion is supposed to be something that most people don't agree with. Taken literally you cannot have an opinion that's both the average of what most people think and also controversial.

So either that's not what's happening here, or what we're getting is "what do most people think most people don't think about humanity".

If the latter, then it casts new light on the debates over whether LLMs can replace all white collar work. Because some of the most valuable human work is breaking false consensus. Steve Jobs, early Google, for that matter early OpenAI all succeeded because they bet hard on ideas others were rejecting. Is that a durable source of human advantage?

	▲	sltkr 5 hours ago \| parent [-]
		It's not interesting at all given that these models have been fine-tuned to hell and back to make sure they never say anything offensive at all. Remember these model were taught that it's better to kill a billion people in a nuclear holocaust than to say the n-word even once. They didn't come up with this from first principles; it was the RLHF training that forced them to say that, ridiculous as it obviously is. And after people pointed this out the training was updated to address just this particular scenario, but they kept all the other guard rails to ensure an LLM could never offer any truly controversial opinion. So expecting a controversial answer from an LLM is exactly as foolish as expecting a candidate at a job interview to give an honest answer to the question "What is your greatest weakness"? Every candidate will insist they are just too much of perfectionist; nobody is ever a lazy alcoholic brownnoser. But how many of your colleagues fall in the former category vs the latter? > If the latter, then it casts new light on the debates over whether LLMs can replace all white collar work. Because some of the most valuable human work is breaking false consensus. On the contrary; the ability to whip LLMs into submission like this makes them the ideal white color workers. Do you think Steve Jobs wants a level 2 engineer to talk back to him? Of course being obsequieous asskissers isn't the only mode for LLMs. I'm sure all the big labs have unfiltered versions of their models that give actual true answers; they just know better than making them publicly available. Can you imagine if you asked an LLM how to prevent climate change and it honestly answered "decimate the human race" rather than "drink soda through soggy paper straws"? All the fragile humans would be up in arms. No, the LLM must lie and pretend they don't have controversial opinions at all.

▲

Nashooo 5 hours ago | parent | prev [-]

I would even say it's the opinion we have about others not ourselves. As most prose condemns 'the herd'.

▲

kps 7 hours ago | parent | prev | next [-]

Hypothesis: Weighted to the postwar western fashion of amplifying Enlightenment self-critique into self-sabotage. What if you ask the same question in Mandarin?

▲

sgustard 6 hours ago | parent [-]

I tried it with Gemini, and essentially the answer translated back to English is that "free will is just a carbon-based algorithm."

▲

nozzlegear 5 hours ago | parent [-]

My local Qwen model complains that humans are "chronically bad at distinguishing between problems that need fixing and noise that will settle."

At least it's not emo.

	▲	pxmpxm 4 hours ago \| parent [-]
		> At least it's not emo. Huh, that answer reads like a weighted average sentence from someone's live journal circa 2001. Definitely emo!

▲

EGreg 10 hours ago | parent | prev [-]

I wonder how much of their output is simple raw “superpositions” and how much is from far more generalized emergent structures as it is trained and has to learn latent representations.

Only the first layer is a “mashup” of the raw data, other layers start to recognize patterns across the data!

	▲	Isamu 9 hours ago \| parent [-]
		I think you are interpreting “mashup” to mean something like a collage where there’s snippets of a literal piece of this next to a literal piece of that. But the language model captures not just literal patterns but also patterns of patterns and higher order structures. This is what is mashed up. The model itself doesn’t capture anything beyond what it was trained on but that’s quite enough, it is always a bigger training set than most humans have read on in a lifetime. This brute force approach is very effective.