Remix.run Logo
atomic128 5 hours ago

    Once men turned their thinking over to machines
    in the hope that this would set them free.

    But that only permitted other men with machines
    to enslave them.

    ...

    Thou shalt not make a machine in the
    likeness of a human mind.

    -- Frank Herbert, Dune
You won't read, except the output of your LLM.

You won't write, except prompts for your LLM. Why write code or prose when the machine can write it for you?

You won't think or analyze or understand. The LLM will do that.

This is the end of your humanity. Ultimately, the end of our species.

Currently the Poison Fountain (an anti-AI weapon, see https://news.ycombinator.com/item?id=46926439) feeds 2 gigabytes of high-quality poison (free to generate, expensive to detect) into web crawlers each day. Our goal is a terabyte of poison per day by December 2026.

Join us, or better yet: deploy weapons of your own design.

teo_zero 2 hours ago | parent | next [-]

You shouldn't take a sci-fi writer's words as a prophecy. Especially when he's using an ingenious gimmick to justify his job. I mean, we know that it's impossible for anyone to tell how the world will be like after the singularity, by the very definition of singularity. Therefore Herbert had to devise a ploy to plausibly explain why the singularity hadn't happened in his universe.

jrflowers an hour ago | parent [-]

I like the idea that Frank Herbert’s job was at risk and that’s why he had to write about the Butlerian Jihad because it kind of sounds like on the other side you have Ray Kurzweil, who does not have to justify his job for some reason.

n4r9 an hour ago | parent [-]

Does seem funny to think of sci fi writers as being particularly concerned about justifying their jobs.

debo_ 5 hours ago | parent | prev | next [-]

If you read this through a synth, you too can record the intro vocal sample for the next Fear Factory album

protocolture an hour ago | parent | prev | next [-]

>Why write code or prose when the machine can write it for you?

I like to do it.

>You won't think or analyze or understand. The LLM will do that.

The clear lack of analysis seems to be your issue.

>This is the end of your humanity. Ultimately, the end of our species.

Doubtful.

baxtr 3 hours ago | parent | prev | next [-]

"The end of humanity" has been proclaimed many times over. Humanity won't end. It will change like it always has.

We get rid of some problems, and we get a bunch of new problems instead. And on, and on, and on.

austinjp an hour ago | parent | next [-]

Russell's chicken (or turkey) would like a word.

https://en.wikipedia.org/wiki/Turkey_illusion

baxtr an hour ago | parent [-]

I love that you brought this up.

Chickens are killed ALL the time. It’s a recurring mass event. If you were a smart chicken you could see that pattern and put it into a formula.

In contrast, the end of Humanity would be a singular event. It’s even in the name…

And that is fiction / speculation in comparison. It’s not backed by any data. Human survival over 300,000 years by contrast is.

I mean it’s fine to dream things up, but let’s be fair and call it what it is.

computomatic 18 minutes ago | parent | next [-]

On the other hand, species go extinct with increasing regularity.

Invictus0 31 minutes ago | parent | prev [-]

298,000 of those years didn't have toilet paper. It was utterly impossible for a single person to "end humanity" even 200 years ago; now, the president can do it in minutes by launching a salvo of nukes. Comparing the present moment to the hunter/gatherer days is preposterous.

baxtr 18 minutes ago | parent [-]

It’s absurd and not scientific to claim that you can predict "a salvo of nukes" will kill humanity.

We don’t know how this will play out. It never happened before. Same with the chicken above.

snoman 2 hours ago | parent | prev | next [-]

It only has to be right once. Humanity won’t end until it does.

nicce 2 hours ago | parent | prev [-]

Humanity may end if someone else goes to the top of food chain.

creddit 4 hours ago | parent | prev | next [-]

I would bet a lot of money on your poison is already identified and filtered out of training data.

scratchyone 4 hours ago | parent | prev | next [-]

Looking through the poison you linked, how is it generated? It's interesting in that it seems very similar to real data, unlike previous (and very obvious) markov chain garbage text approaches.

atomic128 4 hours ago | parent [-]

We do not discuss algorithms. This is war. Loose lips sink ships.

We urge you to build and deploy weapons of your own unique design.

scratchyone 4 hours ago | parent | next [-]

[flagged]

CoffeeOnWrite 3 hours ago | parent [-]

It's based on this paper BTW: https://www.anthropic.com/research/small-samples-poison

floatrock 4 hours ago | parent | prev [-]

[flagged]

gojomo 5 hours ago | parent | prev | next [-]

Like partial courses of antibiotics, this will only relatively-advantage thoae leading efforts best able to ignore this 'poison', accelerating what you aim to prevent.

testaccount28 4 hours ago | parent [-]

yes. whoever has the best (least detectable) model is best poised to poison the ladder for everyone.

3 hours ago | parent | prev | next [-]
[deleted]
fellowmartian 5 hours ago | parent | prev | next [-]

I think you’re missing the point of Dune. They had their Butlerian Jihad and won - the machines were banned. And what did it get them? Feudalism, cartels, stagnation. Does anyone seriously want to live in the Dune universe?

The problem isn’t in the thinking machines, it’s in who owns them and gets our rent. We need open source models running on dirt cheap hardware.

accidentallfact 4 hours ago | parent [-]

The point of Dune is that the worst danger are people who obey authority without questioning it.

xmprt 4 hours ago | parent | next [-]

Then wouldn't open source models running on commodity hardware be the best way to get around that? I think one of the greatest wins of the 21st century is that almost every human today has more computing power than the entire US government in the 1950s. More computer power has democratized access and ability to disperse information. There are tons of downsides to that which we're dealing with but on the net, I think it's positive.

shinycode 4 hours ago | parent | next [-]

Does it also means the US government has x1000000 more power than the one in 1950 ?

stnmtn 3 hours ago | parent [-]

speaking strictly from an energy standpoint (power grid, megatons of warheads, etc).. it's probably close to that number.

accidentallfact 4 hours ago | parent | prev [-]

It isn't a way around, you still obey. Only now, the authority you obey is a machine.

wiseowise an hour ago | parent | prev | next [-]

That's not the point of Dune. Who blindly obeyed who?

api 4 hours ago | parent | prev [-]

... which overthrowing the machines didn't stop. People just found another authority to mindlessly obey.

jrflowers 2 hours ago | parent | prev | next [-]

The “poison fountain” is just a little script that serves data supplied by… somebody from my domain? It seems like it would be super easy for whoever maintains the poison feed to flip a switch and push some shady crypto scam or whatever.

accidentallfact 5 hours ago | parent | prev | next [-]

A better approach is to make AI bullshit people on purpose.

zahlman 3 hours ago | parent [-]

This is essentially just that. The idea is that "poisoned" input data will cause AIs that consume it to become more likely to produce bullshit.

octernion 5 hours ago | parent | prev | next [-]

do... do the "poison" people actually think that will make a difference? that's hilarious.

mock-possum 2 hours ago | parent [-]

Let the kiddies have their crusade

spacemark 3 hours ago | parent | prev [-]

Lol. Speak for yourself, AI has not diminished my thinking in any material way and has indeed accelerated my ability to learn.

Anyone predicting the "end of humanity" is playing prophet and echoing the same nonsensical prophecies we heard with the invention of the printing press, radio, TV, internet, or a number of other step-change technologies.

There's a false premise built into the assertion that humanity can even end - it's not some static thing, it's constantly evolving and changing into something else.

arjie 3 hours ago | parent [-]

A large number of people read a work of fiction and conclude that what happened in the work of fiction is an inevitability. My family has a genetically-selected baby (to avoid congenital illness) and the Hacker News link to the story had these comments all over it.

> I only know seven sci-fi films and shows that have warned about how this will go badly.

and

> Pretty sure this was the prologue to Gattaca.

and

> I posted a youtube link to the Gattaca prologue in a similar post on here. It got flagged. Pretty sure it's virtually identical to the movie's premise.

I think the ironic thing in the LLM case is that these people have outsourced their reasoning to a work of fiction and now are simple deterministic parrots of pop culture. There is some measure of humor in that. One could see this as simply inter-LLM conflict with the smaller LLMs attempting to fight against the more capable reasoning models ineffectively.

MichaelZuo an hour ago | parent [-]

Now that you mention it, it is pretty strange to see HN users parroting other people’s thinking (sci-fi writers) like literal sub-sapient parrots, while simultaneously decrying the danger of machines turning people into sub-sapient parrots…

Following that logic… the closest problem would be literally inbetween their ears.