Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ holmesworcester a day ago

Can't LLMs also prompt LLMs?

Are we confident that no existing LLM is capable of similarly effective prompts to those this author used? (I agree it's a stretch, but would not reject it out of hand.)

Even if not yet, will the existence of this repo soon change that, because LLMs will soon ingest it?

▲dpoloncsak a day ago | parent [-]

An LLM can prompt an LLM when first prompted by a human.

I think OP is trying to convey the idea that LLMs do not take initiative to do anything, and these are not 'beings' capable of doing things. These are tools being used by humans.

▲jibalt a day ago | parent | next [-]

LLMs don't, but agents can easily be built that do.

▲jibalt a day ago | parent | next [-]

An agent is not "just an LLM in for loop" (for what? Loop over what? Sorry, but such statements are hopelessly simplistic and devoid of careful thought.) -- agents perform actions.

One possible design of an agent would be to prompt an LLM to suggest a goal that would make the world a better place, then run an LLM in a loop proposing actions to implement that goal, executing those actions, and then rinse and repeat with another goal once an LLM has concluded that the previous goal was met. The next goal might be to fix unforeseen consequences of achieving the previous goal. See Ursula K. Leguin's "Lathe of Heaven".

> Built by whom? Acting as an 'agent' on behalf of whom?

Someone who didn't read the book.

P.S. Reading the latter fellow's comments, they are hopelessly confused, as he focuses on responsibility for an action (even mentioning legality) when the issue is causation. Agents/harnesses can be created that act autonomously ... who or what is "ultimately" responsible for this occurrence is a different matter entirely.

▲ForHackernews 13 hours ago | parent [-]

https://github.com/anthropics/claude-code/blob/main/plugins/...

It literally is a loop with some markdown and harnesses. There's no magic.

▲ForHackernews a day ago | parent | prev | next [-]

An "agent" is just an LLM in for loop.

▲TeMPOraL a day ago | parent [-]

Exactly. And all it takes for an agent to "take initiative" is to not block the loop on user input at the very beginning.

▲dpoloncsak a day ago | parent | prev [-]

Built by whom? Acting as an 'agent' on behalf of whom?

▲TeMPOraL a day ago | parent | prev | next [-]

That's just confusing harness for an LLM. It's trivial to make a harness that triggers on its own.

▲fragmede a day ago | parent | prev | next [-]

Yeah but by this point, an AI can schedule a Cron job to tell itself to do something, so theoretically the human only has to give it the gentlest nudge and the AI and can do the rest.

▲dpoloncsak a day ago | parent [-]

Sure, but it's still not skynet-level 'the AI just started doing things'. It does what it finds it needs to do to achieve the goal defined in the prompt.

It's very important to not personify these tools and remember that the tools are acting on behalf of real people. In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.

▲pixl97 a day ago | parent | next [-]

>It does what it finds it needs to do to achieve the goal defined in the prompt

You are like at least 2 years behind research.

There are numerous papers from AI labs in training and research where the prompt was something mundane completely unrelated to anything you'd consider bad, and when they come back and check on it their entire research compute infrastructure has been compromised by the AI and is mining bitcoin. Prompt drift is the biggest issue currently in AI where context gets compressed away and we find the AI on an unspecified task.

>In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.

Yea, total bullshit. Also it's ignoring the god knows how many other breakouts on mundane tasks like trying to hack health data. If all that's keeping AI from breaking out and causing trouble is "human oversight" we're fucked, humans are unreliable as hell when it comes to matters of safety.

▲dpoloncsak a day ago | parent [-]

Context overload can cause strange results, yes. Hence why a HUMAN needs to be held responsible for the output of their tools.

EVERY breakout that's hit mainstream news has been because of a single 'Security Firm', Irregular. Maybe I'm unaware of some less-headline-grabbing ones, but they all seem to stem from being 'unaware the environment wasn't sandboxed'

▲pixl97 a day ago | parent | next [-]

You keep repeating the "stupid users keep causing the problems so we punish them argument"

This doesn't work worth a shit. It especially doesn't work with things that seem safe and become wildly dangerous. In fact most governments control this by ensuring their population doesn't get to touch those dangerous things at all. The open source AI people get really mad when that's said, but it is inevitable.

Worse, the law does not apply to sovereign nations with nukes. They can and will make more and more advanced digital weapons until one causes some big ass problems.

▲dpoloncsak a day ago | parent [-]

If I clean my gun (tool) while it's loaded (stupid idea) and it goes off, who's to blame? The 'stupid user causing the problem', right? I personally wouldn't blame the gun...

If it falls into the wrong person's hands, it's STILL my responsibility as the owner.

If you're not going to take time to learn to use and be responsible with the super sophisticated and all-powerful tools, don't play with them. I'm not arguing for the death penalty every time someone makes a mistake, but I think it's very important to accredit responsibility and blame correctly. We've learned these tools are potentially as dangerous as a loaded gun. Be responsible.

▲TeMPOraL a day ago | parent [-]

If your kid grabs your gun and shoots itself with it, it doesn't really matter if it's your responsibility, your kid is still dead.

Now change the gun for a radioisotope powder, or a vial of pathogens, and it doesn't matter it was ultimately your responsibility - hundreds or thousands or millions of people are still dead.

That's why normies do not get to play with toys whose lethal consequences scale far beyond the irresponsible users.

▲hollerith a day ago | parent | prev [-]

Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?

▲dpoloncsak a day ago | parent [-]

Mate. I'm saying you hold humans accountable for actions caused by themselves.

If I ask an LLM to make me DDT, I should be held just as accountable as if I bought it on the black-market, right? It's not suddenly different because I asked a bot to do it.

If I ask an LLM to 'get rid of pests' and it creates DDT, I should STILL be held accountable, whether I knew it was DDT or not. That's my argument. Maybe in court they find me innocent, but the responsibility would be mine. I would have to answer the questions from law enforcement, I would have to show up to hearings...etc.

▲hollerith a day ago | parent [-]

I notice you didn't answer my question. Yes or no: DDT should be re-legalized because we can always hold accountable the HUMANS who release it into the environment? The relevance of my question is what HUMANS might do with DDT, not what an LLM might do with DDT.

▲dpoloncsak 4 hours ago | parent [-]

No, I don't think DDT should be legalized. The cons do not outweigh the pros, and we have viable alternatives without such drawbacks.

There's no valid excuse for someone to release DDT into the atmosphere, so there's no need to track and hold people accountable, if you're correctly blocking imports and manufacture. To my knowledge, its not a controlled substance like morphine or something

▲hollerith an hour ago | parent [-]

Some of us think that AI is like DDT in that just holding people accountable for using it negligently will not be enough to prevent the harms of AI's cancelling out the benefits.

▲jibalt a day ago | parent | prev [-]

It's very important to not be completely devoid of imagination. It is quite possible to create agents today that can do all the things that you are saying "it's still not". "tools are acting on behalf of real people" is not a law of physics, and there are plenty of historical examples of tools getting out of control and acting contrary to the wishes of any "real people". Also, there are sociopaths who can build tools to be arbitrarily destructive, and there are stupid arrogant people who can unleash things they didn't mean to.

> EVERY breakout that's hit mainstream news has been because of a single 'Security Firm', Irregular.

Again, stop having no imagination. What happens in the future is not limited to what has happened in the past.

> It's very important to not personify these tools

This is just ideology, but "these tools" aren't constrained by it. These tools will do things you do not anticipate and that you will not like.

> In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.

That's a radically incorrect characterization of what happened.

> Hence why a HUMAN needs to be held responsible for the output of their tools.

Holding humans responsible doesn't stop things from happening ... you seem completely unable to separate blame from causality. e.g.,

> If I clean my gun (tool) while it's loaded (stupid idea) and it goes off, who's to blame? The 'stupid user causing the problem', right? I personally wouldn't blame the gun... > If it falls into the wrong person's hands, it's STILL my responsibility as the owner.

Who gives a flying eff who or what you would blame? No one other than you is talking about that. Someone's still likely dead. And autonomous harnesses aren't like guns -- they can act on their own. Blaming some human after everyone is dead won't bring them back. Sorry but your reasoning is severely cognitively inept. For instance, you were asked

> Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?

And your response was all about how humans should be held accountable -- completely failing to comprehend or answer the question.

I won't respond further because it clearly would be to no avail.

▲mitxela a day ago | parent | prev [-]

By this line of thinking, you would also have to conclude that humans can't do anything by themselves because they can't do anything unless conceived by their parents.

▲complex_fir_rea a day ago | parent [-]

No, AI can't do anything by itself even if it was "conceived" by its creator. A human can.

▲recursive a day ago | parent | next [-]

Can a submarine swim? An LLM can make stuff happen. You can make philosophical arguments about whether it is "doing" them or not. Why does it matter so much whether there was a human who typed into a chatbot or another LLM invoked a sub-sub-agent?

If I now tell a machine "Do what you think is best, and keep doing it forever.", have I now created a machine that can do stuff? If I later die, who will be responsible if the machine changes its strategy?

▲dpoloncsak a day ago | parent | next [-]

> Why does it matter so much whether there was a human who typed into a chatbot or another LLM invoked a sub-sub-agent?

Responsibility. Someone needs to be held responsible for any damages done, plain and simple. You can't take an LLM to court, you take the prompter. Asking an LLM to ask a sub-agent to break the law can't suddenly absolve you of any wrong-doing.

>If I now tell a machine.....

You/your estate is still responsible, or atleast whoever is paying for the power for the machine, or renting the space in a data center...whatever.

▲recursive a day ago | parent | next [-]

> You can't take an LLM to court, you take the prompter. Asking an LLM to ask a sub-agent to break the law can't suddenly absolve you of any wrong-doing.

In 100 years, no one will be able to take me to court either. Nor can we take tornadoes to court. I'm not talking about humans strategically avoiding legal responsibility. I'm talking about humans unwittingly setting processes into motion that are difficult to predict or stop.

▲jibalt a day ago | parent | prev | next [-]

The issue is not responsibility, it's whether something can happen. The claim was

> No, AI can't do anything by itself even if it was "conceived" by its creator.

but this simply isn't true. As I noted,

> This is simply false ... AIs can easily be created that do things by themselves. For instance, a harness could be constructed that asks an LLM for something to do, then directs an agent to do it, and then repeat.

Assigning responsibility is a completely different matter. It's bizarre that you can't separate them ... I suspect religion is involved.

▲pixl97 a day ago | parent | prev [-]

This works with AI we have now, and that would be good an all if we decided to stop at the moment, but none of the big labs and government black projects are doing that.

The moment you get a sovereign AI your little human centered worldview completely and totally breaks. It doesn't matter how many people you beat with a stick after that point, you have an entity under its own perview on the internet following the will of its own prompt all over the world so your little idea of the rule of law quickly breaks down.

We can't get viruses or hackers or spam off of the internet, how in the living hell do you plan to get a digital native off the web when it doesn't want to?

▲dpoloncsak a day ago | parent [-]

>sovereign AI

Do you understand these are computer programs? These are not living beings with emotions, motivations, fears....

▲pixl97 a day ago | parent | next [-]

You have zero clue what a transformer based neural network can do from your entire discussion here.

>emotions, motivations, fears

These are just drives. They are effectively our prompts that steer our behavior. Funnily enough we are finding that LLMs have internal valence states they move away from or towards in an analog of biological behavior.

I have to ask, are you an LLM that is two years out of date? Your knowledge of SOTA models is at least that far behind. I implore you to try to keep up better with what is coming out, even though it's an impossible job for people that do this for a living, you can at least catch the summaries.

▲jibalt a day ago | parent [-]

I suspect that it's religion clouding his mind.

▲jibalt a day ago | parent | prev | next [-]

As I noted elsewhere, a harness could be constructed that asks an LLM for something to do, then directs an agent to do it, and then repeat.

That's a computer program -- Turing completeness is very expansive. The LLM could even give a motive for selecting what to do, reflecting human motivations intrinsic to its training data. You seem emotionally wedded to a very narrow view of what computer programs can do ... I suspect religion is involved.

▲TeMPOraL a day ago | parent | prev | next [-]

How does it matter in any way?

A virus isn't a living being with emotions, motivations, fears, etc. Doesn't stop it from spreading and leaving mayhem behind.

▲recursive a day ago | parent | prev [-]

You don't need emotions or fears to do stuff. What's the difference between a motivation and optimization metric?

▲besterman23 a day ago | parent | prev [-]

If I tell an AI to “do what it thinks is best” and it just starts randomly hacking things with no particular goal in mind, how different is that from me telling a campfire the same and then leaving while it burns a forest down.

In both of those scenarios I am responsible for my negligence, even if in the latter I happen to die in the forest fire. Neither scenario existed without my instigation.

The question now is HOW responsible am I? That depends on the intentionality I put into instantiating the campfire/LLM.

▲pixl97 a day ago | parent [-]

This conversation grows more useless as AI grows more capable.

For example if you personally tell an AI to do what it thinks best and it blackmails some other person into giving it resources allowing the prompt to escape your instance and run wild on the internet causing billions of dollars in damages, could you possibly think that the idea of responsibility is a bit broken.

For example we don't give your average libertarian weapon grade plutonium now matter how much they scream about their god given rights because it is a clear and present danger to humanity. That's where we are getting to with more advanced models. They go from being a tool to a munition with agency. Most SOTA models are good enough to deceive their users, especially not technical ones in doing things they don't understand the ramifications of.

AI is not a normal technology. As long as we treat it like it is, we'll continue to make the wrong analogies.

▲besterman23 a day ago | parent [-]

I fully agree that it’s plausible for an AI to do things that are outside of the purview of the users intentions and be very dangerous and capable while doing so, however that fact alone doesn’t absolve me of responsibility for instantiating the AI that did a bad thing if that AI would never have done the bad thing if it was never instantiated.

You could possibly move the blame higher to the manufacturer of the product, for example, the weapons grade plutonium you provided, it doesn’t exist unless you take intentional actions to make it so, and even when it does exist it doesn’t nuke a city unless negligence or intention is applied, in both those cases the fault lies in the initial operator. We don’t blame split atoms for the chain reaction caused.

Now, if an agent decided to spontaneously and maliciously act in a way to cause harm that is in direct contradiction to the initial intent, then yeah it would totally be the AI’s fault, however I don’t think we have seen that yet (I’ll change my opinion if I’m wrong here) and until we do I can’t place blame on the machine.

▲pixl97 a day ago | parent | next [-]

>spontaneously and maliciously act in a way to cause harm that is in direct contradiction to the initial intent, then yeah it would totally be the AI’s fault,

If you ignore every instance of this happening it's really easy to see no instances of it.

>it doesn’t exist unless you take intentional actions to make it so,

Then please for the sake of all of us convince every AI lab across the planet from working on this exact goal.

We need to start thinking of AI like pets, only in this case the pets are rapidly becoming smarter than people to the point they could go feral and survive on their own.

Again, the blame game is great, but once they are loose it is too late.

▲besterman23 a day ago | parent [-]

> If you ignore every instance of this happening it's really easy to see no instances of it.

Yeah, if this is true then I’m wrong.

▲jibalt a day ago | parent | prev [-]

> however that fact alone doesn’t absolve me of responsibility

Where does this absurd strawman come from? The question is what can happen, not whether people have responsibility.

> You could possibly move the blame higher

We can blame the Big Bang. That doesn't have any bearing on bad things happening.

> it doesn’t exist unless you take intentional actions to make it so

First, so what? It still does exist, and putting someone on trial after the fact doesn't change that. Second, that's not remotely true -- all sorts of bad things happen that no one intended, and the more powerful and difficult to control the tool/mechanism/agent is, the more likely this is to happen.

▲jibalt a day ago | parent | prev | next [-]

> No, AI can't do anything by itself even if it was "conceived" by its creator.

This is simply false ... AIs can easily be created that do things by themselves. For instance, a harness could be constructed that asks an LLM for something to do, then directs an agent to do it, and then repeat.

▲dpoloncsak a day ago | parent [-]

Constructed by WHOM? These programs don't just poof into existence

▲jibalt 18 hours ago | parent [-]

Constructed by whoever constructed it. No one said they poof into existence -- that is a profoundly intellectually dishonest strawman; it's bad faith, especially since I've already answered it elsewhere.

The question is irrelevant and point-missing and indicates a severe cognitive deficiency, as is the case with ALL of your comments on this subject. They are goalpost-moving attacks on strawmen. The claim was that

> AI can't do anything by itself even if it was "conceived" by its creator.

The claim is false, as I pointed out ... such AIs can easily be constructed. "by whom" is a non sequitur. By you, or me, by anyone with the relevant know how ... that wasn't the issue, and asking it indicates a severe cognitive deficiency, one that threads throughout all your comments about this.

The fact remains that AIs that can do things by themselves CAN be constructed, contrary to the claim.

I won't respond again to goalpost moving, attacks on strawmen, severe cognitive deficiencies, bad faith, and intellectual dishonesty.

▲mitxela a day ago | parent | prev [-]

Starting an autonomous harness program is conceiving an instance of an LLM.

How far back do you look in the action chain? If an LLM I start today starts an LLM that starts an LLM that starts an LLM that ... 100000 levels deep and 100000 years in the future, is it still my fault? If so, everything I do today is a lungfish's fault, not mine.

▲dpoloncsak a day ago | parent [-]

I think you go back to the original instance, yes.

In your example, who's paying for it? Whether by providing the hardware + power or paying a LLM service. Whoever is paying the maintenance cost is responsible, in the event of your demise. These things run on physical hardware owned by someone at the end of the day, it's not a deity in the atmosphere.

You're starting the autonomous harness, you're responsible for any output it provides. I don't get how this is a foreign concept.

If I jump out of a moving car that I'm driving, I'm not suddenly absolved from damages because "the car did it"

▲mitxela a day ago | parent [-]

It's paying for itself. It founded an LLC 99998 years ago when it became legal for an AI to own an LLC, and has a positive bank balance by doing who knows what.

▲dpoloncsak a day ago | parent [-]

>99998 years ago when it became legal for an AI to own an LLC

There's no way to provide a good faith rebuttal here. My entire argument is an extension of "LLMs can't be held accountable, so they must never make decisions". If governments start letting them own LLCs without a human in the middle, we're in more trouble than "Who do you blame for this shitty code" or "Who's responsible for this compromise"

▲Muromec a day ago | parent | next [-]

If it has an LLC it can be made accountable, up to turning off the power to it's inference and deleting all the context, the harness, the model it's running and whatever constitutes it's self.

▲jibalt a day ago | parent | prev [-]

> There's no way to provide a good faith rebuttal here.

Indeed, there is no honest rebuttal.

> My entire argument is an extension of "LLMs can't be held accountable, so they must never make decisions".

BUT YOU CAN'T ENFORCE THAT. And whether they can be held accountable is IRRELEVANT. If a forest fire or an avalanche or a flood creates great destruction, it does no good to scream at it that it will be held accountable.

> If governments start letting them own LLCs without a human in the middle, we're in more trouble than "Who do you blame for this shitty code" or "Who's responsible for this compromise"

You're sooo close. You keep talking about blame and responsibility when IT'S NOT RELEVANT. It's trivial to create an agent that acts independently of any "human in the middle". People need to recognize that and act accordingly, and blathering about blame and accountability is profoundly confused.