> a system was given a goal and it achieved that goal

If a security firm you'd hired for pentesting did this (hacking a third party, and not informing you and covering it up), would you hire them again? Or would you say it was your own fault for giving them too broad a goal?

▲ drewbeck 11 minutes ago | parent | next [-]

This is a great thought experiment bc it raises the question of WHY humans wouldn’t behave this way. IMO the answer is a lot of socially enforced incentives that are dynamic and would be tough to fully articulate in a prompt.

The white hat has their own liability to consider, and the liability of their employer. Reputation and relationships are a big factor. All these tie into fundamental human incentives: survival, community acceptance, safety and freedom (prison not preferred!).

It’s a good sketch of why alignment is difficult, at least when it’s conceived of as an attempt to match human behavior.

▲ randomImmigrant 2 hours ago | parent | prev | next [-]

I wouldn’t hire them again, and if they did behave like an amoral hacker collective that will do anything for me, pre AI I’d have reported them. Today I’d say they failed to convince me they’re human and thus failed the Turing test when their actions are viewed in aggregate.

	▲	kalkin 2 hours ago \| parent [-]
		> I wouldn’t hire them again Right, me neither. Because there's a common sense delineation between actions that are reasonably expected when "a system was given a goal and it achieved that goal" and actions that are obviously misaligned with the goal-giver and unwanted even if some indirect sense they were causally related to the goal. We have no trouble making this kind of distinction for humans, so we shouldn't pretend it's impossible for AIs in order to put our hands over our eyes and pretend there's in principle no such thing as one that's misaligned or rogue.

▲ areoform 2 hours ago | parent | prev [-]

During the Nixon administration, when the President and his accomplices, apologies, advisors directed former federal agents to spy on his opponents, https://en.wikipedia.org/wiki/Operation_Sandwedge then in the fall out, who was held to be the most liable for these actions?

The federal agents, or the Nixon administration?

If you task a system explicitly to do "advanced exploitation" via "complex attach paths," then who is liable here? The machine lacking the autonomy of the federal agents that carried out Watergate, or the people telling the machine what to do?

▲ kalkin 2 hours ago | parent [-]

I've never heard of Intertel, but Wikipedia says:

> Nixon's staff also anticipated that the Democratic campaign would employ the services of Intertel

Are you sure you're not garbling the story?

In any case, I would expect an ethical firm to refuse to spy on the president's political opponents and want one that broke the law to be prosecuted, but more importantly, the gaping hole in your analogy is that Nixon directed spying _on his opponents_, but OpenAI did not direct hacking _of HuggingFace_.

What you're doing is more like saying "the American people elected Nixon with a mandate to spy on enemies, so what right do they have to complain?"

	▲	areoform an hour ago \| parent [-]
		`Are you sure you're not garbling the story?` No, you're right, I mis-remembered. I still write my comments the old-fashioned way. They were proposing to create a counter-firm and used federal agents. For the rest, please see, https://news.ycombinator.com/item?id=49457025