"The useful distinction is between wielding an instrument with skill and delegating work without adequate understanding or oversight."

-----------------

Incidents like the HuggingFace hack really are the elephant in the room here. The people building these tools couldn't wield them "with skill", in the sense that the tools they built and understood better than any other humans alive ran amok in ways they couldn't predict. If they didn't have adequate understanding of what they themselves built, who would?

Agency may prove to be the difference between a tool and a collaborator. Calling AI a collaborator isn't an attempt to anthropomorphize it. We can collaborate with a horse to travel, but the horse has some agency of its own and might escape our control and do things we didn't intend, like eat apples off of a stranger's tree. We may need to think of AI as something more than a mere tool if we're going to harness it to our will.

▲

joebergeron 7 hours ago | parent [-]

> Calling AI a collaborator isn't an attempt to anthropomorphize it. We can collaborate with a horse to travel

Naive question: What, practically speaking, is the delta between anthropomorphization of AI, which it seems like you are trying to avoid (fairly so!), and the sort-of animistic stance you seem more comfortable to apply?

I reckon that a lot of folks who might recoil at ascribing human qualities to LLMs would similarly feel discomfort comparing those models to a horse. While it's probably uncontroversial to say that horses are, coarsely speaking, "simpler" in many ways than humans, I suspect most people would agree that they are more complex than i.e., a "stochastic parrot".

Is the difference related to motive, maybe? That humans can have hidden preferences, and their actions can in fact betray their intended ends? That no matter how closely we observe someone, we may not glean any meaningful insight into their true motives? And we feel some (justified) discomfort in conferring this quality to AI? I think non-human animals can exhibit this sort of behavior too (albeit to a lesser degree). To me, it feels like restricting oneself to analogizing AI to a non-human animal in order to keep some distance from ascribing it consciousness may be less effective to that end than desired.

In place of your horse example, would it feel appropriate to liken AI to, say, a compiler peppered with bugs? Program - prompt - goes in, executable - agent actions - come out; mostly what we expect, but occasionally goes off the rails in ways we cannot understand. The caveat there, I suppose, is that we have not until now created a compiler which we cannot sensibly decompile.

Fun to think about.

	▲	beloch 6 hours ago \| parent [-]
		Scary as well. You can control a tool, but not necessarily a sufficiently intelligent collaborator. If you approach AI as a collaborator, the challenge isn't to understand it and predict it, as OpenAI failed to do with their own product, but to motivate it correctly and and constrain it sufficiently to guarantee safety. You need guardrails for when you get the alignment wrong. How strong can guardrails be made though? The public has this sci-fi idea that AI can be constrained with simple, Asimov style laws. The problem is that LLM's don't think in the way humans do. They can produce results of dizzying complexity but hallucinate or fail to do basic arithmetic. You can't currently rely on AI to follow simple rules like "Don't access an outside network.". This is a problem that needs to be addressed urgently.