Intelligence is only meaningful with respect to a goal or set of goals.

I think organizations' intelligence can only be called superhuman if you ascribe them their own inscrutable goals, different from what people want their goals to be.

But why would you do that?

It's true that a lot of the debate about AI alignment - the good parts, anyway - also apply to human institutions.

▲

TeMPOraL a day ago | parent [-]

> But why would you do that?

Because that is what actually gets pursued. The set of effective goals an organization is distinctly different from those of any of the individuals in that organization.

▲

vintermann a day ago | parent [-]

You're missing what I'm trying to say. There are no goals in nature. Agency and goal-directedness is something we ascribe to things.

The set of things an organization ends up doing may be distinctly different from those of the individuals, but that does in no way oblige us to ascribe agency to the organization on its own.

▲

jeee1 a day ago | parent [-]

Yes there are - the goal is to survive. This is hard coded into any entity.

What’s the primary goal of any for profit business? To survive for eternity - going concern in perpetuity that is hopefully very valuable.

▲

vintermann a day ago | parent [-]

You're still not getting it. "Entity", "survival", these are still arbitrary or circularly defined terms. You decided what an entity is (i.e where one thing ends and a different thing begins) and what the essence which "survives" is (i.e. why it's a big deal whether there's a different chunk of nucleic acids similar but not identical to my own floating around 200 years from now).

These ideas are not wrong, but they are teleological. They are about the purpose of things. And nature doesn't come with purposes, it's all something we impose on it.

▲

svieira a day ago | parent [-]

> And nature doesn't come with purposes, it's all something we impose on it.

That is the presumption others are calling you out for. Things very clearly act appear to act with purpose, they just clearly don't provide it for themselves. You are presuming one solution to the problem (that appearance does not hold in reality), but there is another one.

▲

vintermann 20 hours ago | parent [-]

A storm can appear to act with purpose too, or a planet orbiting a star, yet I doubt you assign it agency. It's fundamentally your choice to assign something agency or not, whether you are right or wrong to do so, is not an empirical question. It is a teleological one. "Call me out" all you want, this is very basic.

	▲	svieira 6 hours ago \| parent [-]
		Actually, everything is infused with purpose (not agency, that's different). Including rain drops and hurricanes and black holes.