There are many aspects to this that people like yourself miss, but I think we need satisfactory answers to them (or at least rigorous explorations of them) before we can make headway in these sorts of discussion.

Imagine we assume that A.I. could be conscious. What would be the identity/scope of that consciousness. To understand what I'm driving at, let's make an analogy to humans. Our consciousness is scoped to our bodies. We see through sense organ, and our brain, which process these signals, is located in a specific point in space. But we still do not know how consciousness arises in the brain and is bound to the body.

If you equate computation of sufficient complexity to consciousness, then the question arises: what exactly about computation would prodcuce consciousness? If we perform the same computation on a different substrate, would that then be the same consciousness, or a copy of the original? If it would not be the same consciousness, then just what give consciousness its identity?

I believe you would find it ridiculous to say that just because we are performing the computation on this chip, therefore the identity of the resulting consciousness is scoped to this chip.

▲

gf000 3 days ago | parent | next [-]

> Imagine we assume that A.I. could be conscious. What would be the identity/scope of that consciousness

Well, first I would ask whether this question makes sense in the first place. Does consciousness have a scope? Does consciousness even exist? Or is that more of a name attributed to some pattern we recognize in our own way of thinking (but may not be universal)?

Also, would a person missing an arm, but having a robot arm they can control have their consciousness' "scope" extended to it? Given that people have phantom pains, does a physical body even needed to consider it your part?

▲

tsimionescu 3 days ago | parent | prev [-]

This all sounds very irrelevant. Consciousness is clearly tied to specific parts of a substrate. My consciousness doesn't change when a hair falls off my head, nor when I cut my fingernails. But it does change in some way if you were to cut the tip of my finger, or if I take a hormone pill.

Similarly, if we can compute consciousness on a chip, then the chip obviously contains that consciousness. You can experimentally determine to what extent this is true: for example, you can experimentally check if increasing the clock frequency of said chip alters the consciousness that it is computing. Or if changing the thermal paste that attaches it to its cooler does so. I don't know what the results of these experiments would be, but they would be quite clearly determined.

Of course, there would certainly be some scale, and at some point it becomes semantics. The same is true with human consciousness: some aspects of the body are more tightly coupled to consciousness than others; if you cut my hand, my consciousness will change more than if you cut a small piece of my bowel, but less than if you cut out a large piece of my brain. At what point do you draw the line and say "consciousness exists in the brain but not the hands"? It's all arbitrary to some extent. Even worse, say I use a journal where I write down some of my most cherished thoughts, and say that I am quite forgetful and I often go through this journal to remind myself of various thoughts before taking a decision. Would it not then be fair to say that the journal itself contains a part of my consciousness? After all, if someone were to tamper with it in subtle enough ways, they would certainly be able to influence my thought process, more so than even cutting off one of my hands, wouldn't they?

▲

prmph 3 days ago | parent [-]

You make some interesting points, but:

> Similarly, if we can compute consciousness on a chip, then the chip obviously contains that consciousness.

This is like claiming that neurons are conscious, which as far as we can tell, they are not. For all you know, it is the algorithm that could be conscious. Or some interplay between the algorithm and the substrate, OR something else.

Another way to think of it problem: Imagine a massive cluster performing computation that is thought to give rise to consciousness. Is is the cluster that is conscious? Or the individual machines, or the chips, or the algorithm, or something else?

I personally don't think any of these can be conscious, but those that do should explain how they figure these thing out.

	▲	hackinthebochs 2 days ago \| parent \| next [-]
		>Is is the cluster that is conscious? Or the individual machines, or the chips, or the algorithm, or something else? The bound informational dynamic that supervenes on the activity of the individual units in the cluster. What people typically miss is that the algorithm when engaged in a computing substrate is not just inert symbols, but an active, potent causal/dynamical structure. Information flows as modulated signals to and from each component and these signals are integrated such that the characteristic property of the aggregate signal is maintained. This binding of signals by the active interplay of component signals from the distributed components realizes the singular identity. If there is consciousness here, it is in this construct.
	▲	tsimionescu 3 days ago \| parent \| prev [-]
		I explained the experiments that you would do to figure that out: you modify parts of the system, and check if and how much that affects the consciousness. Paint the interconnects a different color: probably won't affect it. Replace the interconnect protocol with a different one: probably will have some effect. So, the paint on the interconnect: not a part of the consciousness. The interconnect protocol: part of the consciousness. If we are convinced that this is a real consciousness and thus these experiments are immoral, we simply wait until accidents naturally occur and draw conclusions from that, just like we do with human consciousness. Of course, "the consciousness" is a nebulous concept. It would be like asking "which part of my processor is Windows" to some extent. But it's still fair to say that Windows is contained within my computer, and that the metal framing of the computer is not part of Windows.