That doesn't resolve the question.

  > Not sure where you imagine my inconsistency is.

  >> Let's take a step back. At what point is it me making the toast and not the toaster? Is it because I have to press the level? We can automate that. Is it because I have to put by bread in? We can automate that. Is it because I have to have the desire to have toast and initiate the chain of events? How do you measure that?

You have a PhD and 30 years of experience, so I'm quite confident you are capable of adapting the topic of "making toast" to "playing chess", "doing physics", "programming", or any similar topic where we are benchmarking results.

Maybe I've (and others?) misunderstood your claim from the get-go? You seem to have implied that LLMs understand chess, physics, programming, etc because of their performance. Yet now it seems your claim is that the LLM and I are doing those things together. If your claim is that a LLM understands programming the same way a toaster understands how to make toast, then we probably aren't disagreeing.

But if your claim is that a LLM understands programming because it can produce programs that yield a correct output to test cases, then what's the difference from the toaster? I put the prompts in and pushed the button to make it toast.

I'm not sure why you imagine the inconsistency is so difficult to see.

▲ robotresearcher 16 hours ago | parent | next [-]

When did I say that the chess program was different to a toaster? I don’t believe it is, so it’s not a thing I’m likely to say.

I don’t think the word ‘understand’ has a meaning that can apply in these situations. I’m not saying the toaster or the chess program understands anything, except in the limited sense that some people might describe them that way, and some won’t. In both cases that concept is entirely in the head of the describer and not in the operation of the device.

I think the claimed inconsistency is in views you ascribe to me, and not those I hold. ‘Understand’ is a category error with respect to these devices. They neither do or don’t. Understanding is something an observer attributes for their own reasons and entails nothing for the subject.

▲ simondotau 20 hours ago | parent | prev [-]

Declaring something as having "responsibility" implies some delegation of control. A normal toaster makes zero decisions, and as such it has no control over anything.

▲ robotresearcher 15 hours ago | parent [-]

A toaster has feedback control over its temperature, time control over its cooking duration, and start/stop control by attending to its start/cancel buttons. It makes decisions constantly.

I simply can't make toast without a toaster, however psychologically primary you want me to be. Without either of us, there's no new toast. Team effort every time.

And to make it even more interesting, the same is true for my mum and her toaster. She does not understand how her toaster works. And yet: toast reliably appears! Where is the essential toast understanding in that system? Nowhere and everywhere! It simply isn't relevant.

▲ simondotau 9 hours ago | parent | next [-]

> A toaster has feedback control over its temperature, time control over its cooking duration

Most toasters are heating elements attached to a timer adjusted by the human operator. It doesn’t have any feedback control. It doesn’t have any time control.

> I simply can't make toast without a toaster

I can’t make toast without bread either, but that doesn’t make the bread “responsible” for toasting itself.

> She does not understand how her toaster works.

My mum doesn’t understand how bread is made, but she can still have the intent to acquire it from a store and expose it to heat for a nominal period of time.

▲ godelski 14 hours ago | parent | prev [-]

  > I simply can't make toast without a toaster

You literally just put bread on a hot pan.

▲

robotresearcher 14 hours ago | parent [-]

So despite passing the Toasting Test, a hot pan is not really a toaster?

It’s clear that minds are not easily changed when it comes to noticing and surrendering folk psychology notions that feel important.

	▲	godelski 12 hours ago \| parent [-]
		You said you couldn't make toast without a toaster. Sorry, if I didn't understand what you actually meant