I mean, many companies operate within particular domains but don't have much skin in the domain itself in a way that is meaningful to other participants there -- like Oracle builds software for healthcare, ostensibly, but any actual user of their products will tell you they are nightmarish with regard to usability, and research shows that patients are often harmed as a result. So healthcare isn't their game, it's selling healthcare software.

Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences

> The company learns. It adapts

And potentially steals [1].

[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...

I think it means if we all have to justify ourselves to each other, we collectively come up with an sustainable ecosystem of ideas that we can understand. So if we stop justifying ourselves to each other, the ecosystem doesn't get built or sustained, and we stop understanding things. Not the person you responded to, but that's my reading.

I actually agree with you. LLMs could become participants in the future, but this would require (1) durability of real awareness over time (an architecture change from the current fixed-weights + transient context architecture) and (2) that humans grant them the status of equal participants (probably not a terribly high bar given how people already treat these things). I'm not aware of any real progress toward (1), and all the stand-ins people have developed live in the context layer or textual artifacts, which are poor stand-ins for durable memory.

And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.

Brandom wrote a shorter version of the book, called "Articulating Reasons", which is available in full on his faculty page [1], or from booksellers. He's a very wordy writer (kind of exhausting to read, even for someone who enjoys philosophy), but if you get ahold of Making It Explicit, chapters 2 and 3 have the meat of the ideas I'm talking about here.

[1] https://sites.pitt.edu/~rbrandom/Courses/2022%20Phil%20of%20...

Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.

RAG context memory and modifying the weights are two different things. As far as I know, models can't modify their weights reliably.

Humans change over time due to what they experience. It’s different from waking up from a ten-year coma and having access to a record of what happened to yourself in an alternative timeline where you didn’t fall into the coma.

The irony of complaining about an academic jargon word in a thread about mathematicians.

Because language is mimetic, and people are consuming huge quantities of AI-generated text. I.e. saying that an LLM can't be an equal participant in our language games isn't saying that it can't play by the rules. In fact that's what LLMs are trained to be best at.

What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.

I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...

It would only be true if you consider Mathematics and Language to be the same thing.

Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.

The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.

That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.

And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.

Ideas are contagious and AI is our most powerful vector yet.

I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.