>Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions.

Damn, what a line!

Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselves. LLMs completely lack this, their weights are static. An LLM isn't going to be molded by a bad breakup, or a relative passing away. An LLM isn't going to set up a routine to get stronger with training, nor smarter by reading up on a field.

The thing is... Interactive updates happen, just in a different way than it does for animal brains. The system is updated with new training data more-or-less constantly. Suppose OpenAI (or whoever) collects a week's worth of conversations with up-thumbs and down-thumbs, or rewritten continuations from human operators, then fine-tunes the current version of ChatGPT with that data. That's an interactive update, and learning from experience. It looks mostly nothing like what we humans do... But it does rhyme a little bit!

We humans have mostly frozen weights (neurons), or else we would constantly be having to avoid forgetting how to walk+talk. We have a period of greater plasticity (youth!), and use sleep and dreams to perform 'deeper' updates than occur when we're awake: We tend to suck a bit at picking up new skills from zero, but improve rapidly with practice over days.