It shouldn't surprise anyone that a chatbot trained on countless minecraft videos would have the ability to roleplay as someone playing minecraft. If you've got links to research showing something that isn't that I would be interested in seeing it.

https://www.youtube.com/watch?v=uRDBco-cSK4

https://www.technologyreview.com/2024/11/27/1107377/a-minecr...

sadly paid-walled (and decently coded) article.

The point is, there appears to be emergent behavior happening: the forming of religion, governance, and social interaction that was not in any system-card. Which makes sense, A brain in a vat with no external stimuli is a boring thing indeed, so too would an LLM be outside of a harness and any input. But put either in a society...

More generally: if having an internal model of how the world works improves next token prediction, then the LLM that contains that would be selected for. For humans, that is the case for our predictions of what would come next (we don't use tokens, of course). We have other studies indicating that there appear to be internal representations of concepts: https://arxiv.org/abs/2305.11169

I really do want to push back on the LLM + harness = chatbot position. A chatbot really is just a served LLM, sure there is code around it that may look like a harness: but unless it loops, streams, or has an interval activation, then it is decidedly not a 'harness'. A harness is defined by having such a loop/stream/interval: regardless of tool-use. Agent means it acts agentically: which requires that loop/stream/interval. Harnesses are what turn chat-bots into (possible) agents.