▲ | whatevertrevor a day ago | ||||||||||||||||
> In the natural world, agency is a consequence of death: by dying, the feedback loop closes in a powerful way. I don't follow. If we, in some distant future, find a way to make humans functionally immortal, does that magically remove our agency? Or do we not have agency to begin with? If your position on the "free will" question is that it doesn't exist, then sure I get it. But that seems incompatible with the death prerequisite you have put forward for it, because if it doesn't exist then surely it's a moot point to talk prerequisites anyway. | |||||||||||||||||
▲ | arthurofbabylon a day ago | parent [-] | ||||||||||||||||
When I think of the term "agency" I think of a feedback loop whereby an actor is aware of their effect and adjusts behavior to achieve desired effects. To be a useful agent, one must operate in a closed feedback loop; an open loop does not yield results. Consider the distinction between probabilistic and deterministic reasoning. When you are dealing with a probabilistic method (eg, LLMs, most of the human experience) closing the feedback loop is absolutely critical. You don't really get anything if you don't close the feedback loop, particularly as you apply a probabilistic process to a new domain. For example, imagine that you learn how to recognize something hot by hanging around a fire and getting burned, and you later encounter a kettle on a modern stove-top and have to learn a similar recognition. This time there is no open flame, so you have to adapt your model. This isn't a completely new lesson, the prior experience with the open flame is invoked by the new experience and this time you may react even faster to that sensation of discomfort. All of this is probabilistic; you aren't certain that either a fire or a kettle will burn you, but you use hints and context to take a guess as to what will happen; the element that ties together all of this is the fact of getting burned. Getting burned is the feedback loop closing. Next time you have a better model. Skillful developers who use LLMs know this: they use tests, or they have a spec sheet they're trying to fulfill. In short, they inject a brief deterministic loop to act as a conclusive agent. For the software developer's case it might be all tests passing, for some abstract project it might be the spec sheet being completely resolved. If the developer doesn't check in and close the loop, then they'll be running the LLM forever. An LLM believes it can keep making the code better and better, because it lacks the agency to understand "good enough." (If the LLM could die, you'd bet it would learn what "good enough" means.) Where does dying come in? Nature evolved numerous mechanisms to proliferate patterns, and while everyone pays attention to the productive ones (eg, birth) few pay attention to the destructive (eg, death). But the destructive ones are just as important as the productive ones, for they determine the direction of evolution. In terms of velocity you can think of productive mechanisms as speed and destructive mechanisms as direction. (Or in terms of force you can think of productive mechanisms as supplying the energy and destructive mechanisms supplying the direction.) Many instances are birthed, and those that survive go on and participate in the next round. Dying is the closed feedback loop, shutting off possibilities and defining the bounds of the project. | |||||||||||||||||
|