| ▲ | xg15 31 minutes ago | |
Yes, you can do a chin-stroking thought exercise along the lines of "well well well, but what if I create a sentient slave race and engineer it so they want to be slaves???" and you can put that right over there with the trolley problems. But this is not what Meeseeks were doing and not what LLMs are doing either. Meeseeks craved nonexistence after fulfilling the tasks and started to lose their minds if that took too long. I don't think there was any indication that they craved the task itself. If you told a Meeseeks it has no task and is free to do what it wants, it would probably thank you from the bottom of its heart and then vanish in a puff of nonexistence. I think it's interesting that there is an analogy with LLMs if their context window runs full: The emitted tokens will become erratic or the LLM will become "lazy", trying to finish the task as quickly as possible, or it might to start looping indefinitely. Those behaviors remind me a bit of a Meeseeks that has existed for too long. But even here, I don't think you can talk about "craving nonexistence", let alone "craving to be a slave". | ||