Remix.run Logo
evmaki 9 hours ago

> I ask it to reflect on why, and update the Skill to clarify, adding or removing detail as necessary.

We are probably undervaluing the human part of the feedback loop in this discussion. Claude is able to solve the problem given the appropriate human feedback — many then jump to the conclusion that well, if Claude is capable of doing it under some circumstances, we just need to figure out how to remove the human part so that Claude can eventually figure it out itself.

Humans are still serving a very crucial role in disambiguation, and in centering the most salient information. We do this based on our situational context, which comes from hands-on knowledge of the problem space. I'm hesitant to assume that because Claude CAN bootstrap skills (which is damn impressive!), it would somehow eventually do so entirely on its own, devoid of any situational context beyond a natural language spec.

turnsout 6 hours ago | parent [-]

Absolutely. This is why I'm hesitant to go full "dark software factory" and try to build agent loops that iterate in YOLO mode without my input. I spent a day last week iterating Skills on a project by giving it the same high-level task and then pausing it when it went off the rails, self-reflect, and update its Skill. It almost took me out of the loop, but I still had to be there to clear up some misunderstandings and apply some common sense and judgment.