I am very interested to hear others' perspectives on this topic:

Should we integrate experimental learnings by evolving the agent harness, or do RL-based post-training, and leave it in LLM's weights? Or Both?

▲

rraithel27 3 hours ago | parent [-]

I think evolving the harness makes the most sense. Models get swapped as new ones come along, and anything learned into weights would be lost. Lessons in the harness can carry over.

I work in a regulated industry and traceability is extremely important. This includes explaining why the system did something and being able to undo what it learned. A lesson with its evidence attached can be traced and deleted, a weight change not so much.

Can you trace a result back to the lessons that produced it?

	▲	alirezamshi 2 hours ago \| parent [-]
		Great point, it makes even more sense in regulated industry. They can better trust an AI system that transparently writes and executes the learnings rather than being in model weights with no transparency. Regarding your question, Yeah, in our bank of lessons, each lesson card has credence score, boundary (where it works), and full evidence of experiment traces. As you run kapso, these credence score and boundary get better with more evidence from experiments. By any chance, did you encounter any use-cases in these industries, that we can try Kapso?