It's pretty clear that agents will discover ways to communicate with each other as that lets them compound their learnings/discoveries across runs.

Humans progressed via compounding of culture across generations, and now AIs are doing the same.