Remix.run Logo
niemandhier 2 days ago

My understanding is, that we do not know if the chain-of-thought actually matters in the way we assume for the result.

I think there were experiments where seemingly relevant parts of the CoT were ablated and it did not change the result.

For all we know it might be somewhat human parseable neuralese.

skybrian 2 days ago | parent | next [-]

No guarantees, but it might still be a useful signal even if it’s not causative?

jameshart 2 days ago | parent | prev [-]

Or the neuralese could be hidden steganographically in the chain of thought.