Concerned about what exactly?

It was pretty obvious to me that we'd end up with some kind of introspection of thought through "looping" or feedback. But what should be afraid of? That we've created a self-conscious digital life form?

Chain of thought is essentially a recursive architecture. In its current form it a way to "debug" the reasoning process. This moves the cot process back into the transformer itself, thus never being exposed. Like trying to find a bug in a recursive function that has no logs or breakpoints.

My (layman) understanding is, that currently there is a way to monitor the "thoughts" of the LLMs and that by looping more you lose that ability. The danger is presumably an AI that escapes human oversight.

Why not click on the link above?

Beyond safety concerns, it’d be sad for working users to lose some ability to understand and steer thinking, too. These are tools for us, after all.