I see this when just using some chat bot that shows the “reasoning” steps (ad-hoc observation of course, it’s really cool that people are actually studying it).

It is annoying when the bot seems be “reasoning” correctly and then makes an obvious mistake at the end. And perplexing when it seems to be completely wrong and then pull the right answer out of a magic hat at the end.

I guess it makes sense; the “reasoning” steps aren’t actually doing logic, just adding more context to influence the final generation, right? But it is weird to see.