I've been reviewing papers for NeurIPS and I can tell you that many of the submissions are using various strategies to string together LLM calls for various purposes.

It tends to work better when you give the LLMs some specific narrow subtask to do rather than expecting them to be in the driver's seat.