Thanks for feedback! A couple of things:

Temperature settings only matter for api usage. Nevertheless, stochastic nature of LLM responses does produce a distribution of responses for a single prompt query. It could potentially be a good idea to run the same query couple of times in each iteration of the monitoring tool to get a better look at the distribution of responses.

As for live search, we focus pretty much only on queries that refer to brands or products. Such queries do use web search tool almost 100% of the time so you will not encounter the stale training data issue in our tool.

Happy to discuss it in more details if you want