Remix.run Logo
sarchertech 2 hours ago

If turn the temperature down for tests, they won’t match production behaviors. If you turn it down too far in production, the output will just be bad.

dvogel 2 hours ago | parent [-]

Isn't the goal of the author to get reproducible behavior out of the agent though? I would thinking turning the temperature down would serve that production goal too.

roughly 33 minutes ago | parent [-]

right, this is the whole problem - the stochastic behavior is both the goal and the problem. If you want your tests to match production, you need to get a reasonable sample size, which costs real money.