For these tests, why not tune the temperature and such to reduce the randomness and convert them to almost-always-succeeds vs almost-always-fails? Is it not the iteration count that drives up the cost?

▲

sarchertech 4 hours ago | parent [-]

If turn the temperature down for tests, they won’t match production behaviors. If you turn it down too far in production, the output will just be bad.

▲

dvogel 3 hours ago | parent [-]

Isn't the goal of the author to get reproducible behavior out of the agent though? I would thinking turning the temperature down would serve that production goal too.

	▲	roughly 2 hours ago \| parent [-]
		right, this is the whole problem - the stochastic behavior is both the goal and the problem. If you want your tests to match production, you need to get a reasonable sample size, which costs real money.