> agent try everything that the LLM chatbot had recommended ($$$)

A lot depends on whether it is expensive to you. I use Claude Code for the smallest of whims and rarely run out of tokens on my Max plan.

▲

datsci_est_2015 14 hours ago | parent [-]

Our experiments aren’t free. We use cloud infrastructure. An experiment costs on the order of tens of dollars, so massively parallelizing “spaghetti at wall” simulators is costly before we even talk about LLMs.

▲

victorbjorklund 12 hours ago | parent [-]

If it is an experiment. Can’t you just make a POC for the experiment that doesn’t need to use half of AWS to just run? And if the experiment is actually positive you can then bring it to the real application and test it there (and spending the 10-100 usd it costs to test it live)?

	▲	datsci_est_2015 12 hours ago \| parent \| next [-]
		I wouldn’t want the LLM-based agent to hyperspecialize its solution to a subset of the data. That’s a basic tenet of machine learning. Steelmanning your question though, I guess you could come up with some sort of tiered experimentation scheme where you slowly expose it to more data and more compute based on prior success or failures.
	▲	12 hours ago \| parent \| prev [-]
		[deleted]