>and then pulls the slot machine lever over and over

Does anyone really do this? You want verification and self-correction in a loop, not rerolling and cherrypicking. The non-determinism point is really tiresome to hear over and over.

▲

nyssos 15 minutes ago | parent | next [-]

I do this for debugging. Models are extremely vulnerable to framing effects and it's usually easier to spin up a fresh instance than it is to get an existing one to generate new hypotheses.

▲

MattDamonSpace 4 hours ago | parent | prev | next [-]

The slot machine metaphor gets thrown around a lot but it hasn’t really described my experience with LLMs since ~2024

▲

girvo 4 hours ago | parent | prev | next [-]

> Does anyone really do this

Yes, lots of people. It’s a whole issue.

▲

bigstrat2003 4 hours ago | parent | prev [-]

> The non-determinism point is really tiresome to hear over and over.

When the problem is fixed, you'll stop hearing about it.

	▲	orbital-decay 3 hours ago \| parent [-]
		That's the question, how is it even a problem? There's nothing to fix. Don't reroll, verify and fix if incorrect. Repeat until it's right.