You can make a basic one in minutes based on existing open-source models.

Latency won't be that good, but could still work similarly. Simply force the structured output of a LLM to the given schema.

Probably also easy to train because we can use stronget LLMs to generate input/output data, or even synthetic data is easy to generate.

It's not really a new technology, it's more like a new use-case.

▲

sigbottle 4 hours ago | parent [-]

What even are these new "decision models?" Take an existing LLM, feed it a prompt, force it to pick a choice; decode is 1 token (or rather, the whole logit set for only that last token; token implies selecting one logit) so you made a choice. That's it?

	▲	redox99 3 hours ago \| parent \| next [-]
		Yes, although you probably want to calibrate your model if you want the probabilities to actually be meaningful.
	▲	orbital-decay 3 hours ago \| parent \| prev \| next [-]
		Yes but optimized specifically for the purpose. Using that for "decision making" is also not a new use case, but turned out to be new to many people. Which is great, I hope they make something cool with it!
	▲	popinman322 3 hours ago \| parent \| prev [-]
		That's how Cygnet handles it. https://github.com/blockbrain-ai/cygnet-recipe