> because their output is non-deterministic by design.

It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.

▲

kennywinker 2 hours ago | parent [-]

That’s true, but not true-true. Sure, every time you prompt “what is the weather in kansas” you’ll get the same output, but if you prompt “what is the weather in kansas right now” you’ll get a different output, and then “what is the weather in kansas today” gets a different output. Language being language, there are infinite ways to say things, so there are infinite variations in what the llm can output in response to very similar prompts.

This tool has a finite amount of outputs for an infinite amount of inputs. Which is different from an llm based tool.

	▲	skeledrew an hour ago \| parent [-]
		I think the point being made is that given a particular input string, you can get a deterministic output string back from the LLM.