What does deterministic mean to you?

In this context, it means being able to deterministically predict properties of the output based on properties of the input. That is, you don’t treat each distinct input as a unicorn, but instead consider properties of the input, and you want to know useful properties of the output. With LLMs, you can only do that statistically at best, but not deterministically, in the sense of being able to know that whenever the input has property A then the output will always have property B.

▲

peyton 2 hours ago | parent [-]

I mean can’t you have a grammar on both ends and just set out-of-language tokens to zero. I thought one of the APIs had a way to staple a JSON schema to the output, for ex.

We’re making pretty strong statements here. It’s not like it’s impossible to make sure DROP TABLE doesn’t get output.

	▲	satvikpendem an hour ago \| parent [-]
		And also have a blacklist of keywords detecting program that the LLM output is run through afterwards, that's probably the easiest filter.

▲

tsimionescu an hour ago | parent | prev [-]

I think they mean having some useful predicates P, Q such that for any input i and for any output o that the LLM can generate from that input, P(i) => Q(o).