I do think there's something to be said for the big difference in the current abstraction layer jump - the loss of determinism. It is meaningfully different.

▲

JumpCrisscross 5 hours ago | parent | next [-]

Plenty of physical (and biological) processes are subjectively non-deterministic outside clean rooms. Doesn’t mean our ancestors couldn’t forge and selectively breed.

▲

mrob 5 hours ago | parent | next [-]

There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.

▲

klabb3 4 hours ago | parent | prev | next [-]

In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it.

I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.

	▲	imoverclocked 4 hours ago \| parent [-]
		You’ve clearly not seen monkeys. /sarcasm I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.

▲

gonzalohm 5 hours ago | parent | prev | next [-]

I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.

	▲	zild3d 4 hours ago \| parent [-]
		> Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming. Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ... Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc. If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.

▲

ajuc 4 hours ago | parent | prev | next [-]

There's a pretty cool article about trying to understand electronics and fix a radio the way we try to understand biology. https://mappingignorance.org/2014/07/02/biologists-cant-unde...

So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.

▲

pydry 4 hours ago | parent | prev [-]

Our ancestors also had an infant mortality ratio of 4/10.

Determinism is way undervalued.

▲

tines 4 hours ago | parent | prev | next [-]

People use the word "determinism" when they really mean something akin to "linearity", i.e. the predictability of a change in input on a change in output. Compilers for example are both deterministic and "more linear" in the sense that I can tell what the output will look like given a change in input (yes yes optimizations violate this to a small degree). LLMs can be made totally deterministic, but a seemingly insignificant change in input can create a drastic change in output, which is the characterstic we don't want.

	▲	eikenberry 2 hours ago \| parent [-]
		The generally used term that I'm familiar with for this is "referential transparency", that given the same inputs you'll get the same outputs every time. LLMs can be deterministic (referentially transparent) but almost none are, i.e. when they are given the exact same input they do not return the exact same output.

▲

eptcyka 5 hours ago | parent | prev [-]

If there is something to be said, I have to ask. What specifically is there to be said?

	▲	zephen 5 hours ago \| parent [-]
		Non-determinism is not inherently bad. We have had useful non-deterministic tools for (in computer years) several generations. Non-determinism combined with bugginess? That's a terrible combination. It is impossible to gradient-descent your way into a working prompt.