Remix.run Logo
ipdashc 4 days ago

While this article is a little overenthusiastic for my taste, I think I agree with the general idea of it - and it's always kind of been my pet peeve when it comes to ML. It's a little depressing to think that's probably where the industry is heading. Does anyone feel the same way?

A lot of the stuff the author says resonates deeply, but like, the whole deterministism thing is why I liked programming and computers in the first place. They are complicated but simple; they run on straightforward, man-made rules. As the article says:

> Any good engineer will know how the Internet works: we designed it! We know how packets of data move around, we know how bytes behave, even in uncertain environments like faulty connections.

I've always loved this aspect of it. We humans built the entire system, from protocols down to transistors (and the electronics/physics is so abstracted away it doesn't matter). If one wants to understand or tweak some aspect of it, with enough documentation or reverse engineering, there is nothing stopping you. Everything makes sense.

The author is spot on; every time I've worked with ML it feels more like you're supposed to be a scientist than an engineer, running trials and collecting statistics and tweaking the black box until it works. And I hate that. Props to those who can handle real fields like biology or chemistry, right, but I never wanted to be involved with that kind of stuff. But it seems like that's the direction we're inevitably going.

ACCount37 4 days ago | parent | next [-]

ML doesn't work like programming because it's not programming. It just happens to run on the same computational substrate.

Modern ML is at this hellish intersection of underexplored math, twisted neurobiology and applied demon summoning. An engineer works with known laws of nature - but the laws of machine learning are still being written. You have to be at least a little bit of a scientist to navigate this landscape.

Unfortunately, the nature of intelligence doesn't seem to yield itself to simple, straightforward, human-understandable systems. But machine intelligence is desirable. So we're building AIs anyway.

ath3nd 3 days ago | parent | next [-]

> Modern ML is at this hellish intersection of underexplored math, twisted neurobiology and applied demon summoning

Nah, it's just a very very big and fancy autocomplete with probabilistic tokenization and some extra tricks thrown in to minimize the shortcomings of the approach.

> Unfortunately, the nature of intelligence doesn't seem to yield itself to simple, straightforward, human-understandable systems.

LLMs are maybe artificial but they are not intelligence unless you have overloaded the term intelligence to mean something much less and more trivial. A crow and even a cat is intelligent. An LLM is not.

ACCount37 3 days ago | parent [-]

That's copium.

The proper name for it is "AI effect", but the word "copium" captures the essence perfectly.

Humans want to feel special, and a lot of them feel like intelligence is what makes them special. So whenever a new AI system shows a new capability that was thought to require intelligence? A capability that was once exclusive to humans? That doesn't mean it's "intelligent" in any way. Surely it just means that this capability was stupid and unimportant and didn't require any intelligence in the first place!

Writing a simple short story? Solving a college level math problem? Putting together a Bash script from a text description of what it should do? No intelligence required for any of that!

Copium is one hell of a drug.

ath3nd 3 days ago | parent [-]

> Copium is one hell of a drug.

What is the word for creating an account 12 days ago and exclusively defending the LLMs because they can't defend themselves?

> Writing a simple short story

Ah, allow me to introduce you to the Infinite Monkey theorem.

https://en.wikipedia.org/wiki/Infinite_monkey_theorem

In the case of LLMs it's just the monkey's hand is artificially guided by all the peanut-guided trainings it was trained on but it still didn't use a single ounce of thought or intelligence. Sorry that you get impressed by simple tricks and confuse them for magic.

ACCount37 3 days ago | parent [-]

And this proves what exactly? That any task can be solved by pure chance at pass@k, with k blowing out to infinity as the solution space grows?

We know that. The value of intelligence is being able to outperform that random chance.

LLMs already outperform a typewriter monkey, a keyboard cat, and a non-insignificant amount of humans on a very diverse range of tasks.

voidhorse 4 days ago | parent | prev [-]

You should read some of the papers written in the 1940s and learn about the history of cybernetics. Your glowing perception of the "demon summoning" nature of ML might change a bit.

People want others to think this tech is mysterious. It's not. We've known the theory of these systems since the mid 1900s, we just didn't fully work out the resource arrangements to make them tractable until recently. Yes, there are some unknowns and the end product is a black box insofar as you cannot simply inspect source code, but this description of the situation is pure fantasy.

ACCount37 4 days ago | parent [-]

Good luck trying to use theory from the 1940s to predict modern ML. And if theory has little predictive power, then it's of little use.

There's a reason why so many "laws" of ML are empirical - curves fitted to experimental observation data. If we had a solid mathematical backing for ML, we'd be able to derive those laws from math. If we had solid theoretical backing for ML, we'd be able to calculate whether a training run would fail without actually running it.

People say this tech is mysterious because it is mysterious. It's a field where practical applications are running far ahead of theory. We build systems that work, and we don't know how or why.

skydhash 3 days ago | parent [-]

We have solid backing in maths for it. But the fact is what we are seeking is not what the math told us, but an hope that what it told us is sufficiently close to the TRUTH. Hence the pervasive presence of errors and loss functions.

We know it’s not the correct answer, but better something close than nothing. (close can be awfully far, which is worse than nothing)

ACCount37 3 days ago | parent [-]

The math covers the low level decently well, but you run out of it quick. A lot of it fails to scale, and almost all of it fails to capture the high level behavior of modern AIs.

You can predict how some simple narrow edge case neural networks will converge, but this doesn't go all the way to frontier training runs, or even the kind of runs you can do at home on a single GPU. And that's one of the better covered areas.

skydhash 3 days ago | parent [-]

You can’t predict because the data is unknown before training. And training is computation based on math. And the results are the weights. And every further computation is also math based. The result can be surprising, but there’s no fairy dust here.

ACCount37 3 days ago | parent [-]

There's no fairy dust there, but that doesn't mean we understand how it works. There's no fairy dust in human brain either.

Today's mathematical background applied to frontier systems is a bit like trying to understand how a web browser works from knowing how a transistor works. The mismatch is palpable.

Sure, if you descend to a low enough level, you wouldn't find any magic fairy dust - it's transistors as far as eye can see. But "knowing how a transistor works" doesn't come close to capturing the sheer complexity. Low level knowledge does not automatically translate to high level knowledge.

crabmusket 4 days ago | parent | prev | next [-]

I feel similarly to you.

Even amidst the pressure of "client X needs feature Y yesterday, get it done with maximum tech debt!" it felt like there were still chances to carve out niches of craft, care and quality.

The rise of "probabilistic" software feels like it is a gift to "close enough is good enough" and "ship first, ask questions later".

If only we all became scientists. We'll more likely become closer to tabloid journalists, producing something that sounds just truthy enough to get clicks.

Legend2440 4 days ago | parent | prev | next [-]

> They are complicated but simple; they run on straightforward, man-made rules.

The trouble is that many problems simply cannot be solved using rules. They're too complex and undefinable and vague.

We need systems that are more flexible and expressive to handle the open-ended complexity of the real world.

ethan_smith 4 days ago | parent | prev [-]

The deterministic and probabilistic paradigms will likely coexist rather than fully replace each other - we'll build deterministic interfaces and guardrails around probabilistic cores to get the best of both worlds.