Remix.run Logo
▲ sheepscreek a day ago

CAVEMAN SUMMARY: AI no need theory, only people need. Fable understand OpenAI math. Fable explain well to human. People the problem, not AI.

I was reading a mathematician's take on a recent OpenAI proof (their area of expertise). They argued that models don't lack theory building, they just work at such a high level of abstraction that they don't need it.

Theory is for us, so we can understand what the models are doing. They gave the example of Fable, which understood and explained everything perfectly right away (written by OpenAI's unreleased model in terse mathematical notiations).

This was totally intuitive for the AI (Fable), but it took this person two weeks just to get a high-level grasp of it. And this person is one of the few people on earth who can (he is an expert in this niche of mathematics).

▲brandall10 a day ago | parent | next [-]

I imagine more semantically comprehensible proofs benefit LLMs as much as they benefit humans, from a time/energy/iq perspective.

Ergo, there are likely numerous deeper insights that will emerge when smarter models have the chance to go over brute forced, millions of line Lean proofs. Likely things humans have not conceived, but I believe will truly advance understanding. Post-train the existing models with that knowledge, and see them resolve these same problems with far less effort.

Just a hunch. I'm neither an AI researcher nor a mathematician.

▲gf000 2 hours ago | parent | prev | next [-]

Not trying to question it at all, but I am quite confident that LLMs will always require abstractions, no matter what.

Like math can be made arbitrarily more complex and no matter what we do, an LLM has at most n parameters it can use to "think" (in principle, a singularity that in this hypothetical scenario realize that it is not smart enough for properly encoding the problem and will rather create a better AI than itself to solve it could be possible).

"Resetting" this hard limit at the one minus abstraction level is a must have.

▲TeMPOraL 5 hours ago | parent | prev | next [-]

Reminds me of the Bicameral Order from Blindsight / Echopraxia books by David Brin, where you had post-humans operating on such a different level of intelligence that you needed a translation layer between them and the rest of humanity just for "regular" humans to have a chance at comprehending even a little of the mathematical and scientific discoveries they were making.

▲Kotlopou a day ago | parent | prev | next [-]

Just for the benefit of others, are you referring to this?

https://web.maths.unsw.edu.au/~davidharvey/research/subnlogn...

▲sheepscreek a day ago | parent [-]

Yes! Thank you for linking it here. I couldn't find it for the life of me.

▲pegasus 4 hours ago | parent [-]

The conclusion you drew from that article is the opposite of what the article claimed, though. Check my comment here: https://news.ycombinator.com/item?id=50592554

▲strogonoff 13 hours ago | parent | prev | next [-]

Humans experience reality, and as they do so they gradually build maps—models—that allow them to reason about it with less overhead. Those maps are hopefully useful for some purpose, but also necessarily wrong/incomplete.

LLMs only operate on our pre-existing maps in their training dataset. (The first “L” is the key: language is itself a map.) They may be useful at refining those models, but they do not experience and therefore cannot get us to the next peak over.

▲hardbass 13 hours ago | parent [-]

I love how you wrote these half assed ambiguous/incoherent theories right under a thread where ai's made some of the most immense mathematical developments in perhaps a century.

▲pegasus 4 hours ago | parent | next [-]

I chalk up your rude and dismissive reply to AI psychosis. These achievements you mention are vastly overblown, and they are not at all "immense mathematical developments". There's a saying among mathematicians: "Good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians write definitions (aka build theories)". This is what GP was gesturing at, and it's completely accurate. AI doesn't build theories, regardless of GGP's paraphrased quote, which was incorrect: in fact the mathematician was saying just the opposite. See my comment here: https://news.ycombinator.com/item?id=50592554

It just so happens that in the past minds that were able to crack difficult problems happened to also be the ones who could advance mathematics by breaking new ground and posing new and fertile questions. Same as how before the invention of the calculator, people who could calculate pi to the nth decimal were also people who also had other math skills calculators never could touch.

▲hardbass 2 hours ago | parent [-]

Its a great statement to keep in record. You just said on record that proving Generalized Riemann Hypothesis as "vastly overblown".

I would suggest anyone claiming a work of this monumentality is "vastly overblown" is much better said to suffer from ai psychosis than whoever he thinks is.

▲Ygg2 11 hours ago | parent | prev | next [-]

It's not incoherent. It's describing an abstraction ladder.

It's how you graduate from 1+1+1 to 3*1. And so on.

▲hardbass 10 hours ago | parent [-]

Certainly, I am fully in support of more forms of data being added as the native mode of thought. Visual and tactile and audio based data would be necessary next step for physical robots.

▲strogonoff 12 hours ago | parent | prev [-]

> I love how you wrote these half assed ambiguous theories right under a thread where ai's made some of the most immense mathematical developments in perhaps a century.

Rest assured, not nearly as much as I love how someone on such a respected forum runs out of arguments in their first reply to me.

▲hardbass 11 hours ago | parent [-]

Because you made up incoherent undefined nonsense, and in your previous argument chain literally had to ignore reality to salavge your points. You would have had to define what you mean by experience and explain why sense organs that accept text are not sense organs while sense organs that experience visual and force and audio data are senses. Modela are multimodal in any case these days. And yes I am certainly rooting for ai to debe developed that also traina in visual, audio and force data natively along with language. Wouls be fun to see the next wave of copes and excuses aftet that.

▲hardbass 8 hours ago | parent [-]

s/rooting for ai to debe developed that also traina/rooting for ai to be developed thats also trained

▲pegasus 5 hours ago | parent | prev | next [-]

That's an incorrect interpretation of what that mathematician said, which was in fact the opposite: the models have such power of computation that they can just brute-force a problem without needing to climb a ladder of abstraction. Which explains why the Lean proof to the GRH is 200k page and about 17 million lines long.

Here's an in-depth explanation from Gemini Neon: https://share.gemini.google/lPWabl6Goaxc

The question is, do the models actually understand what they are doing? I would say no, though they can be used to understand the proof later. They are just blindly tunneling through the state space and brute-forcing their way to the solution. With some very unwanted side-effects on the environment to boot.

▲hardbass 2 hours ago | parent | next [-]

Here is the supposed quote from the mathematician in the chat linked above btw:

"Incidentally, when I say that the math(s) dump 'does not appear to be strong on exposition', all I mean is that I, as a human expert in this field, have difficulty understanding it... But as far as Fable seems to be concerned, the exposition in [TC26a] is perfectly clear. So perhaps the problem is not with the exposition, but with us humans. ... Perhaps we need to ask the LLMs to 'explain it to me like I'm human' (ELIH)."

"pegasus"'s full prompt was (minus the hyphen line separators which I added):

----------------------------------------------------------------------------------

I came across this paraphrased quote today: "I was reading a mathematician's take on a recent OpenAI proof (their area of expertise). They argued that models don't lack theory building, they just work at such a high level of abstraction that they don't need it.", which is said to come from this article: https://web.maths.unsw.edu.au/~davidharvey/research/subnlogn...

Read the article and explain this further. My question is: isn't a mathematical theory just that—a higher level of abstraction?

----------------------------------------------------------------------------

I leave it to the reader to think how this statement is the "opposite", and to think what "merely" a higher level of abstraction entails with respect to a thinker's abilities to work such concepts out.

▲hardbass 2 hours ago | parent | prev [-]

If brute force is so easy, why aren't you able to do the same without AI just writing the brute force solver yourself?

▲prmph 15 hours ago | parent | prev | next [-]

This is weird, so you think letting agent loose with "solve cancer, and giving them unlimited compute, with no further guidance, will yield results??

People seems to have lost their sense of judgment. It was the same with Deep Blue and AlphaX, the world's problem did not end. If we are to make progress in many field, it is still people who will have to build new theories, etc.

▲alienbaby 21 hours ago | parent | prev | next [-]

Then It's just a case of unambiguously defining the question, and getting the model to explain the answer.

I swear, if it says 42, I'll be laughing.

▲teiferer 15 hours ago | parent | prev | next [-]

> This was totally intuitive for the AI (Fable)

I'm regularly getting replies from a Fable-related model about our code base and problem space which would suggest that it totally inuitively groks it when it confidently explains and asserts things, and more often than I'd like it turns out that it's wrong, the thing is actually different and/or more complicated than what it made it out to be, and when I point that out it apologizes and agrees with me, rinse and repeat.

I'd be very careful with such claims. LLMs have been around for long enough for us to still fall for its confidence.

▲scribu a day ago | parent | prev [-]

I thought theory building WAS the process of creating powerful abstractions?

▲TheOtherHobbes a day ago | parent [-]

Open question - are these proofs brute forcing, or are they generating new theory at a level of abstraction and complexity we can't understand?

Controversial statement: I think we've always done math by generalising from concrete experience. So concepts like numbers, spaces, shapes, distances, and curvature are based on physical experiences we use as anchors before expanding them far beyond physicality and looking for patterns in the abstractions.

If AI is generalising from patterns in latent space, that's a different kind of abstraction, and one that's much more alien to us, because it's going to be much, much bigger - too big to fit inside even the smartest human mind.

So AI theories and abstractions may be valid in true/false terms, but we will never be able to comprehend them in their raw form - just as we can't experience latent space directly, and will only experience tiny slices of it as outputs to our queries.

▲agnosticmantis 17 hours ago | parent [-]

> If AI is generalising from patterns in latent space, that's a different kind of abstraction

How's latent space conceptually different from neuronal activations in our brains?

> just as we can't experience latent space directly,

Your LLM also doesn't "experience" their activations in the latent space either when generating tokens.

I'm not saying LLMs are not conscious/sentient/etc. Just saying if you ask your LLM which activation was stronger that led to a particular solution, they won't be able to answer without using external tools to inspect it.

It may just be that LLMs have such a huge context window (short-term memory) compared to humans, that they can process a lot more raw data without having to compress them into theories yet.

▲teiferer 15 hours ago | parent [-]

Human short-term memory and LLM context windows are very different things.

More generally, your mental model of how the human brain works seems to be very much aligned with how LLMs work and I would very much caution against that.