Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ a2ff6eeb0 6 hours ago

Does it matter? It's going to produce proofs far more intricate than humans can understand, outdoing humans and opening new frontiers.

The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

▲chongli 6 hours ago | parent | next [-]

We don’t trillions of dollars in LLM investment to build things mathematicians don’t understand. We already have plenty of those, even from ancient times.

As to your second point, Terry Tao already has an answer [1]: the proof isn’t the contribution, shared understanding is. This issue was already raised back when the four-colour theorem was proved. Machine proving and machine proof checking are useful tools but they don’t mean anything without the interpretative work and the communication necessary to build shared understanding.

[1] https://news.ycombinator.com/item?id=49056620

▲atleastoptimal 5 hours ago | parent | next [-]

If humans have nothing to contribute then shared understanding is a pointless endeavor. It makes sense now in the "centaur" period where human + AI > AI alone, but when AI mathematicians are both more rigorous and more elegant, then taking the time dumbing down their proofs to a human level of understanding is like requiring that we ensure all our current proofs be understandable by a monkey.

▲nickysielicki 4 hours ago | parent | next [-]

> If humans have nothing to contribute then shared understanding is a pointless endeavor.

I agree, but as a software engineer this gives me pause because I keep trying to insist on coding standards but I’m unable to come up with a compelling reason why it matters. Ostensibly the reason we cared about things like DRY and code quality was so that it would be easy to understand and easy to maintain and easy to make changes to later. But it now seems like a shared understanding of the codebase is less important than ever, and it’s more about shoveling requirements in without breaking any existing functionality.

Is a well tested slopfest better? That seems to be the conclusion for mathematics, so why not software too?

▲amdgg 2 hours ago | parent | next [-]

> easy to make changes to later

IMO it's still a problem with LLMs; we still have to build in a way that makes it easier for an LLM to make changes later and arguably it's the same things that made software development easier for humans. IME LLMs tend to not know how to do that for themselves and instead just amplify/copy patterns that already exist.

If an LLM can't pave the way for itself then ultimately shared understanding is required to take advantage of LLMs in the first place.

▲KolmogorovComp 4 hours ago | parent | prev [-]

It used to be the same with assembly. Programmers complained the one generated by compilers was not pretty, but now in 99.999% of the cases, it does not matter because nobody look at it.

▲Krei-se 3 hours ago | parent [-]

I beg to differ because a compiler is deterministic.

▲sjjsjsjjsjaj 3 hours ago | parent [-]

Did you check? Do you care if it sometimes does mov ax, 0 or sometimes xor ax,ax? (Forgive my bad memory, it was a long time ago)

Would you personally vouch, at your job, for the importance of proper assembly coding standards?

▲0c3ca83 an hour ago | parent | next [-]

Yes, the same compiler generates the same output given the same input. I'd be willing to put a large amount of money on that result.

I read a lot of assembly.

▲Krei-se 3 hours ago | parent | prev [-]

I was more after the fact i can trust the compiler to give me the same result - even though via an optimized path.

Certainly scopes vary, but in my line of work i define memory layout and how this data will be processed myself - thus it's great a compiler might do that, but the result of the computation will not change.

Now in comparison giving an LLM specs ... i a) cannot be sure what the computation will be b) it might be something else on another run.

▲chongli 4 hours ago | parent | prev | next [-]

Where is the value in an unintelligible gibberish proof?

We already have countless examples of such filling up the arXiv, written by hacks long before LLMs started writing proofs. No one cares about them. You might as well build a box blasting radio static into the void. You could save a lot of electricity that way.

▲atleastoptimal 2 hours ago | parent [-]

An unintelligible but correct proof is better than no proof. These first AI proofs may be overly complex and un-elegant, but they are the worst that frontier math proofs will ever be. AI math in 2030 will be leaps and bounds ahead of humans both in rigor and elegance.

▲chongli 11 minutes ago | parent [-]

Those are some strong claims. I’m deeply skeptical of all of them.

I believe Terry Tao when he says the bottleneck will no longer be the writing of proofs, it’ll be everything else: reading them, reviewing, publishing, and teaching from them. A bunch of proofs that nobody reads are of no use to anyone.

▲donbox 5 hours ago | parent | prev [-]

But but but... Does not AI exists (or has to exist) to only serve us.

▲a2ff6eeb0 6 hours ago | parent | prev | next [-]

Why not? We build cranes to hoist weights construction workers can't lift. We build electron microscopes to measure things physicists can't see.

Why is it so hard to imagine we can build tools to think thoughts we can't comprehend?

If there's commercial value, I think it's inevitable. We don't fund mathematicians because it's cute when they understand a problem, but because their work tends to have applications with commercial value. The value can be captured without understanding the details.

▲chongli 5 hours ago | parent [-]

Can you give an example of an incomprehensible piece of writing (any writing, never mind a proof) that has commercial value commensurate with the costs involved here?

▲akiselev 5 hours ago | parent | next [-]

> Can you give an example of an incomprehensible piece of writing (any writing, never mind a proof) that has commercial value commensurate with the costs involved here?

It depends on what exactly you mean by "commercial value commensurate with the costs involved" but I'd volunteer the 3G/4G/5G specifications and the other documentation required to implement the mobile network protocols. 5G is currently sitting at over 50,000 pages and it's one of the reasons Qualcomm/Broadcom/Apple are the only ones who can realistically make a mobile radio.

I don't think there is a single human to whom more than a few thousand pages would be comprehensible at a time except for the occasional genius.

▲chongli 4 hours ago | parent [-]

Read the rest of the discussion. The claim is about text which is in principle incomprehensible to humans.

If there exists a text which only one person can understand, that person can communicate their understanding to others, even if that doesn't help them with the original text. That dissemination of knowledge is what provides the value, not the mere existence of the text. If that person forgets or dies before they can share their knowledge then it will be lost.

We have many examples of this from history: ancient texts written in a lost language. These texts provide us with no value until the day they can be deciphered, unless you count linguistic puzzle-solving as a virtue.

▲a2ff6eeb0 4 hours ago | parent [-]

> If there exists a text which only one person can understand, that person can communicate their understanding to others, even if that doesn't help them with the original text.

If there exists a text which only an AI can understand, that AI can communicate their key conclusions to others, even if that doesn't help them with the original text.

The only difference here is the amount of meat involved. Perhaps tossing a few steaks on the server racks could help with that.

▲chongli 4 hours ago | parent [-]

If an AI can communicate its findings to us in a way that we can understand it, then its findings are by definition intelligible to humans. Your claim was about texts for which this is not possible.

▲a2ff6eeb0 4 hours ago | parent [-]

So, you're willing to claim that a conclusion that you don't understand the reasoning for is equivalent to comprehension?

Because, again, I can point to hundreds of examples of texts where nobody but the author understands it, and they're only giving summarized "commandments" that you should follow if you want good results.

If I told you "don't use spin locks, call futex instead", do you think have gained an understanding of the Linux scheduler?

▲chongli 2 hours ago | parent [-]

If I told you "don't use spin locks, call futex instead", do you think have gained an understanding of the Linux scheduler?

Yes, because I already knew what schedulers are, what spinlocks are, and if I want to know what futex is I can go look it up. Comprehension is within my grasp.

Your original claim, which you’ve repeatedly distanced yourself from (by trying to use comprehensible examples) but won’t admit to, was about incomprehensible stuff. That is, text that no human could possibly understand, ever.

You’re repeatedly engaging in intellectual dishonesty rather than simply admit that “human comprehension probably will continue for the foreseeable future”, which is really not a controversial idea at all.

▲a2ff6eeb0 an hour ago | parent [-]

Bullshit. Human comprehension will not continue, it will reduced to "when the AI tells us to add Bismuth, strontium, and copper together in this way, we get a superconductor", but that doesn't mean we understand high temperature superconductivity.

For what it's worth, we've known about BSCCO for nearly 40 years, and we still don't have a great physical understanding of how it works, thought we made decent progress in 2022.

▲chongli an hour ago | parent [-]

That’s still comprehensible. Incomprehensible chemistry instructions simply could not be followed by any human. They would be of no use to anyone.

▲a2ff6eeb0 24 minutes ago | parent [-]

You're using a very bad definition of incomprehensible. We don't understand the chemistry behind how these things work, it's that simple. If you're going to argue it's comprehensible, you'd better show how it works and go collect your Nobel prize.

All we know is that if we melt the right rocks together, we get a superconductor that works in some mysterious way that we can't explain. We know it's not Cooper pairs.

▲a2ff6eeb0 5 hours ago | parent | prev | next [-]

Yeah, how many people do you think understand the Linux kernel in full? What percentage of the people using it to great commercial effect can understand it?

How's your understanding of Schroedingers "An Undulatory Theory of the Mechanics of Atoms and Molecules"? You seem to be using the results of it as applied to semiconductor engineering just fine. And, I promise you, most semiconductor engineers haven't read it in full, they just accepted the results as passed on by several layers of teacher.

I have a paper on routing algorithms, which I have attempted to read to my cat. I don't think my cat retained much, but they seem to be enjoying the cat food that got delivered using the results.

I'd suggest that we're going to be a lot closer to the cat than the author of the paper when AI takes off.

▲chongli 5 hours ago | parent | next [-]

You've moved the goalposts. The original claim was about producing mathematics that are in principle impossible for any human to understand.

All the stuff you've listed is understood by some person, and that understanding is the source of its value.

Now that we've cleared that up, can you furnish an example that satisfies the original claim of incomprehensibility and value?

▲a2ff6eeb0 5 hours ago | parent | next [-]

You asked for examples, for a technology that we're still building. Maybe you can see the issue with that?

Anyways, people benefitted greatly from Newton's laws of gravity, even though we still don't have a quantum-compatible set of laws for it. The laws of gravity are still incomprehensible for people, but the approximation that we've observed is still immensely valuable.

▲chongli 5 hours ago | parent [-]

No, I gave you a lot more leeway than that. Take any utterly incomprehensible piece of writing from the entire history of civilization and demonstrate its value.

You keep falling back on "incomprehensible for some people" but that wasn't the claim. It was about a text which is incomprehensible in principle; that is, utterly impossible for any human to ever understand.

▲eks391 4 hours ago | parent | next [-]

His examples work fine but you aren't accepting them because they don't confine to your paradox.

Every writing must be comprehensible to at least the author, regardless of whether it has commercial value or not. If I hit the keyboard a few times, I've created writing, but it doesn't mean anything. It is just gibberish and without meaning, so there is nothing to try to comprehend. So if there is something to be comprehended, then at minimum the author should know it.

Therefore what you keep claiming is the only refute of your argument of "an [...] incomprehensible [...] writing" is actually a paradox, and cannot be disproved itself. However "humans comprehending things" is not a paradox, which means that your specific request to beat your paradox is not actually related at all.

His examples disproving the non-paradox version of your challenge (writing incomprehensible to folks other than the original authors) are sufficient to disprove your statement, as he gave examples of both people not comprehending human made and 'God' made writing (the universe/gravity)

▲chongli 4 hours ago | parent [-]

That's because his examples are obvious and not at all interesting, since we've already seen them.

His original claim amounts to creating an AI that takes its place above humans as some kind of electronic God, delivering edicts to humanity that we cannot comprehend, but which somehow have value to us. It's unskeptical, pseudo-religious nonsense.

▲rowanG077 3 hours ago | parent [-]

I don't see that at all. Nowhere did this person evoke placing AI above people or even remotely evoke religious imagery. You imagined that yourself, and that creeps me out.

▲chongli 2 hours ago | parent [-]

It’s right here:

The age of humans comprehending things is coming to an end [1]

[1] https://news.ycombinator.com/item?id=49312914

▲rowanG077 2 hours ago | parent [-]

That does not place AI above humans and neither does it evoke anything religious.

Look at it this way. AI has destroyed humanity at chess for decades now. That doesn't mean we have placed AI above humans in the general sense.

You have basically totally fabricated what you thought the other person was saying.

▲ 4 hours ago | parent | prev | next [-]
[deleted]
▲a2ff6eeb0 5 hours ago | parent | prev [-]

Can you give me an example single piece of writing that the author didn't claim they understood? I don't think that any utterly incomprehensible writing exists.

If you want examples where nobody but the author understands it, examples are a dime a dozen.

▲chongli 4 hours ago | parent [-]

So you concede the point then. The age of humans comprehending things is not coming to an end. And therein lies the rub. When it comes to intellectual labour:

Understanding == Value

If a mathematician produces something incomprehensible then it has no value. It's meaningless. Indistinguishable from random noise.

An AI which produces incomprehensible text is producing no value. We didn't need to spend trillions of dollars on LLMs to figure that out. Markov chains can do that job perfectly well.

▲a2ff6eeb0 4 hours ago | parent [-]

You may have missed the present tense. I didn't say it's over, but that it will be. Please pay attention and don't make straw men.

Again, do you believe that there are documents, of any value, that humans don't understand?

Maybe an LLM could help you notice what I was saying, since it's clearly beyond at least one human's comprehension!

▲chongli 4 hours ago | parent [-]

Again, do you believe that there are documents, of any value, that humans don't understand?

There is no value in an undeciphered document until understanding is achieved, just as a lode of gold ore in some asteroid orbiting a distant star has no value until we can fly there and extract it.

If an LLM can help us understanding something then it was not incomprehensible, by definition.

▲ 3 hours ago | parent | next [-]
[deleted]
▲calf 41 minutes ago | parent | prev [-]

But this is just arguing over word tenses. You think "cannot", they think "could". Somewhere between there is the actual crux of disagreement.

▲sjjsjsjjsjaj 3 hours ago | parent | prev | next [-]

Nothing is in principle impossible to understand. It just takes too long, is inconvenient and/or economically unviable.

I don’t find it hard at all to imagine that an AI comes up with a fundamental proof applicable to physics which results in some widget we can now produce that would otherwise not have been produced yet nobody takes the time to fully comprehend why it works. Somebody could, in principle, devote their lives to it and possibly get it, but for what purpose?

▲scarmig 4 hours ago | parent | prev [-]

There are plenty of artifacts that, if not impossible for humans to understand, then at least no human has ever completely understood. To start with, the universe as a whole. Despite that, we are able to choose legible pieces of it to model and perform useful actions from.

This will shift your argument--that doesn't count! etc., to the point where it's by construction unsatisfiable and vacuous. And it doesn't matter: an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.

▲chongli 4 hours ago | parent [-]

One person need not completely understand something for it to have value; it's sufficient for there to exist a shared understanding.

an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.

No, that is the entire point. If it is an algorithm which accomplishes something useful, then it is intelligible as such. That which is incomprehensible cannot be understood even in part, so it provides no value as a bit of knowledge (unless you're looking for a strong random number source, I suppose).

▲scarmig 4 hours ago | parent [-]

So, what you're asking for is an example of someone extracting value out of something that is fundamentally irreducible random noise?

▲bluefirebrand 5 hours ago | parent | prev [-]

> how many people do you think understand the Linux kernel in full?

"Not many people understand some things fully" is so massively different from "the human mind is incapable of understanding some things that AI will understand for us"

▲a2ff6eeb0 5 hours ago | parent [-]

Yeah. We're only starting the journey of building tools better at thinking than the human mind, so expecting me to have examples of things it produces is a little hard, don't you think?

The best I can do is things that are incomprehensible to nearly everyone, but still provide value. There's a small leap of imagination to consider an author that understands it and can show others how to leverage results without understanding be mechanical rather than biological.

▲nickysielicki 4 hours ago | parent | prev | next [-]

Can you give an example of a 4-wheeled vehicle that could move at tens of miles per hour before the automobile was invented?

▲lowbloodsugar 5 hours ago | parent | prev [-]

I know right? And there’s only a market for maybe 5 computers in the whole world.

Paul Krugman (1998): predicted the internet’s economic impact would be no greater than the fax machine’s.

The 1876 Western Union memo dismissing the telephone as having too many shortcomings, and the banker telling Horace Rackham not to invest in Ford because the automobile was a novelty.

We are in good company!

▲orphereus 5 hours ago | parent [-]

Past performance does not guarantee future returns.

▲tossandthrow 4 hours ago | parent | prev [-]

The idea about the goal of mathematics being shared understanding seems to come at a convenient time.

Mathematicians have never been known to communicate their ideas very clearly.

Regardless, even that target llms will likely win - an llm will likely be more efficient at teaching me string theory than a professor in a room with 463 other students.

The llm is the shared understanding.

▲cauch 3 hours ago | parent [-]

I don't think "shared knowledge" means "shared knowledge between mathematicians and lay persons" (there is no much point in that, the same way a smartphone technician knows how a smartphone works deep down to the details but there is no big interest for society to have every lay persons being informed about it). I think it means "shared knowledge between mathematicians".

And at this level, while there are anecdotical exceptions, mathematicians have always been pretty decent (with their conferences, workshops, paper publications, international collaborations, ...).

So, it does not mean "teaching the subject", it means "creating a human network of people that share the understanding". LLM can be useful at telling a human, but you still need a human. The point of Tao is not that LLM is not good at providing explanations, it is that "providing explanations" is not the contribution to science, "the human network" is. It's like saying "LLM are great cook, they generate tons of food in space", but the point of having cooks is so that people can eat food and not die. Having LLM generating mathematical proofs is as useless as having LLM generating food that no one can access: the point was never to "generate proofs" or "generate food", the point was "creating a shared human understanding" or "eating the food so human can survive".

▲tossandthrow 3 hours ago | parent [-]

I don't get the argument.

I get that there is a cultural benefit to keeping it alive. Just like we ideally want the languages represented at the universities.

But keeping humans in the loop does not appear to be necessary in order to call it science, and certainly not in order to have progress or dessiminate that progress.

I don't have a problem with people doing math. As long that we don't idiomatically hold on to that way of doing things.

I do, however, find it hard to belive that individual humans will play a big role from here and forward, in any scientific desciplines.

▲cauch 2 hours ago | parent [-]

The goal is indeed to have progress or disseminate that progress.

The point of Tao is that people see LLM providing "proofs" and are concluding that this is all that is needed to "have progress or disseminate that progress". That is the same mistake of thinking that "generating food" is all that is needed to "have people not dying of hunger".

The hard part of "have progress or disseminate that progress" is the human network. A fundamental point of this human network is that it generate trust, accountability and reliability. Generating "useful new theorem" is useless unless the society also built the trust around the theorem to distinguish it from a fake theorem.

Maybe in the future, we will have AI doing some part of it, but this is a totally different AI animal than the one we are able to have now, and people who think the current AI that we see now is able to do that have no understanding how it works. This is demonstrated by the facts in math: current AI is able to provide math proofs, and yet, a lot of human work is still needed to get progress out of current AI.

I would not bet that individual humans will still play a role as big as today in the future. Maybe AI will be different in the future, but the reality is that we don't have any indication if this is even possible.

▲fsmv 6 hours ago | parent | prev | next [-]

The entire point of writing proofs is for advancing human understanding. A giant dump of symbols that passes the lean compiler is meaningless besides human beings understanding it.

▲variadix 5 hours ago | parent | next [-]

In the field of pure mathematics this might be true, but it has implications regardless for applied math, engineering, and physics.

▲pfdietz 5 hours ago | parent | prev | next [-]

> The entire point of writing proofs is for advancing human understanding.

Proofs also enable AIs to direct search and generate knowledge. Verifiability is immensely useful for keeping AI grounded.

One might imagine AI generating enormous numbers of hypotheses and then trying to prove or disprove them, and then mine that data for new abstractions and heuristics.

▲Jtarii 5 hours ago | parent | prev | next [-]

An AI may still be able to apply the results without humans understanding the proof.

▲nullc 5 hours ago | parent | prev | next [-]

Sometimes the purpose of the proof is simply to demonstrate that some construct is a safe assumption for other more interesting work-- and could still serve that purpose even if it was entirely a black box.

▲dyauspitr 4 hours ago | parent | prev | next [-]

No it isn’t, it’s putting it into the corpus which means another LLM doesn’t have to spend a few billion credits the next time.

▲esafak 5 hours ago | parent | prev | next [-]

Is it the AI's fault we can't understand? If the GUT is beyond human comprehension does it matter less? We don't apply this reasoning to other animals or even to less capable humans. Besides, the robots may want to ponder maths for their pleasure.

▲nickysielicki 4 hours ago | parent | prev [-]

was. Not is. Was.

▲orphereus 6 hours ago | parent | prev | next [-]

"The age of humans comprehending things is coming to an end"

That's something AI companies would really want you to believe.

▲ianm218 6 hours ago | parent | next [-]

> That's something AI companies would really want you to believe.

Why would I care what they want me to believe?

Intuitively it would make sense that you can put math ability on a chart with a value for “general public” “smart high schooler” “smart undergrad” “smart PhD/ professional”. And you could place frontier AI somewhere on that chart over time from GPT 2 to now and see the trend.

Then you’d have to consider that either you believe there is a fundamental limit that is below peak human mathematician level or there’s not.

▲orphereus 6 hours ago | parent [-]

> Why would I care what they want me to believe?

How would you not care? Are you a robot?

They can say random stuff with the goal of increasing their shareholder value. Things they spit out do not have to be true. It is not easy to verify things they say, therefore, everything they say should be taken with a huge grain of salt.

▲ianm218 5 hours ago | parent | next [-]

Consider Enron and Amazon at the turn of the millennium. They were both telling you what the future would look like. The right action would’ve been to just ignore what they are saying and try and get data and reason about the world. It didn’t really matter that both Bezos and Jeff Skilling wanted you to believe various things - one was right and one was a scammer.

So that’s what I’m doing here. For what it’s worth I find a lot of the AI people’s worldview very consistent. They believed AI would be the most important technology of our life times and committed their work to it. Some of these same people are total liars so yeah I won’t really hang onto their every word.

▲esafak 5 hours ago | parent | prev [-]

He's saying he can connect the dots without their help.

▲orphereus 5 hours ago | parent [-]

Yeah, my bad.

▲jambalaya8 5 hours ago | parent | prev | next [-]

that does not make it not true, nor does it make those companies or their products not dangerous. I like looking at videos of animals that tear other animals apart and eat them; lion cubs are super cute; but that does not mean I want to be thrown into a cage with a model of a lion that has not been programmed to be disinterested when it is sated. AIs appear never sated; humans using or making AI wanting money, even less so. I suspect the AIs will understand the cost long before the humans will, not that anyone making money would care.

▲breadramen 3 hours ago | parent | prev [-]

[dead]

▲okamiueru 5 hours ago | parent | prev | next [-]

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

That sentiment makes me cringe. If you understand how LLMs work, you'd know it'll never be possible without a fundamental change in how these work.

We're also supposed to be reaching that point, somehow, without the LLMs ever being intelligent (in the dictionary definition sense, not the "high reasoning model" marketing sense).

Based on observations, the ones who are fooled by the supposed emergent properties, are just that, fools. Any sufficiently unintelligent agent will perceive transformer based LLM text predictors as possessing high intelligence.

▲tossandthrow 5 hours ago | parent [-]

Maybe you can enlighten us. In what way aren't LLMs able to make new contributions.

LLMs in agentic harnesses are Turing complete.

To my best knowledge, we don't know of any greater computational model that the brain is a part of, that LLMs are not.

▲pfdietz 5 hours ago | parent | next [-]

Obviously actual intelligence has an ineffable essential aspect, just like unicorn farts do.

▲okamiueru 5 hours ago | parent | prev [-]

First of all, the argument isn't that LLMs (with I assume some automation) cannot be used in searching a problem space. I'm assuming this is what you're referring to, in terms of contributions?

That's the part where LLMs are used as tools. Which there are plenty of places where they are useful.

Also, do you know what turning completeness is? Why are you bringing that up here?

The crowd that AI psychosis has brought to HN is interesting. But not in the "I'd love to learn more" kind

▲tossandthrow 4 hours ago | parent [-]

> That sentiment makes me cringe. If you understand how LLMs work, you'd know

> But not in the "I'd love to learn more" kind

I hope you are able to see the problem in your own communication here.

Computation classes are interesting because they say something about fundamental capabilities.

Two machine that are Turing complete are in theory able to carry out the same computations. They are isomorph mediums of computation.

Regardless. Please keep it sober. If you think you know something, enlighten us. But don't just propagate out lies.

▲okamiueru 4 hours ago | parent [-]

The thing about discussing or explaining constraints, is that it rarely is useful or productive if the other side does not accept (or understand) the reality of them.

Turing completeness is not exactly a high bar, and it's genuinely confusing as to why you bring it up. Your C++ precompiler is exactly as intelligent as whatever is your favorite agentic workflow with whatever harness you're referring to. Both might be Turing complete. Neither are intelligent. But one of them seems to be fooling you to think otherwise.

There have been many times that the C++ precompiler produced some output I couldn't understand. I might even at some point thought it was trying to tell me something profound I was too dumb to comprehend. Turns out it was just a missing semicolon.

▲tossandthrow 4 hours ago | parent | next [-]

Yes, so there is not reason to believe that you can do computations that the precompiler can not do.

There is no reason to believe that that you can not fully simulate intelligence in a C++ precompiler.

The precompiler can be simulated by human intelligence, and human intelligence can simulate a c++ precompiler.

Again, you are the one who arrogantly say they llms can not be intelligent without supplying any argument for such.

▲okamiueru 4 hours ago | parent [-]

> Again, you are the one who arrogantly say they llms can not be intelligent without supplying any argument for such.

Not really. You've provided the arguments yourself, just now. But, you don't understand them. Which, brings me back to the initial remark, as to why this engagement is bound to be unproductive. I'm off to bed. Have a good one.

▲calf 25 minutes ago | parent | prev [-]

I'll bite, I think Turing completeness is relevant in that it has to be used to informally argue via the Church-Turing thesis that biological intelligence cannot exceed the power or expressivity of formal neural network models. It thus is a good counterargument to stochastic parrot dismissals of neural net based AI such as LLMs, which really are still black boxes. The issue is not simply "So you think that humans are as powerful as LaTeX (or vice versa), ha ha how absurd!"

▲rho138 6 hours ago | parent | prev | next [-]

You’re prescribing elegance to a stochastic generator trained on the wealth of humanity, including 4chan. Let’s set our expectations a bit.

▲ryeights 6 hours ago | parent | next [-]

Your brain is a stochastic generator. Have you read 4chan?

▲AndrewDucker 5 hours ago | parent [-]

Your brain is not just a stochastic generator.

▲AndrewDucker 4 hours ago | parent | next [-]

It also creates and updates models, uses those models to make predictions, guides the stochastic generation by comparing the output to those models and reworks them in real time.

▲blovescoffee 5 hours ago | parent | prev [-]

what is it?

▲AndrewDucker 4 hours ago | parent [-]

It also creates and updates models, uses those models to make predictions, and guides the stochastic generation by comparing the output to those models and reworks them in real time.

▲yen223 6 hours ago | parent | prev [-]

Not sure if you're referring to LLMs or humans here

▲rho138 4 hours ago | parent [-]

(¬_¬)

▲ChaseRensberger 6 hours ago | parent | prev | next [-]

> It's going to produce proofs far more intricate than humans can understand, outdoing humans and opening new frontiers.

I agree.

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

I don't know if I see this being true for quite a while, if ever.

▲logicchains 6 hours ago | parent [-]

> It's going to produce proofs far more intricate than humans can understand, outdoing humans and opening new frontiers. > I agree.

There's an infinite space of possible statements and proofs. The only thing that makes certain proofs significant is that human mathematicians consider them significant; if AI came up with a proof of some statement that no humans could understand then no humans would bother investing further resources in building upon it, for the same reason we don't waste computational resources iterating over the infinite space of true statements in first-order logic.

▲amluto 6 hours ago | parent | prev | next [-]

In limited experimentation: AI will certainly make statements that are extremely intricate and hard to understand, in part because they're overcomplicated and in part because they use a bunch of unnecessary terminology.

This is not to say that a human couldn't understand a streamlined version or that the AI would not be better if it made more streamlined statements to begin with.

(I am not saying that everything mathematical that an AI produces is in any sense trivial.)

▲tene80i 6 hours ago | parent | prev | next [-]

It’s possible, but there’s a difference between vastness and difficulty.

Humans can’t compete with AIs on vastness of material they are familiar with, or the depth of effort they are willing and able to throw at a problem.

But scale isn’t the only aspect of difficult scientific endeavours. There’s also theory. And advancements sometimes come through hard graft of knotting together many things. And sometimes they come through the revelation of a deeper truth, or a new framework, a fundamental insight.

AI might help us reach the next level. But that doesn’t mean we won’t understand anything. It could be we have periods of vast intricacy we cannot follow, punctuated by profound elegance we (or at least experts) relatively easily can. And then the scaffolding we needed to get there falls away.

▲a2ff6eeb0 6 hours ago | parent [-]

I don't think that's true. Human intelligence is limited, and our brains are inefficient machines.

The tools we built to replace muscles have mostly obsoleted raw strength for tasks like excavating earth.

There's no reason to think we can't do the same for brains. And then we'll never need to think for a living again. Some people may want to do it as a commercially insignificant hobby, of course, the way people lift and compete in strongman competitions today.

We'll have AI taking care of our needs, the way a good mother takes care of their children.

▲archonis 6 hours ago | parent | next [-]

A good mother doesn't raise children to be dependent upon her for all their needs.

For this to actually work in a way that benefits our species, humans will need to become something else/next through their interaction with the technology.

▲a2ff6eeb0 6 hours ago | parent [-]

It's inevitable -- we won't have the machinery to compete, so we either have an aligned AI taking care of us, or we end up with a big problem.

Anyways, sipping wine on the beach and doing puzzles when I feel like sounds nice.

▲archonis 15 minutes ago | parent | next [-]

Permanant vacation is great for an individual. I'd love one.

However, for our species, extinction follows domestication.

▲chrz 5 hours ago | parent | prev [-]

looking for scraps in the gutter of never seen sun city more like

▲Jtarii 5 hours ago | parent | prev [-]

>our brains are inefficient machines

The human brain is exceptionally efficient.

▲bhaak 2 hours ago | parent | prev | next [-]

Then it's pretty bad that LLMs don't understand anything.

They don't know and can't know. Without an external source of input that corrects them, their output can never be verified.

▲fasterik 6 hours ago | parent | prev | next [-]

You could be right, but you're making a lot of assumptions about how complexity, scientific understanding, and explanations scale. One of the features of a good scientific discovery is that it often simplifies and compresses things that were previously a bunch of scattered facts. Also, as AI systems improve they'll get better not only at making scientific discoveries, but also at producing understandable explanations.

▲palmotea 3 hours ago | parent | prev | next [-]

> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.

Ask yourself: is that really the world you want to live in? It's a world where people, all people, are sidelined.

I think the happy ending of that path is something like Idiocracy. And the more likely ending is something like "automated capitalist economy without the people, because the people couldn't compete."

▲ 6 hours ago | parent | prev | next [-]
[deleted]
▲netz00 6 hours ago | parent | prev | next [-]

If and only if that is actually true, then perhaps nothing matters. Until then, calling out shenanigans remains a noble art.

▲jansport123 5 hours ago | parent | prev | next [-]

That maybe true at some point, but i don't think we are there yet.

▲a2ff6eeb0 5 hours ago | parent [-]

Yeah, it's probably a few years out.

▲mettamage 6 hours ago | parent | prev | next [-]

But apparently we can teach machines to do it for us

▲a2ff6eeb0 6 hours ago | parent [-]

Yeah. We can also teach machines to move hundreds of miles an hour, but we could never do it ourselves.

▲joe_the_user 2 hours ago | parent | prev | next [-]

It's going to produce proofs far more intricate than humans can understand

The thing is, some number of advanced proofs start out "too intricate for most mathematicians to understand" but many of these get rephrase and reframed until they're accessible to undergraduates. Hopefully, AI math can be guided to do that sort of reframing to increase the level of accessible math as well as extend the border of math.

▲logicchains 6 hours ago | parent | prev | next [-]

>produce proofs far more intricate than humans can understand

Math is not magic, a proof is just a series of applications of a set of rules on some axioms. A mathematician could understand any proof given enough time to study it; the only way for AI to make proofs that a human couldn't understand is by making really, really long proofs.

▲dyauspitr 4 hours ago | parent | prev | next [-]

Let’s not jump the gun. Where they are right now they can somewhat match our abilities. We haven’t even gotten to the point where they can self improve.

▲js8 6 hours ago | parent | prev [-]

Why would you want something you don't comprehend? How can you be sure it empowers you?

I think perfect rationality doesn't exist, because it is rational to reject something that you don't understand. So rationality of a given physical system will always be bounded.