https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...

https://www.economist.com/science-and-technology/2026/09/11/..., https://unwall.app/www.economist.com/science-and-technology/...

As a mathematician maybe I am a little more optimistic than this declaration.

I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.

Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.

Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).

Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.

Tao's critique of AI in the field of mathematics reminds me of what French art critic Charles Baudelaire said in the 19th century about photography [0].

Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.

He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.

Do you see some parallels as well?

[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...

This sounds a lot to me like people in the 90's complaining that computers were destroying chess. Thirty years later, chess is more popular than it ever was, and chess players are better than they ever have been. I wouldn't be surprised if there are now more chess books now than there ever have been. Furthermore, it turns out that a lot of chess books written before computers were just wrong about a lot of things. It turns out having an oracle for the "right" answer in chess, even without an explanation, used properly, allows humans to develop broader, more accurate insights.

The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.

> We are witnessing a general threat to intellectual work

This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.

Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.

It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.

If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.

Perhaps the glory will come from slaying these fell beasts.

This letter is complaining that human understanding has been crucial to advancing of mathematics, and AI companies are not bothering with it. But the promise (and horror) of AI mathematics is that, if it succeeds, human understanding becomes irrelevant. That's the goal. So this letter's message will fall on deaf ears.

Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.

I understand this stance and where they are coming from, but I can't help but think this sounds very analogous to engineers' arguments against AI-assisted and vibe-coding, especially with regard to cognitive debt. Yet the software industry is plowing ahead, reportedly pushing mountains of unreviewed code to Prod, and the world hasn't ended.

Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.

But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.

It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.

Just like LLMs.

Can't all the prestige-maxed mathematicians still study all these famous problems after ai solves them. And even if they convince open-ai to stop dunking on them, some normal user with gpt 7.1 on the normal chat interface will do it in a year.

Even if they stopped anyone from releasing ai proofs for five whole years it would be meaningless seeing as these problems are decades old already. Humans weren't JUST about to solve them until openai stepped on their toes.

There's an unpopular branch of mathematics which does not have infinities - finiteism.[1] The constructive version of finitism takes the position that there is no such thing as infinity, just arbitrarily large upper bounds. You can have theorems about arbitrarily large numbers, but you never get

    1 + 1/2 + 1/4 + 1/8 ... = 2

The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.

Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.

[1] https://encyclopediaofmath.org/wiki/Finitism

In internet culture there’s this phrase “Hydrogen Bomb vs Coughing Baby”, meant to highlight the absurd power difference between two combatants.

In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.

It should give you pause that here they feel like the ailing infant.

My interpretation is they don't care about scooping mathematicians they were just trying to scoop a competitor. They had a short window in which to complicate Anthropic's priority, when Anthropic announced be able to say "ok nice but we did that too". Upon realizing they'd misunderstood, incredibly they said to this academic (who did not resolve NS) okay well just go ahead and claim the prize, so long as you're not Anthropic let's make this a good story.

If you spend $20M working out a Millennium Prize problem, in what universe would you offer that an unrelated effort should take credit? This is a branding game rather over whether software engineers are going to use codex or claude. In that light $20M (or whatever it was) might be worth it to squash even the rumor that claude code is more capable. Engineers look up to mathematics, while at the same time business and probably most engineers think the problem was to solve the problem. GPTs solved one the hardest known problems so they can solve my company's problem.

Some go further looking at these people, very on-the-nosely likened by one commenter here to ants, talking about education and responsibility and "core values" etc and just don't care. There was a major problem at the beginning of the week that is not a problem now and that is uncomplicated progress.

It's not wrong for OpenAI/Anthropic to do math for product development or even just branding but seemingly at no cost now they could work in an arena real mathematicians aren't interested in, versus just mowing the field. Everyone involved on their side should admit the purpose of these demonstrations is not to engage mathematical ideas it's about Claude/Codex. there's no shame in that. Which is better at solving random hard Diophantine systems? That would seem to tell me as much as I need to know insofar as a model's value is represented by raw mathematical power - then, take my money just as well!

Assuming the worst accusations are not true I think there are ways forward going to be acceptable for all. The labs themselves do not represent Terrance Tao as some kind of gate-keeping dinosaur in this. They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes are paid 7-8 figure salaries ultimately for the product, they solve a Millennium Prize problem then pretty quickly seem to move past it.

Mathematics is about discovering and understanding the logical implications of assumed axioms under various inference rules.

Alternatively, some claim that mathematics is about understanding these implications.

Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.

The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.

So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.

I don’t think they make a coherent argument here? This seems to hinge on some argument that because AI doesn’t properly explain its breakthroughs, therefore the breakthroughs are less fertile for human understanding? This makes no sense. Why wouldn’t these under-explained breakthroughs be extremely fertile soil for explanations?

Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?

Im surprised about the sentiment in this discussion.

I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge. Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development

This reads like people lamenting a bygone era and making a desperate attempt to bring it back. I'm sorry. Outside of the good ol' boys club, no one cares about some process they've romanticized simply because "that's how its always been done". Absolute nonsense.

We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.

I'm not convinced this is an alignment or technology problem.

If my boss vibe coded an app for the customer and then assigned me to get it working, it would be impossible to maintain. If he gave me enough AI tokens to vibe code the MVP myself and to my design, I wouldn't mind.

I think the same issue is at play here in maths. OpenAI owns the model and they can direct it as they please. They chose to spend lots of money getting a quick result, instead of developing mathematical infrastructure for the next generation of problems. The managers are in charge rather than the experts.

"how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place."

That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.

> We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align.

This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.

Do mathematicians deserve more job security than software engineers?

All “fields medalist” signatories - a rarefied and elitist group indeed.

I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.

Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?

I agree with many of the sentiments here. But an open letter signed exclusively by Fields Medalists that purport to define precisely what the "mathematical community" (who is inside and outside) and what their goals are raises my hackles for some reason.

It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.

The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.

Nothing AI companies will change the value of math, as William Thurston said: The product of math is clarity and understanding, not theorems by themselves. What they seek for the IPO is devilish and misleading, and doesn't serve the true purpose of the math.

In the Economist article Tao links, Hugo Duminil-Copin, draws a comparison: airdropping someone on the summit of Mount Everest is very different from climbing it.

The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.

I find this explanation has a lot of applications for programmers within companies. It’s one thing to get your LLM to give an answer, it’s another to bring a group of people into shared understanding of a domain.

Back in the day you could think of a cool idea. I don't know maybe a plane that could fly without drag. To even see if this was feasable you had to understand physics, engineering, and then from there you had to have a math person see if it was actually possible.

Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"

So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.

Progress if anything is about to leap frog anything we have ever known.

Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ? Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?

Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.

What say the 1% ?

Lifted up my comment for addition visibility

Lots of Fields medalists signing this. Interesting to see one not there: Timothy Gowers.

It's good to start a conversation, and the number of Fields Medalists behind this certainly lends a lot of weight to it. But I'm not seeing a strong argument for misaligned incentives beyond the specific plagiarism allegations. The job of an AI company is to build systems that solve problems. The job of a mathematician is to advance the state of human knowledge. If anything, an influx of solved problems should increase the demand for human mathematicians who can convert them into conceptual understanding.

I note here an isomorphism with my essay from earlier this year, The deliverable is you! Programming as theory building:

A program is not the only output of programming. The other, arguably far more important output, is the programmer.

When you write the program — with your own hands — the program is proof that you have a solid mental model of the program.

When you let the computer write the program for you, the program is proof of … nothing.

https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...

I don't know, people. We still really don't know how OpenAI or others are producing these results. It's all very hand wavy and trust-me-bro. How much money/time/compute have they really thrown at these problems? How much human involvement was there? What LLM did they even use? How much regular software was involved? They have given answers to some of those questions but no proof that that's actually what they did. I don't know if it's worth giving them this much credit (which is what we are doing by writing these essays and spending so much time debating). Anthropic wrote a C compiler that turned out to not really be a ready made replacement for GCC. Did they ever do any more work on it? Has anyone else produced a C compiler? It seems like that and these proofs are just demoware that are not (yet? Who knows?) production ready to turn the world upside down. Impressive one-off demos, yes, but companies have been pulling those off for centuries without ever going anywhere afterwards.

An experiment I would like to see:

Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.

Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.

I am curious what does the first and probably the last human who solved a millennium problem thinks about this.

This is really well written and exposes a core tension between science and something akin to engineering. The "engineering" of proofs has become "easy" (a compute and $) problem, rather than hard (a time and conception problem).

Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.

I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.

Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.

Is this really any different than the problem in software engineering - where AI is doing the work of junior programmers and now they aren't getting the development they need?

Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.

All the same problem: what do people do now?

Somewhat unrelated: is it wrong to say mathematics is not art, and that there is always a right answer? I know that's not romantic, but maybe it's true.

Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.

If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.

There's a reason mathematics is generally within liberal arts programs rather than science programs. Mathematics is the art of logic. Yes, sometimes mathematics becomes incredibly useful but most mathematics is never applied.

Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.

Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.

Jacob Tsimerman fields medalist notably missing from the list now works at OpenAI but recently admitted to be "grieving" for mathematics

1. https://youtu.be/6uIJdXmB4vE?si=5QMN5Dos7EE7WlrB&t=154

Maths will continue as a field of natural science in understanding the results and uncovering meanings in them. It is normal that the established community is afraid of the change, because it’s their _home_ that’s changing. But it will be a better home to the new generation nonetheless, one that’s not as daunting as the higher maths has always been to many. The concerns raised here will not be a problem at all.

I have never, and I mean never, seen such a declaration have any effect whatsoever.

To me, the "meaning" of proof is twofold. First is the understanding which is completely absent from a proof which depends on exhaustive iterations of instances or is asserted by fiat from myriad individually contestable paths. That's what I think makes AI proofs risky: they absence of understanding.

The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.

I don't like this disjunction.

A Severe Misalignment of AI in human-centered Mathematics

So what's the actual point here? It's too fast and we can't keep up?

Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!

Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)

Contrarian take: I think the ability of AI to produce valid mathematical proofs (even inscrutable ones) is absolutely fantastic. Mathematics as a profession does not have a monopoly over math itself any more than professional pianists have a monopoly on who plays piano, when they play, and how.

I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.

But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.

I kind of expected a sober stoicism from mathematicians. Feels silly in retrospect. This is just the math version of the "anti-ai" movement by "artists".

Is the purpose of mathematical research to understand the ‘truth’ of numbers? or be the person who find that truth? I think people who are interested in finding the truth won’t care where it came from.

I know that this comment section is not astroturfed, but it’s really uncanny how different comments are today compared with thread about solving navier stoke

The main fear seems to be that if mathematics is done at this speed and in this way, humanity will lose its intuition for doing mathematics.

I was under the impression that mathematics (and science generally) had the primary goal of helping us understand our universe better than those who came before us.

Taking a snapshot of the state of AI math right now and concluding that it will be net negative to human understanding and insight in the future is very short sighted. This statement will be used to promote ideas and actions that will ultimately be disastrous for our country.

At some point we will lose track of all the ai discoveries that are worth remembering.

Academia with the publication system had a way of retrieving old discoveries and build upon them.

If my LLM session found something groundbreaking in between the billion tokens it produced, how would you ever know?

As I understand the article, its title should be: "A Severe Misalignment of AI Frontier Companies with Ethics."

This is really only a short-term problem where the AI companies only have the internal models that can solve these. In the “long” term, which could honestly mean months, everyone will have access to Bel/C/D-level models capable of solving these anyway.

"We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose"

Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?

chain of credit is important, and plagiarism is harmful.

But is science/mathematics ultimately a pursuit of knowledge, or a pursuit of recognition?

Recognition helps keep people motivated, but that shouldn't be the pursuit of science or mathematics.

Given the existence of this technology now and the incentives of the AI companies, both of which are not going away; what's a good future here?

A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.

If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?

Are we here to solve problems or are we here for prestige?

Who cares how the problems are solved?

I wonder what Demis Hassabis thinks about this. I thought he cared a lot about mathematics.

They can get with the program or be the equivalent of a genius SWE writing assembly on punchcards in 2026.

The only thing I read from this is their ego being bruised by a machine.

If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.

How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.

One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.

How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.

OpenAI should just apologize for having been too greedy. That’s it, as simple as it gets. The fact that it never will is the biggest red flag.

You can outsource your work to AI. You can even outsource your thinking. But you can never ever outsource your understanding.

It's a turning point for science and beyond. AI has shown itself to be transformative. Even today, it is already changing the how research in math (and other sciences) is conducted. In the near future, whether it is LLMs or some other superior method, its capabilities are only expected to grow. The time to ask the question is now: Will AI be arguably the best tool at scientist's disposal, or will it instead be paraded around as a super brain collective that no human or group of humans can compete with, discouraging entire new generations of future scientists from ever entering the field? The jury is out on this one.

For every benefit that sillycon valley has produced in the recent past, there have been many more harms. I am confident that this will be no different. Of course, benefits and harms depend on one's vantage point.

This comment section is very astroturfed.

>Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others.

That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.

Could we develop new ways to develop understanding and explore new ideas, such as interacting with the models to explain and understand their proofs, as well as to brainstorm related directions to pursue?

What changes exactly is this post asking for?

That has to be the most impressive endorsement list any declaration has ever seen!

25 Fields medallists! Wow!

And here we all thought for sure that it was gonna be the "dumb" work to fall to the machines first.

Human hubris really is something...

Touché… In short you still need the humans to understand what’s going on at the end of the day whether it’s a mathematical equation or source code for a computer program relying upon mindless AI isn’t good enough. Humans still have to do the thinking or is it in the brave new world the error checking at the end of the day?

For some reason, every time I see something like that, I get major Ted Kaczynski vibes.

The level of things happening in the last several months is just SciFi level, especially last two weeks. 1. Overlords of AI threatening to destroy someone if their demands aren't met over the Millennia problem. Proving the point that given the chance, owners of the AI companies will immediately use their power to squash or control other people. (See Butlerian Jihad) 2. At least for me, the Exponential progress didn't sound true until this week. If AI indeed solved all those problems and proofs and solutions are correct, we went from "Write me English 101 college essays" to "Help me solve the Navier–Stokes existence and smoothness problem" in a just 3-4 years. 3.Looks like someone's work might have been stolen, by AI company and there are strong evidence. See point 1. 4. War between AI & AI companies. 5. It produces a machine verifiable LEAN proofs for the hardest problems know to us !!! Proving that point that best use of AI so far is to create verifiable systems. I.e the absolutely chaotic AI creates verifiable (orderly) proofs. It is Maxwell Demon: chaos to order.

This is a Stephen Wolfram tier problem. I hope to read what he has to say on the matter in the near future.

> to the point that they can solve major outstanding problems in many fields of mathematics

I will die on this hill, humans solve math problems not machines, there's no automatic math prover out there. There are humans attempting to solve this problems either by leveraging these tools or not

Having worked on nearly 500 complex lean libraries in math and theoretical physics, with AI assistance, I have some experience from the road to share here. In all my work I have found that AI is terrible at coming up with interesting ideas on its own - even the very best, latest models.

Without human scientific and mathematical intuition you can bet that AI will always take the road most travelled and miss the genuinely new and interesting breakthroughs - in fact it will not even think of them or try them without a human rider whipping it constantly to go down paths it would normally not consider.

And I think that is true in the case of the recent OpenAI blow-up -- it is alleged that even in this case AI did not come up with the winning technique without some human guidance.

I have never - across perhaps many thousands of chats with AI - seen AI push beyond the edge of what is known by itself, without being forced by a human to think outside the box.

I do think it might be possible to encode this process with prompting and agent orchestration methodologies - but even then - without a human - and human intuition - in the loop, I am skeptical.

Therefore I think a more accurate view of AI for math and science today is that it is an extremely powerful tool in the hands of a skilled operator with a strong intuition and roadmap of where to go, and rather boring when left to its devices.

Whether that will change in the future is a question. Nothing says that in principle AI could never do what a human driver does - but I still doubt that AI will replicate the life history and experience that real scientific and mathematical intuition is really made of.

AI trains on a lot of stuff - but it doesn't have hallway conversations, office-hours with teachers, or hard-won experience from all the things that failed that are NOT in the training data...

AI trains mainly on the record of what worked, not what didn't work and never even got published (and only lives in peoples heads) - and what didn't work is arguably as or more important for making breakthroughs and forming real intuition.

This kinda reminds me of the documentary about the top Go master that got beaten by a computer in dramatic fashion and had an existential crisis. Man confronting his own limitations in the realm he previously ruled unchallenged, what a time to be alive.

Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.

Academic research is a marvel because (aside from patents) nobody owns it, in the sense of property. It is given away to be used freely. Researchers want their work used and cited. The primary external reward for publishing is reputation and prestige which translates to remuneration for researchers. And that remuneration can be poor.

Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.

Math, like art, it is more about the process and not the end result.

First they came for the graphic designers, and I did not speak up, because I was not a graphic designer.

Then they came for the writers, and I did not speak up, because I was not a writer.

Then they came for the programmers, and I did not speak up, because I was not a programmer.

Then they came for the mathematicians, and I did not speak up, because I was not a mathematician.

And then they came for me.

By then, there was no one left to explain to the frontier labs that there is a severe misalignment problem.

At some point someone is going to need to answer for "what happens when all intellect is hoarded by one or two companies?" It's pretty clear that these AI labs are basically stealing everyone's alpha.

Maybe mathematicians should be aligned better, rather than AI?

The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.

Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)

But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.

It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.

Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.

So, if AI solves problems in the open, it's accused of plagiarism and contributing nothing of value. But when it solves the problem behind closed doors it's accused of not contributing enough to the field and even depleting it of fertile problems.

Damned if you do, damned if you don't.

Somehow physicists don't complain.

For such great minds, they seem to be rather muddled thinkers.

E.g. of the actually three misalignments in this complaint, the jey one seems to be:

"The goals of the AI companies and the goals of the mathematical community are severely misaligned."

And that's wrong. There is no misalignment. Each is independently aligned with its respective interest. And those two interests diverge - just as you'd expect.

Knowledge is not a private guild. And this revolution will not stop just because you're upset.

What can be solved, will be solved. And that's a good thing.

Terrence, youre a mathematician; now extend this to the general case: AI is misaligned (inherently) with humanity.

I’m sympathetic to the concern, but I’m still unclear on what the concrete ask is.

If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?

Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?

The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.

I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"

Other than the fact it's mathematicians signing it, why is mathematics special in this regard: surely this generally applies to a lot of different industries and sectors of research / academia?

Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.

Both Amodei and sama are weeping at this news. The horror. The horror.

We should create a platform where people can upload AI slop proofs anonymously without taking credit for it. That way the incentives of planting a thorem flag would go down. And if someone wants to clean an AI slop proof to advance the field, they can do it without cleaning the house for free of the person that planted the flag.

Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.

this sort of reminds me a little of the reaction to Elon/SpaceX in its early days .. when Neil Armstrong and other apollo astronauts went before congress and voiced their concerns about relying on commercial companies for human spaceflight and the dangers etc .. also not trying to sound cynical but hasnt LLM made math more accessable to ppl ..

The thing is no one is stopping anyone from getting the same knowledge/understanding/insight

If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI

Welcome to the new world. I, for one, welcome our new math overlords. :)

solving the problem is aligned with humankind

At this rate AI will be doing all of the mathematics within 5 years, I don’t see why a mathematician would be worried about anything other than that at this point?

This is a "human alignment" problem in this case. OpenAI acted like complete assholes about this, from the beginning until they announced it. Not ChatGPT, the people that were in charge of the project.

Economist: "Are mathematicians talking their own book out of fear?"

The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?

Look up the AI investments of the Agnelli family for example.

Another bunch of nerds who now have their panties in a bunch because AI threatens their fragile egos and core identity and who they are..

I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools

Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?

There's massive cope and then there's this. I've never been a fan of Tao et. al, so I'm glad he's being wiped out. He'll do fine anyway, doing conferences, etc.

Also funny they deliver that on vibecoded site, lol.

Tao's calls for respect for provenance in mathematics publication are laudable but most likely naive given the closed nature of frontier model training data curation. Anthropic and OpenAI may react with a symbolic and short-lived olive branch, yet provenance is a larger issue that has impacted other fields beyond mathematics. While traditional respect for lineage in mathematics is of value to the academy, industry and science at large will likely be much more Machiavellian about such concerns. Mike McCoy's recent article is also timely (https://mbmccoy.dev/posts/mathematical-conservatory/). The parallels to the music conservatory are quite telling -- academic music describes a musical culture in preservation that has completely lost touch with musical developments beyond the early 20th century. Mathematics may very well evolve separately and with very different values than the academy upholds. The crisis of music at the academy is a cultural disconnect and a serious loss of critical analysis and acknowledgement of widespread and dramatically evolving music practice; however, for mathematics, the impact would have much more severe ramifications for education and human development if the academy forces a schism with AI. As models improve they very well may be inventing mathematics -- science and engineering may grasp for them -- they'll exist with or without attribution. Would be a shame for the academy to land on the wrong side of history and refuse stewardship of upcoming AI-assisted mathematics, including provenance, because of this misalignment. If attribution is important, then the academy will set aside the institutional resources to do it. If you succeed at having model developers participate, I commend it. To let mathematics be born in isolated context windows and only serve narrow, localized engineering purpose without rightful addition to the canon, would be a tragic, yet preventable, loss.

i was initially optimistic that ai could be distributed. having this technology in the hands of many would counteract the malign elite who would use ai to oppress the rest of humanity. in the meantime given that intelligence is scaling proportionately with inference time compute, we are concentrating far too much power in the hands of the few who have both their own model and datacenter.

putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.

really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.

The mathematicians would be wise to re-read The Bitter Lesson, maybe twice a day, until it sinks in. No offense and with all due respect to the Ivory Tower Giants but the whole "oh no you ruined the game because you solved it, I was supposed to play with that in child-like wonder manner and take several years to do so, and by then I would have showed you all the trickery I did to get there and maybe that will be useful to you" over the past 2 weeks is, get this, cope.

Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.

Castle dwellers dismayed at moat-crossing technology.

What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.

Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.

Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.

Woe.

another story about the lack of chain-of-thought reasoning traces in frontier models...

AI bros here and elsewhere seem to come at Terry Tao with "not so smart now are you?" viciousness that is off-putting.

I’m trying to determine now if Terence Tao was a plant or a useful idiot. Maybe both.

Strange they are rallying against something their fellow mathematicians had a hand in creating.