This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?

Here the worry is the social structures of mathematics are eroded such that fewer humans become able to do the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.

The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required for anything that anyone wants in a practical sense.

Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.

Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.

And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.

Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.

I thought one of the implicit points of the open letter was that unsolved problems are not something that falls out of the sky, they are a curated resource that people have spent time on and shared for the benefit of like-minded peers and humanity as a whole. And the AI companies treat them like they treat absolutely everything else: natural resources, literature, art, code etc. as something to be chucked into the ravening maw and pooped out the back as profit. They don't care if mathematics advances, they don't care if they strip-mine the available problems and damage the field. In fact, as with programming I think they see that as in their long-term interest - soon there will be no intelligence or creativity but the one that Sam Altman bills you for.

I cook because I like to tinker, understand how variation in recipes affect flavor, and occasionally stumble upon something novel and tasty. The journey is most important but the goal is nice.

My wife cooks to produce a meal and feed our family. The journey is a necessary evil to achieve a goal.

We are aware of this difference in style and have learned to appreciate what each has to offer. At the same time, we sometimes annoy each other in the kitchen.

There is an analogy here to what this essay says.

> With that interpretation, the issue becomes slightly different: is it more important that the collective understanding of the mathematical community should be as advanced as possible or that there should be answers to as many problems as possible? Or are those two aims valuable in different ways, so that there is no point in declaring one of them more important? Or are they so inextricably linked that it makes no sense to argue that one is more important than the other? And when we say “important”, for whom are we saying it is important: for mathematicians, or for society as a whole?

This is easy to me. Truth should be the North Star. If there is a fundamental truth that can be found via mathematics, then the shortest route to that truth should be preferred. While LLMs are definitely capable of solving problems in search of truth, I agree with Tao that instant "true/false" results threaten to short-circuit the traditional avenues we have used to escape local minima in the search for truth. Their products may be the junk food that provides immediate satiation in exchange for long-term health. Perhaps it's wrong, though.

Would it matter if AI had a proof for RH or P < NP, but the proof was 1000 pages long that Leaf could verify but otherwise impenetrable by a human mind? We generally assume these are true already -- I am not sure a long convoluted proof puts us in a much different place.

why does a fields medallist not have enough money to have a blog that isnt festooned with ads? good god wordpress.

One thing I've found in my own use of AI, which Gowers touches on in his point at the end but which I think deserves more attention: AI makes some previously non-trivial tasks trivial, but that doesn't make the work itself trivial. You naturally expand the scope of what you attempt and take on harder problems than you could before. The floor rises, and so does the ceiling. The open question is whether that still holds once models can also do the harder problems, or whether choosing and framing those problems remains the human part.

My Mom was worried I wouldn't finish my thesis so instead of encouraging me she encouraged an LLM. She says now I don't have to worry about submitting a proof anymore, there is a counter-example.

This is the state of affairs today and how everyday people will talk about it.

> we urgently need to come up with good ways of explaining the value of having a large pool of human mathematical experts, even if it is no longer part of their role to find new proofs of theorems.

This is the main issue, and while I fully agree with that value sentiment, the referenced letter failed to provide convincing arguments for why mathematicians should widely receive funding for merely understanding things, and how competition for postdoc and tenure positions would work.

I think the author did not mention that the way humans construct the knowledge ladder requires low complexity because each step gives power to the next, but the AI with the exponential exploration can give biggers steps but then it stagnates because the next step can be beyond the exponential exploration complexity. In chess a good strategy can be the best tool, in the game of go our experience suggest the same, but it could happen that mathematical thinking requires a type of policy that could be beyond the current ideas. I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.

Solving a theorem is like climbing a new mountain. The mountain is already there, and there is a list of the hardest known mountains to climb. The point of climbing them and not just dropping with a plane on top of the peak is to help develop human climbing skills and expand our knowledge and abilities. Also, already conquered mountains are climbed all the time to test new strategies.

Now, if an LLM proves a theorem, it's like discovering a new mountain and knowing what its peak looks like. Does that mean the problem is finished? No, we still need climbers to actually do the work and advance the field with human understanding.

The job of a mathematician just shifts from proving to inventing new theories and finding new conjectures with the help of AI, which should be even more fun.

I'm not sure what the primary signatories want, exactly. "No mathematics" safeguards added to frontier models, like the "no cybersecurity or bio" ones that currently ship? That seems like a bleak future. Protectionism like tariffs.

Understanding the world and math, and discovering beautiful explanations is a worthy pursuit for humans in and of itself.

However, math and science and engineering and biology/medicine have profound implications - those useful parts of that 'culture' - which we call 'technology' - extend human lifespan, healthspan, prosperity, safety.

The main practical value of investing in maintaining a population of math artisans is that they are/were needed as part of a well-trained science community needed to discover and develop new useful technology.

I think its a rational view to say LLM generated "AI-math" is a net positive if and only if, it results in _more_ high quality "human-math". By human math I mean well digested math residing in human minds, being discussed between humans and being actively re-discovered by humans, including a pool of new human student devotees.

A very bad outcome is where we lose the next generation of scientists/lawyers/mathematicians/engineers/doctors/researchers/authors because we let the AI do it all. In this dark future we outsource all our 'thinking', and avoid the years long training of grappling with hard-to-understand aspects of reality. All our culture is sucked into the event horizon of an AI black box.

AI could be a new renaissance of human math- and science- culture, or it could be the death of it.

AI isnt going away, the financial incentives and high current economic inequality, geopolitics guarantee that it will proceed as fast as possible.

AI has given us the structure of nearly every protein, which may well solve Alzheimers and cancers. AI may well find a solution to stable plasma for nuclear fusion, unlocking vast cheap energy and help us humans halt global warming.

So, how do we make sure that the economic windfalls of AI are reinvested back into human-culture, funding more human-math and human science .. resulting in a deeper pool of well educated scientists, engineers and researchers ?

I don't think that will happen by default - we are likely to have an AI-assisted dumbing-down rather than an AI enabled age of enlightenment.

Most of the general public are happy enough not to learn any math - when it doesn't make you much money and university is such a debt burden anyway. School and university students can just get the LLM to do their assignments. Why should they pay tax to fund science at universities when the cost of living is so high, and the LLMs can do all the research anyway ?

I think part of the solution is a policy to tax the windfalls of AI to mitigate the downsides of AI.

Governments should have taxed the carbon polluters and used those funds to mitigate climate change effects and research new forms of clean energy, but didnt.

We should tax LLM/AI profits and use that to fund science research - not just research to mitigate the effects of runaway AI, but to fund general science and math research and teaching, to guarantee that there is a net increase in human-math and human-science.

It is a cautionary tale that fewer students know their times-tables by heart, because a cheap calculator can do that ... but it gives me some hope that there are still people enjoying playing chess, even though chess programs are super-human.

> One way that might happen is that AI disrupts society so much, or even kills vast numbers of us, that the preservation of something like the current mathematical tradition ceases to be of any concern: all that will matter is the survival of the human race. But that again is a topic for a different blog post (which in fact I am in the middle of writing).

I cannot wait until Gowers and Tao must have an iris scan at Altman's Worldcoin in order to get to their beloved OpenAI. Give it 5-10 years.

Imagine a world where mathematics is not understood by any human. Theorems and mountains of proof papers are constructed on a daily basis by automated machines at a rate far to quick for humans to understand.

These theorems DO benefit humanity but are largely unseen. You ask the LLM to make you an app. The LLM makes the app and utilizes entire languages, theories and processes made by discoveries that you as a human are not a privvy to. It all goes into the this body of knowledge only decipherable by AI.

This is the future AS long as current trendlines stay the same. As long as progress keeps going at the rate it currently is going. This is the best predictor.

Personally, and I have no evidence for this though, I think we'll hit another barrier and AI will stop making progress for about a couple decades or so. So humanity still has some time to have fun before something gets over the hump.

When the newfangled electronic calculators came along, many eminent engineers, and teachers of engineering, objected to them displacing the slide rule.

For those of you who haver never slid a slide rule, here's the part that makes this anecdote make sense: they're a mechanical logarithm, this allows one to do multiplication as addition, but there's a catch: you have to know the "characteristic", that is, the order of magnitude of the result. The slide rule won't tell you that.

They argued that this requirement leads to an intuitive grasp of magnitudes, which is of real benefit to the discipline.

And they were correct. However, engineering is, by all appearances, doing just fine.

Where they erred, and it was a natural mistake to make, is in assuming that pedagogy must recapitulate phylogeny. That engineers should start with the slide rule, just like the profession did, and then graduate to the calculator, just like their professors did.

But this is wrong. It makes the slide rule an impediment: a known-to-be-obsolete object, standing between the student and graduating to the device he or she knows, full well, is what will be used for the rest of his or her career.

Instead, teach the concepts and the foundation with the modern tools. Then, senior year, in addition to the thesis project, teach "OG" engineering: slide rules, graph paper, mechanical pencils. Along with the thesis, which should stretch all the skills already mastered to their limit, require a feat of engineering which is within the student's capability, but: no CAD, no calculators.

This would actually work. This gives them an opportunity to find out what they're missing, build some of that order-of-magnitude intuition, really solidify what a "sketch" in CAD is all about, all of it: and understand that any force multiplier can be a crutch, if you let it.

I've been just delighted with what I can accomplish in "centaur mode" with LLM agent assistance. I've made real progress on longstanding research topics. But to do so, it's essential to never let them be a substitute for one's own understanding. They turn out to be really good at explaining difficult passages in research papers!

My gloss on what's happened with LLMs in mathematics (understand, I am not a mathematician), is that we're discovering that solving conjectures is just not that important. Not the first time this has happened: simplifying and manipulating algebraic equations was very important, and then, it wasn't.

Making conjectures, that's where the action is. Always has been. No one thinks that solving an Erdős conjecture puts one in the same rank as Erdős Pal, because it doesn't.

When the first mainframe computers came along, there was a brief flurry of conjectures, some of them longstanding, which were either proven or disproven by brute exhaustive search. What's happening now is different in magnitude, and even in kind: but more in magnitude than in kind.

It wouldn't be fitting to be glib about the disruptive effect this is having, and perhaps (although I don't think so) we'll cross some threshold at which it doesn't even make sense to contemplate humans doing authentic intellectual labor. That would be bad. If you think that's on the horizon, you're right to be upset about it; I don't, but I won't try and persuade you otherwise, because the point is far from obvious, and both of us are predicting the future, which is known to be a risky business.

The "publish or perish problem" which this new development surfaces is not new to this development. The entire concept is well past its sell-by date, and while mathematics is perhaps the intellectual pursuit least corrupted by the dog-and-pony show, it is and has been undignified, inhumane, and always to some degree dishonest. Time to come up with something better.

> At one end of the spectrum you have mathematicians who are primarily motivated by the wish to solve problems, who see conceptual understanding as a very important means to that end. At the other you have mathematicians who are primarily motivated by the wish to attain conceptual understanding, who see problem-solving as a very important means to that end.

Funny. Reminds me of programmers who like to program, and programmers who like programming because it lets them build things. The first group hates AI, the second loves it.

Knowledge wants to be free.

Watching the discussion unfold, here is what I feel:

Before arguing whether mathematics must strictly be done by humans, there are different motivations at play. Some people love the sense of solidarity within the community that forms during the process. Those excluded from that community might resent it, while others just purely want to solve problems.

Many things are being discussed, but looking at the overarching narrative, it seems that AI's true function isn't necessarily opening new horizons of specific knowledge, but rather excelling at 'serializing' topics that have been heavily fragmented until now.

In that sense, the concern is that because AI is solving the very problems needed to cultivate mathematicians internally, the stepping stones required for human growth are disappearing.

However, on the other hand, as the world and industries become increasingly complex and hyper-specialized, you could also argue that AI is the exact tool needed to unify this fragmentation across academia and industry. It is a highly complex dilemma.

From the perspective of researchers and the mathematical community, those 'problems for growth' must remain. But conversely, AI has the distinct ability to serialize siloed disciplines. Usually, when you go to graduate school, you often hear professors say that even within the exact same major, they cannot understand each other if their sub-specialties differ.

This was also guest-posted on Tao's blog. Tao has a flood of guest posts that cope in a varying degree. Most start with empathy, move on to inevitability and emphasize human understanding.

Few of them mention research theft, none of them mentions concentration of capital and resources.

The fact that Gowers quotes Tsimerman (OpenAI employee) says it all. A couple of mathematicians are determined to bulldoze forward with AI, and I suspect Tao will also continue after the meek concern letter and the cleverly worded guest posts that are subliminal ads for AI.

The problem with the Fields medallists petition was that its central argument relied on reiterating a very fashionable, neoliberal-academic trope, that science as a whole is "done by a culture of experts", implying that there is no value or even science existentially outside of "the social practices of science". It is a predictable extension of centrist-pseudoleft political ideologies (~ ivory tower belief system) which Thomas Piketty for instance has conceptualized as the Brahmin left. Indeed the whole AI/ASI/AGI issue is hugely political and it cannot be dismissed and analyzed as a neutral topic in isolation from people's political premises.

That said, I think there is an articulable concern about the recent AI events as it may threaten human's autonomy and relation to science and mathematics. Just not one premised on that. What's interesting is seeing different academics having predictable reactions, for example Sabine Hossenfelder is more politically libertarian so in this case she said DGAF about human mathematicians at all. Scott Aaronson is another interesting example, he wrote a blog post a couple days ago as well.

TL;DR: Mathematician community broadly unhappy with AI because it might destroy the community by doing all the mathematics for them.

OpenAI must be destroyed.

This is pretty disappointing I have to say. The letter actually has a “pro AI” stance (see quote below), and Gowers’ apparent reason for not signing it is so subtle (not wanting to offend people whose goal in mathematics is not understanding) that I can’t help but view this as being more about optics than its actual content; i.e. he seems to be cleverer than I am in recognizing most people will not see the letter as a push to use AI in ways which are healthier for human flourishing but as advocating for one of two sides in a highly artificial binary (“pro AI” or “anti AI”). Gowers draws a distinction without a difference here and should have just signed the letter.

> AI offers the potential of enhancing and accelerating genuine mathematical study and understanding. Mathematics as a profession will need to adapt to these changes in several ways. However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology.

Siri, can you call the bicameral order, the captain has published again..