Accelerationists may argue that the eroding of proper attribution and proof verification by humans is a meaningless short term struggle of a dying field.

Mathematics seems to be entering an era where human + machine maximizes performance, much like chess in the 1990s. However, imagine a future where even talented mathematicians are nothing but noise in the machine (as is the case in chess now). A future where AI generates and verifies proofs without humans in the loop. Where the mathematics may be beyond human comprehension.

In that future, does it matter that early career mathematicians are inhibited by these developments? Perhaps not. Programming faces the same issue. As AI crawls up the competence ladder, does it matter that fewer people have opportunities to develop the skillset of a senior engineer? Perhaps not.

Much like for many the point of chess is that it's played by humans, with truly superhuman AI relegated to a training aid, mathematics is in many ways about human comprehension. You can use AI to find and proof new theorems. But if you get to the point where humans can't understand it, is it even still math?

An issue I see is who controls the information. The next generation may not recieve the knowledge, it may be gatekept by industry who *will* own the gate.

The future may not have access unless we fight to ensure they do. This is how I read the article.

Surely such AI would also be able to reduce and simplify the math for human understanding. Which is what mathematicians do all the time, from turning base 60 cuneiform into modern number systems to simplifying Maxwell's equations for the students.

On the other hand, it could stall out at: good enough to take the easy problems, not good enough to take over the field, but damaging enough to erode the quality of new entrants. (Which incidentally is the scenario I think plays out for software)

AI (in this form) will never be able to solve things we truly cannot solve yet. It might catch things that we didn't project properly or brute force things no human can , but it will never unify general relativity with quantum mechanics. It's amazing at finding hidden truths in large datasets, but won't win a Nobel unassisted.

In general, most researchers already incorporate LLM into their workflows, as it is quite good at context search. However, the relevant training data is based on the collective works of the field of experts. Collecting current data on that work is what makes the LLM sound relevant, and any improvement of the LLM model requires frequent new data from both researchers and the chat bot users themselves. LLM are not real "AI", and anyone that says otherwise is selling people something.

To phrase this differently, LLM companies conduct unauthorized targeted intelligence gathering on peoples work, codify that act of plagiarism or theft as MoE documentation, and sell unaccountable token output to other users.

There is a reason output becomes more nonsensical as "AI" companies try to use dynamic weight granularity and conceptual compaction. It is not necessarily "AI" hallucinations, but rather people fooling themselves into believing smart people are no longer needed if they willingly become a hapless exploited data source caste. This simply isn't true, as people will leave the field for awhile.

The LLM business model regularly requires copyright theft and plagiarism to persist. It will not magically become sentient/AGI/less-stupid, as these algorithms have been operating for over 40 years. What has changed is the scale of the deployment, data pool size, and the energy consumed.

Scientists are still necessary, as they create the world models LLM try to guess at by statistical inference. Hype and FUD ahead of an IPO for a highly dubious revenue company is expected. We look forward to the low cost liquidated GPU hardware in the near future. =3