> The easiest way of solving math problems with an LLM is to make sure that very similar problems are included in the training set.

An irony here is that math blogs like Tao's might not be in LLM training data, for the same reason they aren't accessible to screen readers - they're full of math, and the math is rendered as images, so it's nonsense if you can't read the images.

(The images on his blog do have alt text, but it's just the LaTeX code, which isn't much better.)

▲

alansammarone 2 days ago | parent | next [-]

As others have pointed out, LLMs have no trouble with LaTeX. I can see why one might think they're not - in fact, I made the same assumption myself sometime ago. LLMs, via transformers, are exceptionally good any _any_ sequence or one-dimensional data. One very interesting (to me anyway) example is base64 - pick some not-huge sentence (say, 10 words), base64-encode it, and just paste it in any LLM you want, and it will be able to understand it. Same works with hex, ascii representation, or binary. Here's a sample if you want to try: aWYgYWxsIEEncyBhcmUgQidzLCBidXQgb25seSBzb21lIEIncyBhcmUgQydzLCBhcmUgYWxsIEEncyBDJ3M/IEFuc3dlciBpbiBiYXNlNjQu

I remember running this experiment some time ago in a context where I was certain there was no possibility of tool use to encode/decode. Nowadays, it can be hard to certain whether there is any tool use or not, in some cases, such as Mistral, the response is quick enough to make it unlikely there's any tool use.

▲

throwanem a day ago | parent [-]

I've just tried it, in the form of your base64 prompt and no other context, with a local Qwen-3 30b instance that I'm entirely certain is not actually performing tool use. It produced a correct answer ("Tm8="), which in a moment of accidental comedy it spontaneously formatted with LaTeX. But it did talk about invoking an online decoder, just before the first appearance of the (nearly) complete decoded string in its CoT.

It "left out" the A in its decode and still correctly answered the proposition, either out of reflexive familiarity with the form or via metasyntactic reasoning over an implicit anaphor; I believe I recall this to be a formulation of one of the elementary axioms of set theory, though you will excuse me for omitting its name before coffee, which makes the pattern matching possibility seem somewhat more feasible. ('Seem' may work a little too hard there. But a minimally more novel challenge I think would be needed to really see more.)

There's lots of text in lots of languages about using an online base64 decoder, and nearly none at all about decoding the representation "in your head," which for humans would be a party trick akin to that one fellow who could see a city from a helicopter for 30 seconds and then perfectly reproduce it on paper from memory. It makes sense to me that a model trained on the Internet would "invent" the "metaphor" of an online decoder here, I think. What in its "experience" serves better as a description?

▲

kaffekaka 17 hours ago | parent [-]

I assume you're referring to Stephen Wiltshire: https://en.m.wikipedia.org/wiki/Stephen_Wiltshire

	▲	throwanem 13 hours ago \| parent [-]
		I am! Good grief, it must have been thirty years ago I saw that news story, and apparently I misremembered several whole decades onto his age; I hadn't imagined he would still be alive. Thank you!

▲

prein 2 days ago | parent | prev | next [-]

What would be a better alternative than LaTex for the alt text? I can't think of a solution that makes more sense, it provides an unambiguous representation of what's depicted.

I wouldn't think an LLM would have issue with that at all. I can see how a screen reader might, but it seems like the same problem faced by a screen reader with any piece of code, not just LaTex.

▲

mbowcut2 a day ago | parent | prev | next [-]

LLMs are better at LaTeX than humans. ChatGPT often writes LaTeX responses.

	▲	neutronicus a day ago \| parent [-]
		Yeah, it's honestly one of the things they're best at! I've been working on implementing some E&M simulations with Claude Code and it's so-so on the C++ and TERRIBLE at the actual math (multiplying a couple 6x6 matrix differential operators is beyond it). But I can dash off some notes and tell Claude to TeXify and the output is great.

▲

QuesnayJr 2 days ago | parent | prev | next [-]

LLMs understand LaTeX extraordinarily well.

▲

constantcrying a day ago | parent | prev | next [-]

>(The images on his blog do have alt text, but it's just the LaTeX code, which isn't much better.)

LLMs are extremely good at outputting LaTeX, ChatGPT will output LaTeX, which the website will render as such. Why do you think LLMs have trouble understanding it?

	▲	astrange a day ago \| parent [-]
		I don't think LLMs will have trouble understanding it. I think people using screen readers will. …oh I see, I accidentally deleted the part of the comment about that. But the people writing the web page extraction pipelines also have to handle the alt text properly.

▲

MengerSponge 2 days ago | parent | prev [-]

LLMs are decent with LaTeX! It's just markup code after all. I've heard from some colleagues that they can do decent image to code conversion for a picture of an equation or even some handwritten ones.