Remix.run Logo
▲ markasoftware an hour ago

Have you tried doing this recently? For my work on software at least, the latest LLMs have a very low false positive rate for bugs. They almost never hallucinate issues. My understanding is the same applies to math.

▲root_axis an hour ago | parent | next [-]

I'm not arguing that LLMs aren't useful tools for mathematicians, I'm questioning the quality of discourse that amounts to "I pasted this into the LLM and it said there are errors".

▲fasterik an hour ago | parent | next [-]

There's value in knowing whether a claim is true or false. Plugging things into an LLM doesn't necessarily give 100% certainty or tell us why something is true or false in a human-comprehensible way, but it can move our credences and suggest directions for investigation.

▲whimsicalism 39 minutes ago | parent | prev [-]

classic HN isolated demand for rigor. i found it interesting

▲mahogany an hour ago | parent | prev | next [-]

I would guess that LLMs are much worse at math without having a theorem prover to validate approaches with. Since this draft is not formalized, an LLM is operating purely on the text which is more tenuous.

▲shmoil an hour ago | parent | prev | next [-]

Pre-chatgpt 5.5 it hallucinated quiet a bit on hard math. Starting from 5.6 it is incredibly good.

▲gregorygoc an hour ago | parent | prev | next [-]

I’m doing it daily and it finds a bunch of issues which are not issues in practice.

▲simplyluke 31 minutes ago | parent | prev | next [-]

I suspect LLMs are much better trained and more reliable at bug identification in react apps than draft-state proofs on the bleeding edge of math, on problems that are specifically highlighted as extremely difficult to solve without error in their training.

▲acedTrex 9 minutes ago | parent | prev | next [-]

> the latest LLMs have a very low false positive rate for bugs.

Man people live in different worlds, astra for me finds 25/6 bugs all the time.

▲SubmarineClub 27 minutes ago | parent | prev [-]

Cool story.

I have the opposite experience in my line of work.

▲chaps 25 minutes ago | parent [-]

What's your line of work?