| ▲ | markasoftware an hour ago | |||||||||||||
Have you tried doing this recently? For my work on software at least, the latest LLMs have a very low false positive rate for bugs. They almost never hallucinate issues. My understanding is the same applies to math. | ||||||||||||||
| ▲ | root_axis an hour ago | parent | next [-] | |||||||||||||
I'm not arguing that LLMs aren't useful tools for mathematicians, I'm questioning the quality of discourse that amounts to "I pasted this into the LLM and it said there are errors". | ||||||||||||||
| ||||||||||||||
| ▲ | mahogany an hour ago | parent | prev | next [-] | |||||||||||||
I would guess that LLMs are much worse at math without having a theorem prover to validate approaches with. Since this draft is not formalized, an LLM is operating purely on the text which is more tenuous. | ||||||||||||||
| ▲ | shmoil an hour ago | parent | prev | next [-] | |||||||||||||
Pre-chatgpt 5.5 it hallucinated quiet a bit on hard math. Starting from 5.6 it is incredibly good. | ||||||||||||||
| ▲ | gregorygoc an hour ago | parent | prev | next [-] | |||||||||||||
I’m doing it daily and it finds a bunch of issues which are not issues in practice. | ||||||||||||||
| ▲ | simplyluke 31 minutes ago | parent | prev | next [-] | |||||||||||||
I suspect LLMs are much better trained and more reliable at bug identification in react apps than draft-state proofs on the bleeding edge of math, on problems that are specifically highlighted as extremely difficult to solve without error in their training. | ||||||||||||||
| ▲ | acedTrex 9 minutes ago | parent | prev | next [-] | |||||||||||||
> the latest LLMs have a very low false positive rate for bugs. Man people live in different worlds, astra for me finds 25/6 bugs all the time. | ||||||||||||||
| ▲ | SubmarineClub 27 minutes ago | parent | prev [-] | |||||||||||||
Cool story. I have the opposite experience in my line of work. | ||||||||||||||
| ||||||||||||||