We've all what a terrible job LLMs do in some problems. But they are doing well at burning down Erdos's list of problems. Perhaps mathematicians aren't a good measure of "truly smart".

If you're seriously taking that angle on comparison .. how well are they doing on generating interesting and challenging Erdős like problems?