Put your math hats on for a minute, and let’s check out what this beef from mid-October was about. It’s an ideal instance of what’s mistaken with AI proper now.
Bubeck was excited that GPT-5 appeared to have someway solved numerous puzzles generally known as Erdős issues.
Paul Erdős, some of the prolific mathematicians of the twentieth century, left behind a whole bunch of puzzles when he died. To assist preserve observe of which of them have been solved, Thomas Bloom, a mathematician on the College of Manchester, UK, arrange erdosproblems.com, which lists greater than 1,100 issues and notes that round 430 of them include options.
When Bubeck celebrated GPT-5’s breakthrough, Bloom was fast to call him out. “It is a dramatic misrepresentation,” he wrote on X. Bloom defined that an issue isn’t essentially unsolved if this web site doesn’t checklist an answer. That merely means Bloom wasn’t conscious of 1. There are thousands and thousands of arithmetic papers on the market, and no person has learn all of them. However GPT-5 in all probability has.
It turned out that as an alternative of developing with new options to 10 unsolved issues, GPT-5 had scoured the web for 10 current options that Bloom hadn’t seen earlier than. Oops!
There are two takeaways right here. One is that breathless claims about huge breakthroughs shouldn’t be made by way of social media: Much less knee jerk and extra intestine verify.
The second is that GPT-5’s means to search out references to earlier work that Bloom wasn’t conscious of can also be superb. The hype overshadowed one thing that ought to have been fairly cool in itself.
Mathematicians are very concerned about utilizing LLMs to trawl by huge numbers of current outcomes, François Charton, a analysis scientist who research the appliance of LLMs to arithmetic on the AI startup Axiom Math, informed me after I talked to him about this Erdős gotcha.
