Why ChatGPT Is Getting Dumber at Basic Math
Since becoming widely available to the public last year, artificial-intelligence chatbots have dazzled people who experimented with them, kicked off a global development race and even contributed to the strike in Hollywood over their impact on writers and actors. AI tools have also generated fear that they will inexorably improve and threaten humanity. OpenAI’s ChatGPT debuted to the public in November, sparking the current frenzy, followed by Chat GPT-4 in March, meant to be more powerful than its predecessor.
But new research released this week reveals a fundamental challenge of developing artificial intelligence: ChatGPT has become worse at performing certain basic math operations. The researchers at Stanford University and the University of California, Berkeley said the deterioration is an example of a phenomenon known to AI developers as drift, where attempts to improve one part of the enormously complex AI models make other parts of the models perform worse. “Changing it in one direction can worsen it in other directions," said James Zou, a Stanford professor who is affiliated with the school’s AI lab and is one of the authors of the new research.
“It makes it very challenging to consistently improve." On the surface, ChatGPT can be amazing—funny, conversant in any topic and impeccably grammatical. Some people have given ChatGPT standardized tests that it nailed. But other times the chatbot will flub even basic mathematics.
The goal of the team of researchers, consisting of Lingjiao Chen, a computer-science Ph.D. student at Stanford, along with Zou and Berkeley’s Matei Zaharia, is to systematically and repeatedly see how the models perform over time at a range of tasks. Thus far, they have tested two versions of
. Read more on livemint.com
