Study Urges Overhaul of AI Testing to Curb False Claims
A new study by OpenAI warns that despite progress, hallucinations—false but plausible statements made by chatbots—pose a persistent problem for artificial intelligence.
The researchers argue that the way AI systems are tested may worsen the issue. Current evaluation methods reward accuracy but do not account for the harm of confident errors. This, they say, leads models to produce misleading answers instead of acknowledging uncertainty.
The findings were highlighted through tests on a popular chatbot. It repeatedly gave incorrect responses when asked basic questions about Adam Tauman Kalai, one of the study’s authors.
The paper suggests changing the evaluation framework to discourage blind guessing, comparing the problem to multiple-choice exams where guessing can lead to accidental success.
Read More: Obama Beats Attenborough, Hanks, Elba to Emmy Win
By penalising wrong answers more heavily and rewarding expressions of uncertainty, researchers believe chatbots could become more reliable.
The study also stresses that hallucinations arise from the way models are trained. Since pretraining relies on predicting word sequences without distinguishing truth from fiction, errors with rare facts remain common.
Experts say addressing these flaws is essential to build public trust in AI. Without new evaluation systems, hallucinations will continue to undermine the credibility of technologies like ChatGPT and other advanced models.