OpenAI researchers have found a way to stop ChatGPT from making things up, but their solution could force the artificial intelligence (AI) to admit ignorance on nearly one-third of user questions.
The September 2025 research paper explains why large language models confidently generate false information. Current testing methods reward AI systems for guessing wrong answers rather than saying “I don’t know.”
“Language models are optimized to be good test-takers, and guessing when uncertain improves test performance,” the OpenAI researchers wrote.
Like in a multiple-choice test, students who guess might get lucky. Those who leave blanks get zero points. AI models face the same pressure.
When researchers asked various AI models for co-author Adam Kalai’s birthday, one system gave three different wrong dates across separate attempts. None came close to his actual autumn birthday.
The mathematical fix involves changing how AI systems get graded. Instead of only measuring accuracy, evaluations should heavily penalize confident mistakes while rewarding expressions of uncertainty.
But this solution creates a serious problem. OpenAI’s analysis suggests models would need to abstain from answering up to 30% of queries to avoid hallucinations.
Users expect instant, authoritative responses from ChatGPT. An AI that frequently admits ignorance might drive people toward competitors that prioritize confidence over accuracy.
The trade-off mirrors real-world scenarios. Wei Xing, who studies AI at the University of Sheffield, compared it to air-quality monitoring in Salt Lake City. When systems flag measurement uncertainties, user engagement drops noticeably compared to displays showing confident readings.
GPT-5 already shows reduced hallucination rates, especially when allowed to browse the web for information. On one benchmark testing citation accuracy, GPT-5 made errors 39% of the time without internet access, but only 0.8% with web browsing enabled.
“For most cases of hallucination, the rate has dropped to a level” that seems “acceptable to users,” said Tianyang Xu, an AI researcher at Purdue University. However, technical fields like law and mathematics still trip up GPT-5.
The economic factors complicate matters further. Uncertainty-aware models require significantly more computational power. They must evaluate multiple possible responses and estimate confidence levels for each query.
Such costs make sense for high-stakes applications, such as medical diagnosis or financial trading, where mistakes can cost millions. For everyday consumer use, the economics become prohibitive.
The research team argues that widespread adoption requires changing industry evaluation standards.
Major AI benchmarks from Google, OpenAI, and leading leaderboards use a binary grading system. Nine out of ten benchmarks examined by the researchers award zero points when models express doubt, creating an “epidemic” of penalizing honest responses.
OpenAI’s proposed reforms could reshape how the entire AI industry develops and tests language models. The question remains whether users will accept more honest but less confident AI assistants.