AI models misrepresent news almost half of the time, study finds | Media news

According to the research, 45 percent of responses to questions about the news had at least one “significant” topic.

AI models like ChatGPT routinely misrepresent news events and provide wrong answers to questions almost half of the time, according to a study.

The study published on Wednesday by the European Broadcasting Union (EBU) and the BBC evaluated the accuracy of more than 2,700 responses given by OpenAI's ChatGPT, Google's Gemini, Microsoft's Copilot and Perplexity.

Twenty-two public media outlets, representing 18 countries and 14 languages, posed a common set of questions to AI attendees between late May and early June for the study.

Overall, 45 percent of responses had at least one “significant” problem, according to the research.

Sourcing was the most common issue, with 31 percent of responses including information not supported by the cited source or incorrect or unverifiable attribution, among other issues.

Lack of precision was the next largest contributor to erroneous responses, affecting 20 percent of responses, followed by lack of appropriate context, at 14 percent.

Gemini had the largest issues, primarily related to sourcing, with 76 percent of responses affected, according to the study.

According to the research, all AI models studied made basic factual errors.

Errors cited include Perplexity claiming surrogacy is illegal in Czechia and ChatGPT naming Pope Francis as acting pontiff months after his death.

OpenAI, Google, Microsoft and Perplexity did not immediately respond to requests for comment.

In a foreword to the report, Jean Philip De Tender, deputy general at the EBU, and Pete Archer, head of AI at the BBC, called on technology companies to do more to reduce errors in their products.

“They have not prioritized this issue and they need to do so now,” De Tender and Archer said.

“They must also be transparent and regularly publish their results by language and market.”