Research reveals that source verification issues undermine the reliability of ChatGPT.

2023-05-15

ChatGPT seems to be everywhere. Morgan Stanley, Duolingo, Snapchat, Coca-Cola, and Instacart have already signed up, as have programmers, web designers, pharmaceutical companies, writers, musicians, translators, and all types of businesses.

Source: Pixabay/CC0 Public Domain

ChatGPT is only available for general use within six months and already has over 100 million users worldwide, processing over 10 million queries every day. There are stories almost every day detailing its potential to disrupt everything from business models to personal daily life. Everyone wants a piece of the action. However, some have raised concerns about the negative impact of ChatGPT, which is the fastest-growing application in history. The most notable warning came in a letter signed by over 1,000 experts in March, urging a pause in the expansion of artificial intelligence technology until its risks and dangers are better understood. While skeptics primarily focus on the possibility of malicious actors manipulating AI for harmful purposes or envision AI models autonomously deciding to engage in malicious actions, others are concerned about a more pressing issue: accuracy. The Wall Street Journal recently warned that "AI chatbots and other generative AI programs reflect the data they consume. They repeatedly input and mix input data, with both great success and massive failures." A human-centered AI research group at Stanford University published a paper on the reliability of data retrieved in large language model searches on the arXiv preprint server last month. "A prerequisite for a trustworthy generative search engine is verifiability," said Nelson Liu, a Stanford doctoral student and one of the authors of the report. After studying the output of four popular search engines, Liu and his colleagues Tianyi Zhang and Percy Jiang reported that the results were "fluent and informative but often contained unverified statements and inaccurate citations." The generative search engines they studied were Bing Chat, NeevaAI, perplexity.ai, and YouChat. The topics ranged from biographical data on singer Alicia Keys to review questions on social media. The researchers examined four features: fluency, perceived utility (how helpful the answers were), citation recall (how well the citations fully supported the consistent generated statements), and citation accuracy (the proportion of generated citations supporting relevant statements). Trustworthy generative search engines were defined as those capable of achieving high citation recall rates and accuracy. The results were discouraging. The team found that responses "typically had high fluency and perceived utility but often contained unverified statements or inaccurate citations." Only about half of the generated sentences received sufficient citation support, and a quarter of the citations did not support the relevant sentences. Furthermore, the researchers found a negative correlation between citation recall rates and accuracy with fluency and perceived utility. They observed, "Answers that appear more helpful tend to be those with unsupported statements or inaccurate citations." As a result, they concluded, "This veneer of credibility increases the likelihood of existing generative search engines misleading users." An article published this week on ExtremeTech discussed the issue of sources: "Chatbots like ChatGPT and Bing Chat excel at making false information appear true. It's difficult to distinguish accuracy from falsehood, especially within the few seconds users spend on search engine result pages where most chatbot results lack citations." The Stanford researchers stated that their findings are "particularly concerning for systems that may serve as the primary tool for information-seeking users, especially considering their credibility." The researchers expressed hope that their study will "further drive the development of trustworthy generative search engines and help researchers and users better understand the shortcomings of existing commercial systems."

RECENT AI NEWS

RECENT AI TOOLS