OpenAI halts its AI writing detector due to "low accuracy"

2023-07-27

On Thursday, OpenAI quietly retired its artificial intelligence classifier, an experimental tool designed to detect AI-written text. Decrypt was the first to notice the retirement, which was not announced with much fanfare, but rather with a small note on the official OpenAI AI Classifier webpage stating, "As of July 20, 2023, the AI Classifier is no longer available due to its low accuracy. We are actively incorporating feedback and are currently researching more effective text attribution techniques, and we commit to developing and deploying mechanisms."

OpenAI's AI Classifier was released on January 31, with educators expressing strong concerns about students potentially using ChatGPT to write articles and assignments. From the beginning, OpenAI acknowledged that its AI Classifier was not "fully reliable," correctly identifying only 26% of AI-generated text as "likely AI-written" and erroneously marking human-written works 9% of the time.

As we pointed out on Ars, the accuracy of AI writing detectors such as OpenAI's AI Classifier, Turnitin, and GPTZero is insufficient to rely on them for trustworthy results. Their methodologies still involve speculation and unverified issues, and these tools are frequently used to falsely accuse students of cheating.

Humans can write articles similar to those generated by AI models, and AI models can also write articles similar to those written by humans if given appropriate prompts. However, this has not stopped the emergence of a small number of commercial AI detectors in the past six months.

"If OpenAI cannot make its AI detection tool work properly, no one else can," wrote AI writer and futurist Daniel Jeffries on Twitter. "I've said before that AI detection tools are snake oil scams, and this just further proves they are useless. Don't trust them. They are all nonsense."

These views are currently supported by recent research (Sadasivan et al., 2023) and testimonies from educators. Educators often find their own human writing being flagged as AI-generated. Additionally, AI writing detectors have been found to unfairly penalize non-native English writers and writers with differing viewpoints.

Research is currently underway to determine if watermarks can be added to AI-generated text by deliberately manipulating the frequency of words in the AI-generated output. However, the aforementioned research shows that text watermarks can easily be defeated by rewriting the output of AI models.

At present, AI writing has become commonplace. Looking ahead, AI-enhanced text is likely to flow undetected within great human works if used skillfully. Perhaps it is time to go beyond the composition of text and ensure that it accurately expresses the specific human intent, which is the purpose of all effective communication.



Source: https://arstechnica.com/information-technology/2023/07/openai-discontinues-its-ai-writing-detector-due-to-low-rate-of-accuracy/

RECENT AI NEWS

RECENT AI TOOLS