Detecting AI-generated Text in Scientific Journals

In an era when the academic community is highly concerned about the generation of AI-generated papers, reassuring news has come from the University of Kansas.

Researchers have developed an AI text detector for scientific papers that can almost 100% distinguish between human-written content and computer-generated content.

In a study published on November 6th in Cell Reports Physical Science, Professor Heather Desaire pointed out that although there are several AI detectors for general content, their effectiveness is not ideal when applied to scientific papers.

"Most people in the field of text analysis hope for a truly universal detector that can be applied to various domains," Desaire said. In contrast, her team focuses on detecting reports in chemistry journals.

"We are really pursuing accuracy," she said.

The team's detector was trained on journals published by the American Chemical Society. They collected 100 introductory articles written by professionals and then programmed ChatGPT to write its own introductions based on the journal abstract or just the report title.

When the ChatGPT detector scans these three types of reports, it correctly identifies human-written content almost 100% of the time, and it can accurately identify reports generated based solely on the report title. For reports that rely on introductory paragraphs, the correct identification rate is 98%.

However, competitors like ZeroGPT have an accuracy rate of 98% in detecting general AI articles, but perform poorly in chemistry-related reports. The average accuracy rate for reports based on the same title is only 37%, and the accuracy rate for reports based on introductory text is only a few percentage points higher.

The situation is even worse for the second competitor, OpenAI, which cannot correctly identify the authors of papers with an average accuracy rate of up to 80%.

"Academic publishers are working hard to cope with the rapid proliferation of new AI text generators," Desaire said. "This new detector will enable the scientific community to assess the penetration of ChatGPT into chemistry journals, determine the consequences of its use, and quickly introduce mitigation strategies when problems arise."

Scientific journals are rewriting their rules regarding article submissions, with most journals prohibiting AI-generated reports and requiring disclosure of any other AI processes used in writing the reports.

Desaire listed some concerns about the risk of AI-generated content infiltrating scientific journals: "Their excessive use may lead to an abundance of low-value articles. They may cause highly cited papers to be overrepresented while overlooked emerging works remain unknown."

"Most concerning," she added, "is that these tools tend to create 'illusions' and fabricate unreal facts."

As an example, Desaire included a biography written by ChatGPT about herself. It claimed that she "graduated from the University of Minnesota, is a member of the Royal Society of Chemistry, and has been awarded the Biemann Medal." These impressive achievements are all false.

"While this example is amusing, it is not funny at all to have lies infiltrate scientific literature," Desaire said.

But she remains optimistic. She said that some people believe that resisting the emergence of AI-generated content is inevitable, and they think that "developing such tools is participating in a (human vs. AI) arms race that humans cannot win."

She said that editors must take the lead in detecting AI pollution.

"Journals should take reasonable measures to ensure compliance with their policies regarding AI writing, and we believe that staying ahead in detecting AI issues is entirely feasible," she said.