AI tool achieves 94% accuracy in telling apart fake from real research papers
ChatGPT. In a set of 300 fake and real scientific papers, the AI-based tool, named 'xFakeSci', detected up to 94 per cent of the fake ones.
This was nearly twice the success rate seen among the more common data-mining techniques, the authors from the State University of New York, US, and Hefei University of Technology, China, said.
"… we introduce xFakeSci, a novel learning algorithm, that is capable of distinguishing ChatGPT-generated articles from publications produced by scientists," they wrote in the study published in the journal Scientific Reports.
For developing the AI-based algorithm, the researchers developed two distinct datasets. One of them contained almost 4,000 scientific articles taken from PubMed, an open database housing biomedical and life sciences research papers and maintained by the US National Institutes of Health.
The other consisted of 300 fake articles, which the researchers created using ChatGPT.
«I tried to use exact same keywords that I used to extract the literature from the PubMed database, so we would have a common basis of comparison. My intuition was that there must be a pattern exhibited in the fake world versus the actual world, but I had no idea what this pattern was,» the study's co-author Ahmed Abdeen Hamed, a visiting research fellow at the State University of New York, said.
Of the 300 fake articles, 100 each were related to the medical conditions Alzheimer's disease, cancer, and depression. Each of the 100 included 50 chatbot-created articles and 50 authentic abstracts
