Scientific research created with AI is polluting the Internet’s knowledge ecosystem, according to a worrying report published by the Harvard Kennedy School’s. Misinformation Review.
The research team investigated the frequency of research articles with pseudo-generated text evidence in Google Scholar, an academic search engine that facilitates the search for historically published research in a wealth of academic journals.
The team specifically investigated the misuse of pre-trained generative transformers (or GPTs), a type of large-scale language modeling (LLM) that includes software now known as OpenAI’s ChatGPT. These models are able to quickly interpret text input and quickly generate responses, in the form of figures, images, and long lines of text.
In the study, the team analyzed a sample of scientific papers found on Google Scholar that featured GPT usage. Selected papers contain one or two common phrases used by chat agents (usually, chatbots) for LLMs. The researchers then investigated the extent to which those questionable documents were distributed and handled across the Internet.
“The risk of what we call ‘evidence hacking’ increases significantly when AI-generated research is distributed to search engines,” said Björn Eksström, researcher at the Swedish School of Library and Information Science, and author of the paper. release of the University of Borås. “This can have significant consequences as the negative effects can permeate the community and possibly many areas.”
The way Google Scholar pulls research from the Internet, according to the latter team, does not release papers by their authors without scientific collaboration or peer review; The engine will pull in academic bycatch—student papers, reports, past papers, and more—as well as research that has passed a high level of scrutiny.
The team found that two-thirds of the papers they read were at least partially produced by undisclosed uses of GPTs. Of the GPT papers, the researchers found that 14.5% were related to health, 19.5% related to the environment, and 23% related to computing.
“Most of these GPT papers are found in non-refereed journals and working papers, but some cases include research published in general scientific journals and conference proceedings,” the team wrote.
Researchers have identified two main risks posed by this development. “First, the abundance of fictional ‘lessons’ entering every area of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the history of science,” the group wrote. “The second danger is the increasing possibility that content that looks scientifically convincing is actually created artificially by AI tools and re-optimized for retrieval by publicly available search engines, especially Google Scholar.”
Because Google Scholar is not an academic database, it is easy for the public to use when searching for scientific literature. That’s good. Unfortunately, it is difficult for members of the public to separate the wheat from the chaff when it comes to reputable journals; even the difference between a piece of peer-reviewed research and a working paper can be confusing. Additionally, the AI-generated text was found in some of the peer-reviewed works as well as in those unreviewed works, indicating that GPT-generated work is muddying the waters of the entire online educational information system—not just that work. it exists without many official channels.
“If we cannot trust that the research we read is true, we risk making decisions based on incorrect information,” said study co-author Jutta Haider, who is also a researcher at the Swedish School of Library and Information Science, in the same release. “But as much as this is a question of scientific misconduct, it’s a question of media and literacy.”
In recent years, publishers have failed to successfully investigate a number of scientific articles that were in fact complete nonsense. In 2021, Springer Nature was forced to withdraw more than 40 papers from Arabian Journal of Geoscienceswhich, despite the magazine’s title, discussed a variety of topics, including sports, air pollution, and children’s medicine. In addition to being off topic, these articles were poorly written—to the point of nonsensical—and the sentences often lacked a clear line of thought.
Artificial intelligence exacerbates this issue. Last February, Frontiers publishers participated in publishing a paper in their journal Cell again Developmental biology featuring images generated by AI Midjourney software; especially, a lot anatomically correct images of the signaling pathways and genitalia of the mouse. Frontiers retracted the paper a few days after it was published.
AI models can be a boon to science; systems can decipher faint inscriptions from the Roman Empire, discover the previously unknown Nazca Lines, and reveal hidden details in dinosaur fossils. But the impact of AI can be as good or as bad as the person who runs it.
Peer-reviewed journals—and perhaps hosts and search engines for academic writing—need vigilance to ensure that technology works for scientific discovery, not against it.