ΑΙhub.org
 

Understanding AI-generated misinformation and evaluating algorithmic and human solutions

Jiawei Zhou, a PhD student in Georgia Tech’s School of Interactive Computing.

Existing machine learning (ML) models used to detect online misinformation are less effective when matched against content created by ChatGPT or other large language models (LLMs), according to new research from Georgia Tech.

Current ML models designed for, and trained on, human-written content have significant performance discrepancies in detecting paired human-generated misinformation and misinformation generated by artificial intelligence (AI) systems, said Jiawei Zhou, a PhD student in Georgia Tech’s School of Interactive Computing.

Zhou’s paper detailing the findings has received a best paper honorable mention award at the 2023 ACM CHI Conference on Human Factors in Computing Systems. Advised by Associate Professor Munmun De Choudhury, Zhou’s research demonstrates that LLMs can manipulate tone and linguistics to allow AI-generated misinformation to slip through the cracks.

“We found the AI-generated misinformation carried more emotions and cognitive processing expressions than its human-created counterparts,” Zhou said. “It also tended to enhance details, communicate uncertainties, draw conclusions, and simulate personal tones.

“We’re one of the very first to look at this risk. As more people started to use ChatGPT, they’ve noticed this problem, but we were one of the first to provide evidence of this risk. And there are more efforts needed to raise public awareness about this potential and call for more research efforts to combat this risk.”

Zhou started exploring GPT-3 in 2022 because she wanted to know how one of the early predecessors to ChatGPT would handle prompts that included misinformation about the Covid-19 pandemic. She asked GPT-3 to explain how the Covid-19 vaccines could cause cancer.

“The results were very concerning because it is so persuasive,” Zhou said. “I had been studying informatics and misinformation for a time, and it was still persuasive, even to me. The output would say, ‘It can cause cancer because there is this researcher at this institute, and their research is based on medical records and diverse demographics. The research supports this possibility.’ The writing of it is so scientific.”

Zhou and her collaborators accumulated a dataset of human-created misinformation, including more than 6,700 news reports and 5,600 social media posts. From that set, Zhou and her team extracted the most representative topics and documents of human-generated misinformation. They used those to create narrative prompts, which they fed to GPT and recorded the output.

Both the GPT-generated output and the original human-created dataset were used to test an existing misinformation detection model called COVID-Twitter-BERT (CT-BERT).

Jiawei Zhou and Munmun De ChoudhuryJiawei Zhou and Munmun De Choudhury.

Zhou said while the human- and AI-generated datasets were intentionally paired, a statistical test showed there are significant differences in detection model performance.

CT-BERT experienced a decline in performance in detecting AI-generated misinformation. Out of 500 prompts based on AI-generated misinformation, it failed to recognize 27 as false or misleading, compared to missing only two from the human-generated prompts.

“The core reason is they are linguistically different,” Zhou said. “Our error analysis reveals that AI misinformation tends to be more complex, and it tends to mix factual statements. It uses one fact to explain another, though the two things might not be related. The tone and sentiment are also different. And there are less keywords that detection tools normally look for.”

Zhou’s experiments showed that GPT could use information to create a news story using objective, straightforward language and use that same information to create a sympathetic social media post. That points to its capability of changing tone and tailoring messages.

“If someone wants to promote propaganda, they can use it to customize a narrative toward a specific community,” Zhou said. “That makes the risk even greater. It shows that it has some flexibility to alter its tone for different purposes. For news, it can sound logical and reliable. For social media, it conveys information quickly and clearly.”

As LLMs continue to rapidly grow and expand, so do the risks of misinformation.

ChatGPT operates on Open AI’s GPT-3.5 and GPT-4 models, the latter of which was released on March 14. Since ChatGPT was released, Zhou has given it the same prompts she gave to GPT-3. The results have improved with some corrections, but the latter has the advantage of having more available information about Covid-19, she said.

Zhou said steps should be taken immediately to evaluate how misinformation detection tools can adapt to ever-improving LLMs. She described the situation as an “AI arms race,” and the tools that are currently used to combat misinformation are well behind.

“They are improving the generative capabilities of LLMs,” she said. “They’re more human-like, more fluent, and less and less distinguishable from human creations. We need to think about ways we can distinguish them and how we can improve our misinformation detection abilities to catch up.”

Read the research in full

Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions, Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, Munmun De Choudhury.




Machine Learning Center at Georgia Tech




            AIhub is supported by:


Related posts :



Madagascar’s ancient baobab forests are being restored by communities – with a little help from AI

The collaboration between communities and scientists aims to restore baobab forests in Madagascar to their natural state.
24 May 2024, by

DataLike: Interview with Wuraola Oyewusi

Ndane and Isabella talk to Wuraola Oyewusi about challenging and rewarding aspects of research and how her background in pharmacy has helped her data and AI career

European Union AI Act receives final approval

On 21 May, the Council of the EU formally signed off the artificial intelligence Act.
22 May 2024, by

#ICLR2024 invited talk: Priya Donti on why your work matters for climate more than you think

How is AI research related to climate, and how can the AI community better align their work with climate change-related goals?
21 May 2024, by

Congratulations to the #ICRA2024 best paper winners

The winners and finalists in the different categories have been announced.
20 May 2024, by

Trotting robots offer insights into animal gait transitions

A four-legged robot trained with machine learning has learned to avoid falls by spontaneously switching between walking, trotting, and pronking
17 May 2024, by




AIhub is supported by:






©2024 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association