ΑΙhub.org
 

Understanding AI-generated misinformation and evaluating algorithmic and human solutions

Jiawei Zhou, a PhD student in Georgia Tech’s School of Interactive Computing.

Existing machine learning (ML) models used to detect online misinformation are less effective when matched against content created by ChatGPT or other large language models (LLMs), according to new research from Georgia Tech.

Current ML models designed for, and trained on, human-written content have significant performance discrepancies in detecting paired human-generated misinformation and misinformation generated by artificial intelligence (AI) systems, said Jiawei Zhou, a PhD student in Georgia Tech’s School of Interactive Computing.

Zhou’s paper detailing the findings has received a best paper honorable mention award at the 2023 ACM CHI Conference on Human Factors in Computing Systems. Advised by Associate Professor Munmun De Choudhury, Zhou’s research demonstrates that LLMs can manipulate tone and linguistics to allow AI-generated misinformation to slip through the cracks.

“We found the AI-generated misinformation carried more emotions and cognitive processing expressions than its human-created counterparts,” Zhou said. “It also tended to enhance details, communicate uncertainties, draw conclusions, and simulate personal tones.

“We’re one of the very first to look at this risk. As more people started to use ChatGPT, they’ve noticed this problem, but we were one of the first to provide evidence of this risk. And there are more efforts needed to raise public awareness about this potential and call for more research efforts to combat this risk.”

Zhou started exploring GPT-3 in 2022 because she wanted to know how one of the early predecessors to ChatGPT would handle prompts that included misinformation about the Covid-19 pandemic. She asked GPT-3 to explain how the Covid-19 vaccines could cause cancer.

“The results were very concerning because it is so persuasive,” Zhou said. “I had been studying informatics and misinformation for a time, and it was still persuasive, even to me. The output would say, ‘It can cause cancer because there is this researcher at this institute, and their research is based on medical records and diverse demographics. The research supports this possibility.’ The writing of it is so scientific.”

Zhou and her collaborators accumulated a dataset of human-created misinformation, including more than 6,700 news reports and 5,600 social media posts. From that set, Zhou and her team extracted the most representative topics and documents of human-generated misinformation. They used those to create narrative prompts, which they fed to GPT and recorded the output.

Both the GPT-generated output and the original human-created dataset were used to test an existing misinformation detection model called COVID-Twitter-BERT (CT-BERT).

Jiawei Zhou and Munmun De ChoudhuryJiawei Zhou and Munmun De Choudhury.

Zhou said while the human- and AI-generated datasets were intentionally paired, a statistical test showed there are significant differences in detection model performance.

CT-BERT experienced a decline in performance in detecting AI-generated misinformation. Out of 500 prompts based on AI-generated misinformation, it failed to recognize 27 as false or misleading, compared to missing only two from the human-generated prompts.

“The core reason is they are linguistically different,” Zhou said. “Our error analysis reveals that AI misinformation tends to be more complex, and it tends to mix factual statements. It uses one fact to explain another, though the two things might not be related. The tone and sentiment are also different. And there are less keywords that detection tools normally look for.”

Zhou’s experiments showed that GPT could use information to create a news story using objective, straightforward language and use that same information to create a sympathetic social media post. That points to its capability of changing tone and tailoring messages.

“If someone wants to promote propaganda, they can use it to customize a narrative toward a specific community,” Zhou said. “That makes the risk even greater. It shows that it has some flexibility to alter its tone for different purposes. For news, it can sound logical and reliable. For social media, it conveys information quickly and clearly.”

As LLMs continue to rapidly grow and expand, so do the risks of misinformation.

ChatGPT operates on Open AI’s GPT-3.5 and GPT-4 models, the latter of which was released on March 14. Since ChatGPT was released, Zhou has given it the same prompts she gave to GPT-3. The results have improved with some corrections, but the latter has the advantage of having more available information about Covid-19, she said.

Zhou said steps should be taken immediately to evaluate how misinformation detection tools can adapt to ever-improving LLMs. She described the situation as an “AI arms race,” and the tools that are currently used to combat misinformation are well behind.

“They are improving the generative capabilities of LLMs,” she said. “They’re more human-like, more fluent, and less and less distinguishable from human creations. We need to think about ways we can distinguish them and how we can improve our misinformation detection abilities to catch up.”

Read the research in full

Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions, Jiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G Parker, Munmun De Choudhury.




Machine Learning Center at Georgia Tech




            AIhub is supported by:


Related posts :



Latest AI announcements from the US Government include updated strategic plan

Find out more about the latest initiatives pertaining to responsible AI in the USA.
26 May 2023, by

Interview with Haotian Xue: learning intuitive physics from videos

A framework for learning 3D-grounded visual intuitive physics models from videos of complex scenes.
25 May 2023, by

Using engineered bacteria and AI to sense and record environmental signals

Synthetic biologists engineer bacterial swarm patterns to visibly record environment and use deep learning to decode patterns.

Writing with AI help can shift your opinions

A study investigates whether a language-model-powered writing assistant that generates some opinions more often than others impacts what users write – and what they think.
23 May 2023, by

AI is helping astronomers make new discoveries and learn about the universe faster than ever before

As the technology has become more powerful, AI algorithms have begun helping astronomers tame massive data sets and discover more about the universe.
22 May 2023, by

The Good Robot Podcast: live from the AI Anarchies conference in Berlin

In this episode, hosts Eleanor Drage and Kerry Mackereth chat to Christina Lu and Grace Turtle.
19 May 2023, by





©2021 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association