ΑΙhub.org
 

Use of AI to fight COVID-19 risks harming “disadvantaged groups”, experts warn

by
28 July 2021



share this:

covid world map
COVID-19 world map. Credit: Martin Sanchez. Published under a CC-BY 3.0 licence.

Rapid deployment of artificial intelligence and machine learning to tackle coronavirus must still go through ethical checks and balances, or we risk harming already disadvantaged communities in the rush to defeat the disease.

This is according to researchers at the University of Cambridge’s Leverhulme Centre for the Future of Intelligence (CFI) in two articles published in the British Medical Journal, cautioning against blinkered use of AI for data-gathering and medical decision-making as we fight to regain normalcy in 2021.

“Relaxing ethical requirements in a crisis could have unintended harmful consequences that last well beyond the life of the pandemic,” said Dr Stephen Cave, Director of CFI and lead author of one of the articles.

“The sudden introduction of complex and opaque AI, automating judgments once made by humans and sucking in personal information, could undermine the health of disadvantaged groups as well as long-term public trust in technology.”

In a further paper, co-authored by CFI’s Dr Alexa Hagerty, researchers highlight potential consequences arising from the AI now making clinical choices at scale – predicting deterioration rates of patients who might need ventilation, for example – if it does so based on biased data.

Datasets used to “train” and refine machine-learning algorithms are inevitably skewed against groups that access health services less frequently, such as minority ethnic communities and those of “lower socioeconomic status”.

“COVID-19 has already had a disproportionate impact on vulnerable communities. We know these systems can discriminate, and any algorithmic bias in treating the disease could land a further brutal punch,” Hagerty said.

In December, protests ensued when Stanford Medical Centre’s algorithm prioritized home-workers for vaccination over those on the Covid wards. “Algorithms are now used at a local, national and global scale to define vaccine allocation. In many cases, AI plays a central role in determining who is best placed to survive the pandemic,” said Hagerty.

“In a health crisis of this magnitude, the stakes for fairness and equity are extremely high.”

Along with colleagues, Hagerty highlights the well-established “discrimination creep” found in AI that uses “natural language processing” technology to pick up symptom profiles from medical records – reflecting and exacerbating biases against minorities already in the case notes.

They point out that some hospitals already use these technologies to extract diagnostic information from a range of records, and some are now using this AI to identify symptoms of COVID-19 infection.

Similarly, the use of track-and-trace apps creates the potential for biased datasets. The researchers write that, in the UK, over 20% of those aged over 15 lack essential digital skills, and up to 10% of some population “sub-groups” don’t own smartphones.

“Whether originating from medical records or everyday technologies, biased datasets applied in a one-size-fits-all manner to tackle COVID-19 could prove harmful for those already disadvantaged,” said Hagerty.

In the BMJ articles, the researchers point to examples such as the fact that a lack of data on skin colour makes it almost impossible for AI models to produce accurate large-scale computation of blood-oxygen levels. Or how an algorithmic tool used by the US prison system to calibrate reoffending – and proven to be racially biased – has been repurposed to manage its COVID-19 infection risk.

The Leverhulme Centre for the Future of Intelligence recently launched a Master’s course for ethics in AI. For Cave and colleagues, machine learning in the Covid era should be viewed through the prism of biomedical ethics – in particular the “four pillars”.

The first is beneficence. “Use of AI is intended to save lives, but that should not be used as a blanket justification to set otherwise unwelcome precedents, such as widespread use of facial recognition software,” said Cave.

In India, biometric identity programs can be linked to vaccination distribution, raising concerns for data privacy and security. Other vaccine allocation algorithms, including some used by the COVAX alliance, are driven by privately owned AI, says Hagerty. “Proprietary algorithms make it hard to look into the ‘black box’, and see how they determine vaccine priorities.”

The second is ‘non-maleficence’, or avoiding needless harm. A system programmed solely to preserve life will not consider rates of ‘long covid’, for example. Thirdly, human autonomy must be part of the calculation. Professionals need to trust technologies, and designers should consider how systems affect human behaviour – from personal precautions to treatment decisions.

Finally, data-driven AI must be underpinned by ideals of social justice. “We need to involve diverse communities, and consult a range of experts, from engineers to frontline medical teams. We must be open about the values and trade-offs inherent in these systems,” said Cave.

“AI has the potential to help us solve global problems, and the pandemic is unquestionably a major one. But relying on powerful AI in this time of crisis brings ethical challenges that must be considered to secure public trust.”

AIhub focus issue on reduced inequalities

tags: ,


University of Cambridge




            AIhub is supported by:


Related posts :



The Good Robot Podcast: featuring Hayleigh Bosher on generative AI, creativity, and what AI means for the music industry

In this episode, Eleanor and Kerry talk to about Hayleigh Bosher about generative AI, creativity, and what AI means for the music industry
03 October 2023, by

Artificial Intelligence tools shed light on millions of proteins

Researchers have constructed an interactive network of 53 million proteins with AlphaFold structures.
02 October 2023, by

2023 AI Song Contest entries online

The 35 song entries to the this year's competition have been released.
29 September 2023, by

AIhub monthly digest: September 2023 – uncovering concepts in deep-learning models, pollinator policies, and all questions answered

Welcome to our monthly digest, where you can catch up with AI research, events and news from the month past.
28 September 2023, by

CLAIRE AQuAs to return in autumn with trending AI topics. Join in!

Find out about the forthcoming CLAIRE "all questions answered" sessions.
27 September 2023, by

Which strategies for supporting pollinators?

How can Bayesian Networks be used to inform pollinator abundance strategies?
26 September 2023, by





©2021 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association