Major study shines spotlight on AI healthcare bias

Published in The Lancet Digital Health and NEJM AI, the ‘STANDING Together (STANdards for data Diversity, INclusivity and Generalisability)’ study involved more than 350 experts from 58 countries. STANDING Together is led by researchers at University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham.

The new recommendations aim to ensure that datasets used to train and test medical AI systems represent the full diversity of the people that the technology will be used for. This is because AI systems often work less well for people who aren’t properly represented in datasets. People who are in minority groups are particularly likely to be under-represented in datasets, so may be disproportionately affected by AI bias. 

“Data is like a mirror, providing a reflection of reality,” said chief investigator Dr Xiao Liu, Associate Professor of AI and Digital Health Technologies at the University of Birmingham. “When distorted, data can magnify societal biases. But trying to fix the data to fix the problem is like wiping the mirror to remove a stain on your shirt.

“To create lasting change in health equity, we must focus on fixing the source, not just the reflection.”

Recommendations set out in the report include:

• Encouraging medical AI to be developed using appropriate healthcare datasets that properly represent everyone in society, including minoritised and underserved groups
• Helping anyone who publishes healthcare datasets to identify any biases or limitations in the data
• Enabling those developing medical AI technologies to assess whether a dataset is suitable for their purposes
• Defining how AI technologies should be tested to identify if they are biased, and so work less well in certain people

“It is crucial that we have transparent and representative datasets to support the responsible and fair development and use of AI,” said Dominic Cushnan, deputy director for AI at NHS England.

“The STANDING Together recommendations are highly timely as we leverage the exciting potential of AI tools and NHS AI Lab fully supports the adoption of their practice to mitigate AI bias.’’