New AI tool offers precision pathology for cancer and beyond

The tool, called iStar (Inferring Super-Resolution Tissue Architecture), was developed by researchers at the Perelman School of Medicine at the University of Pennsylvania.

The imaging technique provides detailed views of individual cells, as well as the full spectrum of how people’s genes operate, which researchers said could allow doctors and researchers to see cancer cells that might otherwise have been ‘virtually invisible’.

In addition, the tool can be used to determine whether safe margins were achieved through cancer surgeries and automatically provide annotation for microscopic images, paving the way for molecular disease diagnosis at that level.

In a statement, Mingyao Li, PhD, professor of Biostatistics and Digital Pathology and co-lead of the study, said iStar has the ability to automatically detect critical anti-tumour immune formations called ‘tertiary lymphoid structures,’ whose presence correlates with a patient’s likely survival and favourable response to immunotherapy.

Immunotherapy currently requires high precision in patient selection, and Li said that iStar could be a powerful aid for determining which patients would benefit most from the treatment.

Researchers adapted a machine learning tool, the Hierarchical Vision Transformer, and trained it on standard tissue images, beginning with analysis of individual tissue imaging and the ‘fine detail’, to then ‘grasp broader tissue patterns.’

The AI system within iStar applies the information from the Hierarchical Vision Transformer to predict gene activities, often at near-single-cell resolution.

“The power of iStar stems from its advanced techniques, which mirror, in reverse, how a pathologist would study a tissue sample,” Li said. “Just as a pathologist identifies broader regions and then zooms in on detailed cellular structures, iStar can capture the overarching tissue structures and also focus on the minutiae in a tissue image.”

To test the efficacy, researchers evaluated iStar on different types of cancer tissue, including breast, prostate, kidney, and colorectal cancers, mixed with healthy tissues. iStar was able to automatically detect tumour and cancer cells that were hard to identify just by eye.

According to the researchers, the tool is significantly faster than similar existing AI tools. For instance, when analysing the available breast cancer dataset, iStar finished its analysis in nine minutes. Researchers said that the best competitor AI tool took more than 32 hours to establish a similar analysis.

The tool could aid clinicians in their analysis, as well as be applied to a large number of samples, to aid large-scale biomedical studies, such as 3D and biobank samples prediction.

Researchers said they aim to develop the tool to help researchers gain better understandings of the microenvironments within tissues, which could provide more data for diagnostic and treatment purposes moving forward.

The research was funded by the National Institutes of Health, and published in Nature Biotechnology, which can be accessed here.