Article • AI provides prognostic information

Next-generation deep learning models predict cancer survival

Deaths from cancer are currently estimated at 10 million each year worldwide. Conventional cancer staging systems aim to categorize patients into different groups with distinct outcomes.

Report: Bernard Banga

Image source: Adobe Stock/Siarhei (Graph); Medical University Lodz (H&E)

‘However, even within a specific stage, there is often substantial variation in patient outcomes,’ Markus Plass, academic researcher from the Medical University of Graz, Austria, explained to Healthcare in Europe. Hence the rapid growth of Artificial Intelligence (AI), machine learning and deep learning in providing novel prognostic information that is not captured in current staging guidelines.

portrait of Athena Davri
Athena Davri

Deep learning, a subdivision of machine learning, uses convolutional neural networks to devise informative representations of raw input data automatically, without requiring manual feature engineering. ‘This is particularly useful for image segmentation and classification in histology slides,’ said Athena Davri, biologist at the Department of Pathology in the Faculty of Medicine, School of Health Sciences, University of Ioannina in Greece. Currently, histopathology examination of tissue remains the “gold standard” for diagnosing colorectal cancer, the second most common cancer in women and the third most common in men. However, routine pathology lab tests are taking up a lot of time and effort, due to the high incidence of this type of cancer. Furthermore, the worldwide shortage of pathologists has led to delays in diagnosis. AI models initially made it possible to automate and speed up the work before integrating parameters from the tumour ecosystem. 

So much so in fact, that in 2022 alone, Davri's team has listed, in a systematic review published in August, around a hundred scientific articles devoted to deep learning on histopathology images for diagnosing colorectal cancer. According to this systematic review, ‘algorithms based on deep learning have the potential to assist with diagnosis, identify histological features relating to prognosis and associated with metastasis, and assess the specific components in the tumour microenvironment,’ said Davri. 

Contextual histopathology features from whole-slide images

Today, the revolution in prognosis is coming from the relationship between deep learning and whole slide imaging. Also known as virtual microscopy, whole slide imaging refers to scanning a complete microscope slide and creating a single high-resolution digital file. A typical whole slide image may contain 100,000×100,000 pixels. Analysing and viewing whole slide images is often constrained by computer memory or screen size. A common workaround for these issues is capturing many smaller, high-resolution image tiles or strips, which are then stitched together to create a full image of a single histological section. This works because whole slide scanners take separate images of each field of view across the entire slide at high speed. The images acquired separately are then stitched together during the scanning process to generate a single digital image at full resolution. 

Many experts agree that machine learning is the future for digital pathology. A Chinese team from the Sun Yat-sen University Cancer Center, Guangzhou, affiliated to the laboratory of Precision Medicine for Gastrointestinal Tumors at Nanfang Hospital, and US scientists from the Department of Radiation Oncology at Stanford University School of Medicine in California have designed a multitask deep learning platform for simultaneously predicting peritoneal recurrence and disease-free survival using preoperative CT images. ‘We trained it using a retrospective, multi-institution study on 2,320 subjects and evaluated the prognostic accuracy of the model as well as its association with chemotherapy response,’ said Prof Guoxin Li from the Department of General Surgery & Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor. 

Findings were published in The Lancet Digital Health in May 2022. When informed by the AI model, clinician performance was significantly enhanced for predicting peritoneal recurrence. Additionally, the AI was able to identify which patients with stage II and stage III gastric cancer were most likely to benefit from adjuvant chemotherapy.

Multimodal deep learning close to clinical routine

All forms of cancer stand to benefit from the prediction capabilities of machine learning models. One team, from Seoul National University in the Republic of Korea, is applying the machine learning approach to the analysis of whole-slide images of kidney, breast, lung and uterine cancers. In August, the researchers tested their deep learning graph on 3,950 patients with these four types of cancer. ‘Deep graph neural networks that derive contextual histopathology features from whole slide images may aid diagnostic and prognostic tasks,’ said Prof. Kyung Chul Moon from the Department of Pathology at Seoul National University College of Medicine. 

portrait of Peter Mildenberger
Prof Peter Mildenberger

Currently, no AI-based models are being used in clinical practice. However, multimodal approaches linking biology, histopathology imaging and gene expression are bringing experimental deep learning models closer to routine clinical application. In 2021, teams from the Department of Urology and Pediatric Urology, the Department of Diagnostic and Interventional Radiology, and the Institute of Pathology all at the University Medical Center in Mainz, Germany, did develop and evaluate a multimodal deep learning model for prognosis prediction in clear-cell renal cell carcinoma (ccRCC). This disease is the most common type of kidney cancer with more than 175,000 associated deaths each year. In contrast to other tumour types, there is no clearly defined set of biomarkers used in clinical routine. Clinical management of ccRCC usually involves various disciplines including urology, radiology, oncology, pathology, and more besides. ‘This results in a vast amount of medical data on each patient, such as CT/MRI scans, histopathology images and other clinical information,’ explained Prof Axel Haferkamp, Director of the Department of Urology and Pediatric Urology. 

There are several clinical tools for prognosis prediction in ccRCC, such as the UCLA Integrated Staging System (UISS) or the International Metastatic Renal Cell Carcinoma Database Consortium risk model. ‘But while prognostic clinical nomograms might be helpful, they can be cumbersome to use and often only incorporate a selection of the available information – both of which potentially limit their performance,’ said Peter Mildenberger, radiologist, senior consultant and associate professor at the Department of Diagnostic and Interventional Radiology in Mainz. Their multimodal deep learning model showed great performance in predicting the prognosis of clear-cell renal cell carcinoma patients, with a mean accuracy of 83.43%. Furthermore, this latest tool’s prediction was an independent prognostic factor which outperformed other clinical parameters. 

Recommended article

Recommended article


Malignant tumour management

’Our machine learning model achieved 88.9% accuracy in predicting the sarcoma-specific survival rate’

Clinical management of soft tissue sarcoma is particularly challenging. Dr Sebastian Foersch, researcher at the Institute of Pathology at the University Medical Center in Mainz, Germany, has used a deep learning model for diagnosis and prognosis prediction of soft tissue sarcoma using conventional histopathology slides.

Recommended article


Supervised learning approach

A new deep learning-based algorithm to predict relapse-free survival in papillary thyroid carcinoma

The tall cell variant (TCV) is an aggressive subtype of papillary thyroid carcinoma (PTC). Sebastian Stenman, researcher from the Institute for Molecular Medicine, and the Department of Pathology at the University of Helsinki, Finland, is developing and training a deep learning algorithm using supervised learning to detect and quantify the proportion of tall cells in PTC.


Read all latest stories

Related articles


News • WSI-based analysis

AI-driven classification of diffuse gliomas skips molecular testing

Research from Shenzhen proposes an integrated diagnosis model for automatic classification of adult-type diffuse gliomas directly from annotation-free standard whole-slide pathological images.


News • Deep learning refinement

Colorectal cancer: AI tool improves diagnostic accuracy

Researchers from Finland have developed an artificial intelligence tool for automatic colorectal cancer tissue analysis that outperforms prior methods.


Article • Breast cancer diagnostics

Precision pathology: an AI-led alternative to molecular diagnostics

Experts have highlighted how precision pathology using Artificial Intelligence can provide an effective alternative to molecular diagnostics. This, say a team from the Karolinska Institutet (KI) in…

Related products

Subscribe to Newsletter