2 PAT H O L O G Y / L A B O R AT O R Y AI provides prognostic information Next-generation deep learning models predict cancer survival m o c . e b o d a . k c o t s – ) E & H ( z d o L y t i s r e v i n U l a c i d e M ; ) h p a r G ( i e h r a i S © Image source: Jiang et al., Lancet Digital Health 2022 (CC BY 4.0) Deaths from cancer are current- ly estimated at 10 million each year worldwide. Conventional cancer staging systems aim to categorize patients into different groups with distinct outcomes. ‘However, even within a specific stage, there is often substantial variation in patient outcomes,’ Markus Plass, academic research- er from the Medical University of Graz, Austria, explained to European Hospital. Hence the rapid growth of Artificial Intelligence (AI), machine learn- ing and deep learning in provid- ing novel prognostic information that is not captured in current staging guidelines. Deep learning, a subdivision of machine learning, uses convolu- tional neural networks to devise informative representations of raw input data automatically, without requiring manual feature engineer- ing. ‘This is particularly useful for image segmentation and classifica- tion in histology slides,’ said Athena Davri, biologist at the Department of Pathology in the Faculty of Medicine, School of Health Sciences, University of Ioannina in Greece. Currently, histopatho- logy examination of tissue remains the ‘gold standard’ for diagnosing colorectal cancer, the second most common cancer in women and the third most common in men. However, routine pathology lab tests are taking up a lot of time and effort, due to the high incidence of this type of cancer. Furthermore, the worldwide shortage of patholo- gists has led to delays in diagnosis. AI models initially made it possible to automate and speed up the work before integrating parameters from the tumour ecosystem. So much so in fact, that in 2022 alone, Davri's team has listed, in a systematic review published in August, around a hundred scien- tific articles devoted to deep learning on histopathology ima- ges for diagnosing colorectal can- cer. According to this systematic review, ‘algorithms based on deep learning have the potential to assist with diagnosis, identify histolog- ical features relating to progno- sis and associated with metastasis, and assess the specific components in the tumour microenvironment,’ said Davri. Contextual histopathology features from whole-slide images Today, the revolution in progno- sis is coming from the relation- ship between deep learning and whole slide imaging. Also known as virtual microscopy, whole slide imaging refers to scanning a com- plete microscope slide and creat- ing a single high-resolution digital file. A typical whole slide image may contain 100,000×100,000 pix- els. Analysing and viewing whole slide images is often constrained by computer memory or screen size. A common workaround for these issues is capturing many smaller, high-resolution image tiles or strips, which are then stitched together to create a full image of a single histological section. This works because whole slide scan- ners take separate images of each field of view across the entire slide at high speed. The images acquired separately are then stitched togeth- er during the scanning process to generate a single digital image at full resolution. Many experts agree that machine learning is the future for digital pathology. A Chinese team from the Sun Yat-sen University Cancer Center, Guangzhou, affiliated to the laboratory of Precision Medicine for Gastrointestinal Tumors at Nanfang Hospital, and US scientists from the Department of Radiation Oncology at Stanford University School of Medicine in California, have designed a multitask deep learning platform for simultaneous- ly predicting peritoneal recurrence and disease-free survival using pre- operative CT images. ‘We trained it using a retrospective, multi-in- stitution study on 2,320 subjects and evaluated the prognostic accu- racy of the model as well as its association with chemotherapy response,’ said Prof Guoxin Li from the Department of General Surgery & Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor. Findings were published in The Lancet Digital Health in May 2022. When informed by the AI model, clinician performance was signif- Pediatric Urology, the Department of Diagnostic and Interventional Radiology, and the Institute of Pathology all at the University Medical Center in Mainz, Germany, did develop and evaluate a mul- timodal deep learning model for prognosis prediction in clear-cell renal cell carcinoma (ccRCC). This disease is the most common type of kidney cancer with more than 175,000 associated deaths each year. In contrast to other tumour types, there is no clearly defined set of biomarkers used in clini- cal routine. Clinical management of ccRCC usually involves various disciplines including urology, radi- ology, oncology, pathology, and more besides. ‘This results in a vast amount of medical data on each patient, such as CT/MRI scans, histopathology images and other clinical information,’ explained Prof Axel Haferkamp, Director of the Department of Urology and Pediatric Urology. There are several clinical tools for prognosis prediction in ccRCC, such as the UCLA Integrated Staging System (UISS) or the International Metastatic Renal Cell Carcinoma Database Consortium risk model. ‘But while prognostic clinical nomo- grams might be helpful, they can be cumbersome to use and often only incorporate a selection of the available information – both of which potentially limit their per- formance,’ said Peter Mildenberger, radiologist, consultant and associate professor at the Department of Diagnostic and Interventional Radiology in Mainz. Their multimodal deep learning model showed great performance in predicting the prognosis of clear- cell renal cell carcinoma patients, with a mean accuracy of 83.43%. Furthermore, this latest tool’s pre- diction was an independent prog- nostic factor which outperformed other clinical parameters. senior Report: Bernard Banga icantly enhanced for predicting peritoneal recurrence. Additionally, the AI was able to identify which patients with stage II and stage III gastric cancer were most likely to benefit from adjuvant chemo- therapy. Multimodal deep learning close to clinical routine All forms of cancer stand to ben- efit from the prediction capabili- ties of machine learning models. One team, from Seoul National University in the Republic of Korea, is applying the machine learning approach to the analysis of whole- slide images of kidney, breast, lung and uterine cancers. In August, the researchers tested their deep learn- ing graph on 3,950 patients with these four types of cancer. ‘Deep graph neural networks that derive contextual histopathology features from whole slide images may aid diagnostic and prognostic tasks,’ said Prof. Kyung Chul Moon from the Department of Pathology at Seoul National University College of Medicine. Currently, no AI-based models are being used in clinical practice. However, multimodal approach- es linking biology, histopathology imaging and gene expression are bringing experimental deep learn- ing models closer to routine clini- cal application. In 2021, teams from the Department of Urology and Deep learning-based algorithm for Papillary thyroid carcinoma to According the World Health Organization, the tall cell variant (TCV) is an aggressive subtype of pap- illary thyroid carcinoma (PTC) featur- ing at least 30% epithelial cells two to three times longer than they are wide. In practice, applying this distinction is difficult, leading to substantial varia- tions between observers. ‘That’s why we are developing and training a deep learning algorithm using supervised learning to detect and quantify the proportion of tall cells (TCs) in pap- illary thyroid carcinoma,’ explained Sebastian Stenman, researcher from the Institute for Molecular Medicine, and the Department of Pathology at the University of Helsinki, Finland. In summer 2022, his research team tested it on an independent data set, and further validating it on an independent set of 90 papillary thy- roid carcinoma samples from patients treated at the Hospital District of Helsinki and Uusimaa (HUS) between 2003 and 2013. The Finnish scientist compared the algorithm-based tall cell ratio to independent scoring by a human investigator and looked at how those scores were associated with disease outcomes. The results published revealed that the deep learning algorithm detected tall cells with a sensitivity of 93.7% and a specificity of 94.5%. In the validation set, the deep learning algorithm tall cell scores correlated with a diminished relapse-free survival. ‘We showed that the DL-based algorithm was better than the human observer in identifying tall cell variants,’ said Stenmann. The algorithm could prove useful as a clini- cal tool for pathologists when evaluat- ing PTC samples and could potentially significantly improve the consistency of TCV case assessment. So far, no such algorithm has been described. EUROPEAN HOSPITAL Vol 31 Issue 4/22