From left: Dr. Robert J. Gillies, Dr. Jhanelle Gray, Dr. Matthew Schabath...
From left: Dr. Robert J. Gillies, Dr. Jhanelle Gray, Dr. Matthew Schabath (standing) and Dr. Wei Mu (seated) review findings

Article • Clinical decision support

AI deep learning of PET/CT images to support NSCLC treatment

A software tool to predict the most effective therapy for non-small cell lung cancer (NSCLC) developed by applying deep learning artificial intelligence (AI) to positron emission tomography/computed tomography (PET/CT) images has been developed by researchers at H. Lee Moffitt Cancer Center and Research Institute in Tampa, Florida. The tool is designed to provide a noninvasive, accurate method to analyze a patient’s tumor mutations and biomarkers and to determine the most effective course of treatment, making modifications as needed.

Report: Cynthia E. Keen

Over two million cases of lung cancer are diagnosed annually, according to the World Health Organization (WHO), approximately 85% of which are NSCLC. Two therapeutic strategies -- epidermal growth factor receptor (EGFR)-targeted tyrosine kinase inhibitors (TKIs) and program-death ligand (PDL-1) targeted immune checkpoint inhibitors (ICIs) are extending survival for patients with advanced NSCLC. Matching patients to the most effective therapy currently requires specimen-based assays. In addition to tissue-based biopsies being an invasive procedure, malignant tumors can be heterogeneous and the biopsied tissue used for testing may not be reflective of the entire tumor. Additionally, tumors can mutate during a course of therapy, rendering the initial treatment ineffective.

While liquid biopsy from blood collections may represent an alternative approach for tissue-based molecular testing, not all patients’ tumor markers can be detected in the blood

Wei Mu

For these reasons, there is a critical need to be able to accurately and noninvasively assess tumor EGFR mutational status to guide the use of TKIs, or PDL-1 expression status to guide the use of ICIs. Notably, patients who respond well to TKIs generally do not respond well to ICIs, and vice-versa. Thus, distinctions between the two are needed. Imaging the uptake of the radiotracer 18F-Fluorodeoxyglucose (18F-FDG) during a PET/CT exam can identify sites of abnormal glucose metabolism and help accurately to characterize tumors. Importantly, FDG PET/CT scans are commonly used for patient staging.

18F-FDG PET/CT is widely used in determining the staging of patients with NSCLC, and this glucose radiotracer used is also known to be affected by EGFR activation and inflammation,” said Matthew Schabath, Ph.D., of Moffitt’s Cancer Epidemiology Department. “EGFR is a common mutation found in NSCLC patients. EGFR mutation status can be a predictor for treatment, as patients with an active EGFR mutation have better response to TKI treatment.”

“Prior studies have utilized CT-based radiomics as a noninvasive approach to predict EGFR mutation,” added Wei Mu, Ph.D., a postdoctoral fellow in the Cancer Physiology Department. “While liquid biopsy from blood collections may represent an alternative approach for tissue-based molecular testing, not all patients’ tumor markers can be detected in the blood. This creates the need that PET/CT-based radiomics may represent an opportunity,” commented Jhanelle Gray, M.D.,chair and program leader of the Thoracic Oncology Department.

The graphical user interface of the AI based non-invasive treatment decision...
The graphical user interface of the AI based non-invasive treatment decision support software. When using this software, the PET/CT images need to be imported into this software by clicking “import data” button firstly. Then all the series could be listed in the Series section. After choosing the right series, the needed PET and CT images could be shown. To have a better view of these images, the contrast could be adjusted manually or automatically. If needed, manual registration could be performed by clicking “start” and “finish” buttons in the Registration section. Next, by clicking “ROI selection” button and choose the proper size of the box, the region of interest to be analyzed need to be contoured with a red square manually. By clicking “calculate” button, the PD-L1 positive probability and EGFR mutant probability of this slice could be obtained and provided. After repeating this ROI selection and calculation process for all the slices that include the tumor and clicking “Finish” button, the patient level PD-L1 positive probability and EGFR mutant probability could be obtained and provided in the EGFR prediction and PD-L1 prediction section. At the same time, the suggested treatment type will be pointed in the treatment suggestion section.

We [...] want to develop an internationally useful tool regardless of ethnicity, so we combined the data from China and the United States for our analysis

Robert J. Gillies

Led by Robert J. Gillies, Ph.D., chair of Cancer Physiology and vice-chair of Radiology, the researchers used data from 681 NSCLC patients treated at the Shanghai Pulmonary Hospital, the Fourth Hospital of Hebei Medical University in Hebei, China, the Fourth Hospital of Harbin Medical University in Harbin, China, and the Moffitt Cancer Center to develop and then validate the model. Dr. Gillies told Healthcare in Europe that data from China was generously provided to the Moffitt researchers as part of an international collaboration. “Recently, the incidence of lung cancer has increased rapidly in China. The mortality rate of lung cancer is also increasing there, while it has decreased gradually in the United States,” he explained. “Thus, there is renewed effort in China to routinely obtain genetic alterations assays in NSCLC patients, allowing us to enroll more valid data from Chinese institutions. Additionally, we also want to develop an internationally useful tool regardless of ethnicity, so we combined the data from China and the United States for our analysis.”

The 18F-FDG PET/CT-based deep learning model, described in Nature Communications, demonstrated high accuracy in EGFR mutation status prediction across all patient cohorts from the different cancer treatment centers. A high EGFR deep learning score was significantly and positively associated with longer progression-free survival in patients treated with EGFR-TKIs, and negative associated with response to ICI. Patients were further analyzed with a similar deep learning model to predict PD-L1 status, and those who scored high in this test had considerably longer progression free survival following ICI treatment than those who received EGFR-TKI treatment. The combination of these two scores can accurately identify the right treatment strategy for individual patients.

“Compared to other studies, our analysis yielded very high accuracy to predict EGFR mutation status and had many advantages. These included training, validating and testing the deep learning score with multiple cohorts from four institutions, which increased its generalizability,” commented Dr. Wu.

The researchers plan to share the model with other institutions for research studies as well as conducting additional studies themselves. “The tool serves as an innovation that can help to match and personalize treatments. We are working closely with Moffitt’s image analysis team to move prospective initiatives forward that will continue to build on these models,” said Dr. Gray. “We hope that our model may serve as a predictive biomarker to identify NSCLC patients sensitive to EGFR-TKI treatment and to identify patients most likely to benefit from ICI treatment. Due to the advantage of routine acquisition and lack of sampling bias of 18F-FDG PET/CT images, we prudently propose this model as a future clinical decision support tool for different treatments pending in larger and prospective trials,” the researchers concluded.


Read all latest stories

Related articles


News • Deep learning CAD in sonography

Computer-aided diagnosis improves breast ultrasound expertise

A new study suggests that deep learning–based computer-aided diagnosis (CAD) for breast lesion classification on ultrasound can significantly improve radiologists' diagnostic performance.


News • AI milestone

ChatGPT-4 passes radiology board exam

The latest version of ChatGPT passed a radiology board-style exam, highlighting the potential of large language models but also revealing limitations that hinder reliability, new research shows.


News • Deep learning text-to-image

Can AI image generator DALL-E 2 advance radiology?

A new paper describes how generative AI text-to-image models such as DALL-E 2 could represent a promising future tool for image generation, augmentation, and manipulation in health care.

Related products


Artificial Intelligence

Fujifilm · REiLI

Advanced intelligent Clear-IQ Engine for MR

Artificial Intelligence

Canon · Advanced intelligent Clear-IQ Engine for MR

Canon Medical Systems Europe B.V.
AI-Rad Companion

Artificial Intelligence

Siemens Healthineers · AI-Rad Companion

Siemens Healthcare GmbH
contextflow Advance Chest CT *

Artificial Intelligence

Dedalus · contextflow Advance Chest CT *

Dedalus HealthCare GmbH
CT Image Reconstruction

Artificial Intelligence

Canon · CT Image Reconstruction

Canon Medical Systems Europe B.V.
CXDI Elite series

DR Detectors

Canon · CXDI Elite series

Canon Medical Components Europe B.V.
Subscribe to Newsletter