Finding solutions for radiology tasks

Putting AI algorithms under the microscope

With a growing number of Artificial Intelligence (AI) algorithms to support medical imaging analysis, finding the best solution for specific radiology tasks is not always straightforward. Many are cloaked in secrecy and commercial sensitivity and while an algorithm may perform well in one area, evidence of performance and adaptability in another environment is not always available.

Report: Mark Nicholls

Image source: Unsplash/Walkator

One approach to tackle these uncertainties and deliver a fair comparison of algorithms on the same data are AI challenges – or competitions – which act as a type of testbed for AI algorithms and scrutinize their adaptability across different environments and scenarios.

portrait of bram van ginneken
Bram van Ginneken is Professor of Medical Image Analysis at Radboud University Medical Center

Professor Bram van Ginneken - who has organised many of these challenge events - will highlight this approach in his ECR 2021 presentation “Benefits of AI challenges to clinical practice.” In it, he will discuss AI challenges and their organisation, offer an understanding of what happens with the data collected, and point out algorithms which won AI challenges and made it to the clinic. In addition, the work Professor van Ginneken and his associates are doing in this field is also enabling the development of other algorithms, often in very specialist areas that would not normally attract the development interest of major vendors.

Speaking ahead of the ECR virtual session, he said the challenge events create an opportunity to compare algorithms for particular tasks. With some 150 products for AI in radiology with CE certification for use in Europe, commercial sensitivity among vendors means the content of some algorithms remains unclear, said Professor van Ginneken, who is Professor of Medical Image Analysis at Radboud University Medical Center in The Netherlands. Van Ginneken’s group created a platform – grand-challenge.org – where challenges and the algorithms are hosted. Some are so effective that they have made it to market. He emphasised that the competitions are primarily aimed at the research community, and make datasets available for researchers who want to work on medical data.

Recommended article

A key aspect lies in comparing the algorithms on the same test data with like-with-like comparisons and checking how an algorithm that has been tested on a dataset for one scenario such as in a hospital, for example, works in different populations. While the quality control aspect of Professor van Ginneken’s work is important, he also underlines the value of making high-quality training datasets available that can lead to other algorithms being developed.

Some of the challenges involve working with vendors, though not all are comfortable with participating because of the risk of an algorithm not performing well when tested in different contexts. But algorithms that do come out of these competitions can be licensed to companies and support product development. Examples Professor van Ginneken will discuss include an algorithm developed for the CO-RADS scoring system in the first wave of the coronavirus pandemic in May, which under the rapid certification scenario surrounding the pandemic, was on the market by June. Another example is the outcome from a 2017 competition to predict the likelihood of patients developing lung cancer within a year from a CT scan of the lungs. A liver segmentation algorithm will also be highlighted.

While AI development looks like a big breakthrough, there are still lots of areas where there is no software solution

Bram van Ginneken

The challenges mean researchers can compare different approaches on the same data set, and they also address the broader issue of scalability for AI in radiology. Professor van Ginneken says despite the range of AI products on the market, AI only covers a small element of radiology work and developing products for rare cases and disease has limited commercial appeal to major companies. “There are rare tasks for which we also need solutions and I think these challenges/competitions can provide that,” he said. “This is where there will be a role for algorithms that are developed in the research community.”

Sharing data remains an issue and can be hampered by privacy considerations and getting permission to use datasets for research. “While AI development looks like a big breakthrough, there are still lots of areas where there is no software solution,” he said, “so these challenges are one element that can really help to broaden the scope, the repertoire of tools that radiologists can use.”

The session will also look at presentations exploring the value of AI in chest x-ray prioritisation, and discuss the French DRIM AI study.

*Special Focus Session - SF 20 - Clinical impact of artificial intelligence (AI): status report, Thursday, March 4, 2021, 10:30-11:30am.


Profile:

Bram van Ginneken is Professor of Medical Image Analysis at Radboud University Medical Center and chairs the Diagnostic Image Analysis Group. He is also a founder of Thirona, a company that develops software and provides services for medical image analysis. A member of the Editorial Board of Medical Image Analysis, he pioneered the concept of challenges in medical image analysis.

05.03.2021

Read all latest stories

Related articles

Photo

Incidental findings identification

AI system for brain MRIs could boost workflows

An artificial intelligence (AI)-driven system that automatically combs through brain MRIs for abnormalities could speed care to those who need it most, according to a new study. “There are an…

Photo

Radiology collaboration

Improved workflow and a touch of Disney magic

Improving workflow is one of the major challenges that radiology departments face. The need to be more efficient, deliver timely and effective patient care, and keep an eye on costs are all factors…

Photo

AI-assisted MRI segmentation

Deep learning boost for prostate cancer workflow

Prostate cancer radiotherapy treatments guided by MRI are increasingly being explored to help improve patient outcomes and reduce toxicities after treatment. However, this development is being held…

Related products

Siemens Healthineers – AI-Rad Companion

Artificial Intelligence

Siemens Healthineers – AI-Rad Companion

Siemens Healthineers
Fujifilm - FDR EX-M1 AI box

Aritificial Intelligence

Fujifilm - FDR EX-M1 AI box

FUJIFILM EUROPE GmbH
Fujifilm · REiLI

Artificial Intelligence

Fujifilm · REiLI

FUJIFILM EUROPE GmbH
Hitachi - Diagnostic Image Support Solutions

Artificial Intelligence

Hitachi - Diagnostic Image Support Solutions

Hitachi Medical Systems Europe (Holding) AG
medavis - cockpit4med Radiology Dashboard

Business Intelligence

medavis - cockpit4med Radiology Dashboard

medavis GmbH
Siemens Healthineers - teamplay Usage

Business Intelligence

Siemens Healthineers - teamplay Usage

Siemens Healthineers
Subscribe to Newsletter