Virtual assistants and digital twins advance personalised medicine

The virtual medical assistant and digital patient twin are two key aspects of a research project ‘Models for Personalised Medicine’. Scientists at the Innovation Centre for Computer Assisted Surgery (ICCAS) in the Medical Faculty at Leipzig University, aim to use these tools to improve the treatment of cancer patients. The project has received funding of around €5.1 million from the Federal Ministry of Education and Research and is being implemented with the help of companies in Saxony.

Professor Thomas Neumuth is the technical director of the Innovation Centre Computer Assisted Surgery (ICCAS) in Leipzig, Germany

Different technologies have been designed with various pilot applications: ‘The objective is to support the medical treatment of cancer patients with the help of IT,’ explains Professor Thomas Neumuth, who heads the project at ICCAS, and is also the centre’s Deputy Director. Targets include, for instance, patients with head/neck tumours.

IT support starts with the tumour board, i.e. interdisciplinary discussion between surgeons, radiologists, radiotherapists or pathologists. ‘Experts from different medical disciplines discuss the medical condition of the respective patient,’ Neumuth explains. ‘It’s a type of briefing where all information relevant to the decision-making process is evaluated. The experts discuss treatment options based on this information and make a joint decision.’

In the future, the virtual assistant is to be present at these discussions as well. ‘This means we won't lose any information,’ he points out. However, among programming challenges will be the different volumes of sound and different positions of those who are talking in the room, which must be detected and accounted for. ‘Unclear pronunciation or strong accents should also not impact on word recognition.’ In contrast to conventional software, language recognition used in the context of medicine must be programmed for the specialist terminology and must also adhere to the strictest guidelines on confidentiality.

Enter: the digital twin

The Leipzig-based scientists have another vision: the so-called digital twin. This is an organised collection of all information about a patient and their anamnesis: radiological images, information on underlying medical conditions and previous surgery as well as molecular-genetic data. ‘This is much more complex than the electronic patient file which we already have,’ Neumuth explains.

The data in the patient file has not yet been linked in a meaningful way, so comprehensive, patient-specific analyses supported by AI is not yet possible. However, the digital twin now ‘paves the way for the step from the analogue into the digital world,’ he believes. This also includes storing treatment steps, playing through options, and updating information, explains the project manager. This objectivises medical work, makes information accessible to all experts in the team in equal measure and facilitates improved prediction of the effectiveness of treatment.

During diagnosis and therapy, the information of the data twin is compared to digital models of the clinical picture, which are optimised with the relevant studies and latest scientific findings. The computer should then support doctors with personalised treatment recommendations for cancer patients. ‘The final decision on treatment will obviously continue to be made jointly by patients and doctors,’ Neumuth points out.

It is also envisaged that a patient-data explorer will link patient data from radiological images and medical reports via web technology, and that it will integrate molecular-genetic tumour information into the decision-making process, or calculate patient-specific therapy profiles for surgery and radio or chemotherapy. ‘Different types of information contained in the digital twin should be directly linked and analysed by AI,’ Neumuth adds.

Many developments and tests are still needed to ensure that this and other technologies in the field of personalised cancer medicine can be directly integrated into clinical routine. ‘This will take three to five years,’ he estimates. The objective is to create a scientific and methodological basis for personalised cancer treatment assisted by AI. In the ideal case, this means that a patient will receive personalised treatment based on the latest scientific findings, which takes into account the patient’s personal situation and specific needs, with everything being transparent and explained in an understandable way.

Profile:

Professor Thomas Neumuth is an engineer and IT specialist. His research focuses on model-based medicine, intelligent biomedical technology, and medical IT systems. Born in Leipzig, he is the technical director of the Innovation Centre Computer Assisted Surgery (ICCAS) and heads the ICCAS Research Department for Model-based Medicine. His work on intelligent medical IT systems was chosen as a reference project for the Digital Summit 2017 by the Federal Ministry of Education and Research and confirms the position of the ICCAS as one of the leading research centres in Germany.

23.09.2020

• AI (675)
• data management (573)
• IT (883)
• oncology (405)
• personalised medicine (242)
• speech processing (34)
• tumour board (9)