Source: shutterstock/Gorodenkoff
Article • VR, AR, XR and more
Promising applications of mixed realities in medicine
Extended reality applications like virtual reality (VR) and augmented reality (AR) are mostly known for their use in the gaming or movie industries. However, in recent years, clinicians have begun exploring potential medical applications for those immersive technologies. In a Coffee and Talk session at ECR 2019, researchers from the Netherlands, Sweden, and the UK talked about practical applications in medical education and training as well as preprocedural planning and visualization during a surgery.
Report: Sascha Keutel
In the Chairperson’s introduction, Peter van Ooijen, PhD, a computer scientist at the University Medical Center Groningen in the Netherlands, explained the fundamental differences between the various technologies. Virtual reality is a completely digital environment with no connection to the real world. It provides complete control over the user’s visual and auditory experience as they interact within a completely synthetic environment. Augmented reality allows the user to see the native environment while adding augmented data to it. The real world remains central to the experience but is enhanced by virtual data. In merged reality or mixed reality (MR) the real and the virtual world are intertwined, it allows for interaction with digital objects while preserving a sense of presence within the true physical environment.
The key question Ooijen asked, was why radiologists should actually bother about those applications when they already have their advanced imaging technologies and computer screens? To get a real benefit out of those technologies, one of the most important aspects is getting the proper imaging data, so the scan has to be right. “You have to do correct segmentation of the data to put it into that kind of environment. You have to build up the teaching cases. So, the role of radiology is really fundamental,” Ooijen concluded.
MR in medical education
In his presentation, Professor Dr Anders Persson from the Department of Radiological Sciences at Linköping University, Sweden, focused on the use of mixed realities in medical education and training. He explained how radiologists and 3D imaging specialists at the university’s Center for Medical Image Science and Visualization supply the imaging data they use for teaching their medical students and residents.
Persson picked up on Ooijen’s explanation that VR is a closed environment, arguing that it is quite tricky to use VR for real-time clinical exams of CT or MR images. “You need to move the data from the scanner, segment it, work with it and make a mold. So, it’s not the real thing. Making a diagnose with that kind of images doesn’t work – yet,” Persson said. “For education, mixed reality is really good,” he said, “because you can communicate with the teachers.” In addition, multiple students can move around freely and examine a 3D anatomical model at the same time, allowing them to take it apart layer by layer and thus better understand the body’s anatomy.
Persson also pointed out, that to use extended realities, one doesn’t have to use goggles. “We use a visualization table. That is a real thing, you can take the data directly from the scanner and can look at it in 3D,” he said.
At the end of his presentation, Persson pointed out that these technologies could also be used for the patients’ education. Advances in this technology allow them and their family members to better understand their conditions, helping them to make more informed decisions around their medical care.
MR in the OR
In another presentation, Dr Dimitri Amiras of Imperial College Healthcare National Health Service (NHS) Trust in the UK, talked about the use of MR in surgical application. It enables physicians to gather critical imaging information and to visualize complex medical data, particularly before and during medical procedures, Amiras pointed out.
Furthermore, MR technology lets clinicians see the patients’ internal anatomy displayed directly on the body. At St Mary’s Hospital in London, Amiras and colleagues applied augmented reality to facilitate plastic reconstructive surgery. The team segmented the imaging data of a patient scheduled to undergo leg surgery and then converted the data into 3D models compatible with AR software. Afterwards they highlighted the patient’s vascular anatomy to indicate where the surgical team could safely perform the procedure without perforating any veins or arteries.
To successfully prepare such models, “the most important part is the segmentation of the data,” Amiras said. “Because without the right information segmented, the surgeon could operate on the wrong vessel or he could misinterpret the images.”
In an outlook, Amiras prophesized that “mixed reality will be a game-changer for surgery. Using that technology will not only make things quicker and faster, we may start to make the inoperable operable,” he concluded.
13.03.2019
