Metaverse in medicine: between potential and pioneering spirit

At the Medica Health IT Forum, Dr Anke Diehl briefly made her audience see double: the expert, standing on the speaker’s podium, suddenly also appeared in a virtual operating theatre on-screen. With a few inputs, she manoeuvred her digital alter ego within the 3D environment, operating devices, checking clinical values, and interacting with other persons in the OR. 

After all, interactivity is a key feature of the metaverse: unlike many other VR worlds, it is designed for user exchange and joint activities. It is by no means a coincidence that Meta, the company behind Facebook, WhatsApp and Instagram, has positioned itself to be the most prominent player and driving force for this technology. Social interaction lies at the very core of the metaverse DNA, with some even regarding it as the logical expansion of social media.

Team-up in the avatar hospital

Dr Anke Diehl

Photo: O. Hartmann

But does this virtual cooperation also work in a medical context? Currently, Dr Diehl and her team are getting to the bottom of this question. ‘For this, we have built an avatar hospital – with some locations resembling actual clinical facilities, and others that only exist in the metaverse, where you can showcase 3D projections, for example,’ the expert reports. ‘At first, many of the locations were inspired by the Covid-19 pandemic; we were looking for new ways to meet colleagues and discuss with them.’ To this end, 3D models were created to serve as avatars for the employees, which they can use to move freely within the digital environments. ‘For example, I can direct my avatar onto a virtual stage to deliver a presentation, looking at the digitized versions of my colleagues sitting in the audience. Exactly this kind of spatial experience is a vital part of the technology.’ 

The virtual spaces are mostly accessed via desktop PCs using mouse/keyboard controls. ‘The metaverse is often associated with VR headsets, but that would be too costly in our context. And it’s not really necessary, either,’ Diehl explains. Immersion – the sense of “being there” in the virtual realm – is instead achieved by using small animations for the avatars. ‘For example, when a user talks, the character’s mouth is moving and using gestures to convey the action.’ This type of interactivity could also benefit doctor-patient communication, the expert suggests.

Added value for students and paediatric patients

However, the clinical test metaverse is far more than a glorified 3D conference call, she stresses: ‘We see a lot of added value in medical education. For example, the teacher can assume the role of a patient avatar for practising anamnesis interviews. Due to the immersive nature of the metaverse, students quickly forget that they are interacting with their teacher, and the situation loses its test character – which, for many students can feel rather intimidating.’ 

During a recent AI event in Essen, hospital visitors were also able to gain insights into the medical metaverse. Meanwhile, several research projects established benefits of the technology, especially for paediatric patients – for example, to help children overcome their fears of the unfamiliar hospital environment and its instruments. One study provided the young patients with assistance from the “Pengunauts” app; before an MRI scan, the virtual characters prepared the children for the noisy machine and encouraged them to brave the examination.

Another VR game currently in trial phase addresses paediatric patients undergoing chemotherapy, placing them, along with their parents, in an underwater setting. ‘In the game, the child is wearing a deep-sea diving suit, exploring the surrounding fish and interacting with them,’ Diehl explains. ‘At a certain point, the doctor gives a signal, for example when a disinfectant is applied. However, in the game, it’s just a small leak in the suit, so the child is not scared of the sudden cold sensation.’ For the expert, this “gamification” approach is one of the technology’s biggest strengths – a notion that is backed by the study’s promising intermediary results.

Doctors in the metaverse? Only if the benefits are clear

Despite most metaverse apps being geared towards a younger audience of “digital natives”, Diehl does not believe that the technology has an inherent generation gap: ‘The most important factor is being open-minded towards new things, which has no age limit, in my experience. For many doctors, the pivotal element will be whether the technology works and bears medical benefits. If that is the case, they will adopt it.’

That said, the expert encourages any colleague to bring a certain willingness for tinkering, as many aspects are still noticeably rough around the edges: ‘We are currently trying out many applications – some work better than others, due to the technology not being sufficiently matured.’ One notable limitation, she points out, is that vital parameters – such as ECG curves, diagnostic imaging or blood levels – cannot be displayed in real time on the metaverse’s virtual monitors. While such a feature would greatly enhance educational settings with virtual patient models, it poses a significant challenge for both technology and data privacy requirements. ‘We are still a long way from building a viable IT infrastructure and the legal frameworks to support this,’ the expert concedes.

The future: digital complementation of established procedures

‘The metaverse is certainly not the future of healthcare,’ Diehl is convinced. ‘For us, patient care has the highest priority, with no one being left behind. So, whenever a person is unable or unwilling to use the technology to participate in a digital treatment, we must provide an analogue alternative. This will not change anytime soon.’ 

Still, in times of medical staff shortages, the technology has great potential, the expert says: ‘We are desperately looking for new ways to provide care and therapies – for example via telemedicine – and in this regard, the metaverse could prove to be a valuable addition to our clinical toolset.’ 

Profile: 

Dr Anke Diehl is Chief Transformation Officer and Head of the Executive Department for Digital Transformation at University Medicine Essen, Germany. In this role, she is in charge of realizing the institution’s “smart hospital” strategy. Since March 2021, she also leads the consortium “Smart Hospital.NRW” cluster of excellence for AI in medicine. As a doctor, she worked in radiology, neurology, neuroradiology and psychiatry before moving to work for the state of North Rhine-Westphalia in 2010, where she headed the department for care structure development until 2018. In addition to digital medicine, she is currently working on gender issues in AI and digitalisation and is the winner of the 2021 Medical Woman of the Year Award. She further represents health IT users on the national expert committee Interop Council and is a member of the German Medical Association's Digitalisation Committee.

04.06.2024

• digitisation (66)
• IT (882)
• virtual (75)