AR helps surgeons to see, feel and understand

University Professor Dr Rüdiger von Eisenhart-Rothe

Photo: Nicki Schäfer

University Professor Dr Rüdiger von Eisenhart-Rothe, Chair of Orthopaedics and Sports Orthopaedics at the Technical University Munich, explains the advantages of different AR applications. Interface between the engineers and the surgeons, it is Professor von Eisenhart-Rothe’s task to provide feedback from the physician’s perspective whenever new technology is about to be implemented in his OR so the engineers can tailor their solutions to the actual needs and requirements of the OR staff. 

Virtual reality (VR) has become an important element of the training for joint replacement surgery. Using head-mounted displays OR processes are simulated in an entirely computer-generated environment. Augmented reality in contrast merges reality and virtual reality: virtual content is shown in the surgeon’s field of view to supplement the visible information on the real-world OR table. Thus the physician’s eyes can return to the patient rather than having to focus on the screen. 

AR is useful for training purposes since a further dimension is added. “In VR there is usually no haptic feedback. The user doesn’t get the physical sensation for example when a saw or a biopsy needle hits a harder object,” Professor von Eisenhart-Rothe reports. In AR different devices can provide haptic feedback on the interaction with the patient thus, he says, “the additional reaction significantly improves the learning effect.” 

Interface between engineer and surgeons

Any AR solution requires a lot of fine-tuning of the hardware and the software before it can be used in the OR. The goggles, for example, need to be lightweight so they don’t become a nuisance during long interventions. Important questions need to be answered in extensive tests such as the kind of data that is important and the location of the data on the surgeon’s display. “Often enough the engineers don’t know what the surgeons really need in the OR – and the surgeons don’t know what’s technically feasible. Therefore it is crucial to have an interface between these two worlds,” Professor von Eisenhart-Rothe describes his task.

In orthopaedic surgery implants of different manufacturers are used and fixed with special tools. Here AR can help: detailed instructions can be shown in the head-mounted display and even a product specialist can be brought in who sees the current situation via a camera and can virtually show the steps or devices needed. But the entire OR team benefits as Professor von Eisenhart-Rothe points out: “The OR nurse will also receive this information and can prepare all necessary instruments,”

X-ray vision for smaller incisions

Superimposing image data from CT or MRI is still being tested and not yet used in the real-world OR. A major obstacle is the fact that current hardware does not offer the required computing power. “Obviously, it depends which kind of data needs to be displayed in the surgeon’s field of view,” says the orthopaedist. Showing vital signs or the tilt of an instrument is not a problem. If however a live CT scan has to be displayed that shows the exact site of a fracture – that’s an entirely different ballgame. Immense computing power is required since complex data has to be displayed in real-time with millimetre precision on the patient body. “This would give the surgeon x-ray vision. Such functionality is still in the experimental stage since precise matching is a major challenge,” he underlines. In addition to high-performance computers, specific markers are required. They are placed on the patient body to tell the camera where certain parts of the anatomy are located when the patient is moved or when the surgeon changes her viewing angle. 

Von Eisenhart-Rothe is sure that the effort will be worthwhile particularly for tumour surgery: “Patients with abdominal tumours often require large incisions for the surgeon to get a good overview of complex anatomy. If the exact site of the pelvis and the tumour were shown in the head-mounted display, much smaller incision could be made.” Combined with intelligent navigation systems ‘collateral damage’ during surgery could be further reduced.

Through the eyes of the surgeon

In remote observation, the events in the operating room can be followed from the surgeon's perspective. This improves the learning effect.

Image source: DePuy Synthes

Another very promising idea according to Professor von Eisenhart-Rothe is “remote sitting-in” which is already being applied in select cases: a surgeon performs an intervention in her field of expertise. The data goggles allow less experienced colleagues to join. “Particularly now – during the pandemic – this has many advantages”, the expert says. But even in non-Corona times this technology could make long and expensive travel obsolete. Moreover: “In minimally invasive” interventions the incisions are only a few centimetres. Even colleagues right at the OR table scarcely see anything. When the steps can be observed from the point of view of the surgeon that is much more instructive,” von Eisenhart-Rothe explains. And vice versa – an experienced surgeon can be called upon to follow the surgery and support the colleague when needed. 

Profile:

University Professor Dr Rüdiger von Eisenhart-Rothe is Chair for Orthopaedics at the Medical Faculty of the Technical University of Munich and Director of the Clinic and Polyclinic for Orthopedics and Sports Orthopedics at the Rechts der Isar Hospital of the University of Technology Munich. He is Vice President of the German Knee Society, Founding President of the German Hip Society and Member of the Board of the German Society for Joint Replacement (Deutsche Gesellschaft für Endoprothetik). Von Eisenhart-Rothe publishes the journal “Der Orthopäde” and has (co-)authored more than 300 scientific books and papers.

17.11.2021

• IT (882)
• medical technology (1474)
• orthopaedics (78)
• surgery (610)
• telemedicine (267)
• virtual (75)