Video • Digital and real world combine
World premiere for mixed reality surgery
The Assistance Publique Hôpitaux de Paris (AP-HP) has organized, in partnership with TeraRecon, Vizua, Microsoft and Digital Evolutis, the live broadcast of the first surgery performed in the world with a collaborative platform of mixed reality at the Avicenne Hospital AP-HP, and interacting with remote doctors.
Dr. Gregory Thomas, Head of Orthopedic Surgery and Traumatology Hospital Avicenne AP-HP (specializing in surgery of the hand and upper limb) made a shoulder operation, assisted the HoloPortal TeraRecon platform and Microsoft HoloLens holographic computer. During the procedure, Dr. Gregory could see, holograms, 3D modeling of anatomical pictures of his patient, projected in real time in the operating room with HoloLens. He also had access to patient medical data (planned or not the patient himself) and interactive tutorials during the operation.
Dr. Gregory also exchanged during surgery via Skype with Dr. Emery and Dr. Reilly (Imperial College London), Professor Yan Lae Moon (Chosun University Hospital, South Korea), Dr. John Sledge (hospital Lafayette, Louisiana, USA) and Dr. April Armstrong (Pennsylvania, USA) that could interact during surgery by proxy holograms. This is the first time ever that such a comprehensive use of immersive and collaborative aspects of this mixed reality device is done in an operating room while maintaining optimum quality of holographic images. The completion of this transaction and positions the Avicenne Hospital AP-HP as a pilot site for the use of mixed reality Microsoft common practice in surgical and medical specialties in technical procedures.
An interactive and innovative surgery
The mixed reality makes living holographic experiences while remaining link to the outside world. HoloLens is a standalone computer that requires no cable. Worn like a helmet by the surgeon during the operation, it allows him to capture information from the environment in which it is thanks to its built-in microphones and sensors and to share with others, block or remotely via Skype. The surgeon keeps total freedom of movement and action in an enriched environment, and holograms information on request. Connected to HoloLens the HoloPortal platform that has been specifically developed by TeraRecon and Vizua and utilizes the computing power of cloud Azure from Microsoft, allows the surgeon to view real-time medical information from the patient could be useful to him during operation: his medical records, his biological assessments digitized 2D imaging (radiological native cuts). The surgeon can also directly access the scanners and MRI (without converting templates / formats).
The contributions of mixed reality in the patient's care
Since the pre-operative consultation to post-operative assessment, mixed reality improves the visualization of patient medical images. It can be used at different stages of the patient's support for:
- Inform patients in consultation on its pathology and intervention program. The patient is left and actor of his own care course and can take ownership of its operation;
- Exchange in multidisciplinary discussions on specific pathological cases or difficult assignments. An immersive meeting allows medical and nursing teams to visualize a patient's virtual twin in a meeting room;
- Access patient medical information before and during the operation;
- Discuss with the patient and his colleagues the surgery has been done. The mixed reality can indeed help train surgeons and the acquisition of new techniques and skills.
With mixed reality, the surgeon has additional support in his medical practice. By facilitating interaction with information that may be helpful to him, it also helps to reduce the response time for better care for patients.
Source: Assistance Publique Hôpitaux de Paris
12.12.2017
