Article • A valuable tool for reconstruction
Augmented reality lets surgeons ‘see’ inside limbs
Researchers at Imperial College London (ICL) have shown how the Microsoft HoloLens headset can be used during reconstructive lower limb surgery. Surgeons at London’s St Mary’s Hospital are using the device, a self-contained computer headset that immerses the wearer in ‘mixed reality’, enabling them to interact with holograms visible through the visor. In effect, the limb’s interior is visible during the procedure.
Report: Mark Nicholls
Identifying critical blood vessels under the skin
Testing the technology, the ICL team used the system to overlay images of CT scans – including the position of bones and key blood vessels – on to each patient’s leg so the surgeon could better locate and reconnect key blood vessels during reconstructive surgery. ‘We are one of the first groups in the world to use the HoloLens successfully in the operating theatre,’ Dr Philip Pratt, a Research Fellow in the Department of Surgery and Cancer, confirmed. ‘Through this initial series of patient cases we have shown that the technology is practical, and that it can provide a benefit to the surgical team. With the HoloLens, you look at the leg and essentially see inside of it. You see the bones, the course of the blood vessels, and can identify exactly where the targets are located.’
When a patient sustains serious injury, with tissue damage or open wounds, they may require reconstructive surgery using fasciocutaneous flaps of tissue taken from elsewhere on the body, including the skin and blood vessels, to be used to cover the wound and enable it to close and heal properly. Successfully connecting the blood vessels of the new tissue to those at the wound is a vital step in the process. Presently, handheld scanners using ultrasound are used to identify blood vessels under the skin. ‘Augmented reality offers a new way to find these blood vessels under the skin accurately and quickly by overlaying scan images onto the patient during the operation,’ Pratt explained.
Virtual 3-D arrows guide the surgeon
During initial trials of the technology, five patients requiring reconstructive leg surgery underwent CT scans to map the structure of the limb, including the position of bones and blood vessels. Images from the scans were then segmented into bone, muscle, fatty tissue and blood vessels by Dr Dimitri Amiras, a consultant radiologist at Imperial College Healthcare NHS Trust (ICHNT), and loaded into intermediary software to create 3-D models of the leg. These models were then fed into specially designed software that renders the images for the HoloLens headset, which in turn overlays the model onto what the surgeon can see in the operating theatre.
The HoloLens is used to identify where the blood vessels are in 3-D space and uses virtual 3-D arrows to guide the surgeon. Clinical staff can then manipulate the augmented reality images to make fine adjustments to correctly line up the model with surgical landmarks on the patient’s limbs. Jon Simmons, plastic and reconstructive surgeon at Imperial College Healthcare NHS Trust, carried out the trial procedures. Already, the hospital’s surgical teams believe that the HoloLens approach is more reliable and less time-consuming than the ultrasound method of locating the blood vessels.
Improving the system for future use
In the future we hope to automate the process further
Philip Pratt
While the first cases have been carried out on legs, which have clearly visible surgical ‘landmarks’, using this technique for surgery on the abdomen may be more complicated. However, as the technique is refined it could be used in other areas of reconstructive surgery requiring tissue flaps, such as breast reconstruction following mastectomy. ‘In the future we hope to automate the process further,’ Pratt added. ‘We can use software to improve the alignment and will attach markers to the patient when they have the scan, with the same markers present during the operation to use as additional points of reference.’ With further trials planned in a larger set of patients, he acknowledged that further improvements are needed, but said that from the number of cases so far the technique could be a valuable tool during reconstructive surgery.
Profile:
Dr Philip Pratt is a Research Fellow in the Faculty of Medicine, Department of Surgery & Cancer at Imperial College London, UK. Initially working in the banking sector, he began to explore research opportunities at the Institute of Biomedical Engineering, Imperial College, which ultimately led to a career change. He was appointed Research Fellow at the Hamlyn Centre for Robotic Surgery, Imperial College. From within the Department of Surgery and Cancer, he now undertakes very active research in image-guided surgery, and has successfully translated new technology and software into clinical practice in the operating theatre.
27.02.2018
