AR visor could help slice surgery times

News

A pan-European project is underway to develop an augmented reality (AR) visor that could reduce surgery times by more than 10 per cent.

Known as VOSTARS (Video Optical See-Through Augmented Reality surgical System), the device will be able to superimpose X-ray data onto a patient’s body. The head-mounted display (HMD) will also provide information on the anaesthetic used, as well as heart rate, body temperature, blood pressure and respiration. Initial trials will focus on procedures around the head, including maxillofacial surgery and neuro surgery.

“With this...highly ergonomic visor, we intend to provide all the information required to improve surgery,” said project coordinator Dr Vincenzo Ferrari, a biomedical engineering researcher at the University of Pisa.

“The primary goal is to reduce not just surgery times, but also the time spent under anaesthetic and the cost involved in any operation. For the patient, this means saving off 20 minutes of every three hours of surgery and the guarantee of an extremely accurate intervention.”

VOSTARS uses a head-mounted camera to capture what the surgeon sees. The system then merges this footage with the patient’s medical images, from CT, MRI, or 3DUS scans. According to the researchers, the device is a combination of a video see-through (VST) system and an optical see-through (OST) head-mounted display.

VST is the technology at play in VR headsets such as the Oculus Rift and is useful for accurately aligning the real and virtual worlds. However, surgery required a clear view of the patient, which is where OST comes in. It operates similarly to Microsoft’s Hololens, where virtual elements are imposed onto the real world.

“For something as critical as an operation, we had to have the naturalistic ‘feel’ of OST, while having the fluid interaction of the VST,” said researcher Fabrizio Cutolo, an engineer and expert in AR systems. “Therefore a brand new device had to be made from scratch, rather than extend an existing technology. The VST-OST Augmented Reality hybrid was born.”

As well Italy, VOSTARS includes participants from Germany and France, with the UK also contributing via North Shields’ VREO Innovation. The three-year project has received almost €4m in funding via the Horizon 2020 scheme, and a working prototype is expected in May 2018.