Augmented reality for remote surgery mentoring

The system, described in journal The Visual Computer, enables medical expertise to be shared using vocal instructions and digital annotations, which appear on a clear tablet between the operating surgeon and the patient. The mentoring surgeon can see the patient via the camera on the tablet, and can draw lines indicating where incisions should be made.

STAR uses algorithms to keep the annotations aligned with the changing images of the surgery, and the transparent tablet screen allows the medic to view their hands, the patient and the overlaid instructions all in the same frame.

“The surgeon sees the operating field, the instruments, and their hands as if the display were not there, yet the operating field is enhanced with the mentor’s graphical annotations,” said Voicu S. Popescu, a Purdue associate professor of computer science.

While telementoring has been around for some time, existing methods require the acting surgeons to alternate their attention between the patient and a screen. STAR removes the need for surgeons to shift their focus away from the operating table. According to the researchers, the technology could be used in a number of scenarios.

“Two primary applications are surgeries in the battlefield and in rural regions where specialists might not be available,” said Gerardo Gomez, a professor of surgery and director of the Division of Trauma, Surgical Critical Care and Emergency Surgical Services at the Indiana University School of Medicine.

“Optimal surgery and trauma treatment integrates different surgical skills frequently unavailable in rural and field hospitals, and telementoring can provide the missing expertise.”

So far, STAR has only been trialled on animals and surgical mannequins, but early data indicates an improvement on traditional telementoring techniques. Moving forward, the focus will be on refining the anchoring of the annotations in the changing field of the tablet, allowing greater surgical precision.

The researchers say that the system uses only off-the-shelf consumer electronics, and has even been trialled with Google Glass. In the future, they believe it could be combined with surgical robots, allowing specialists to actually perform procedures remotely.