US research brings high quality VR to smartphones

The advance from Purdue University in the US also provides new opportunities for enterprise, education, health care and entertainment applications.

The system, called Coterie, uses a novel way to manage the challenging task of rendering high-resolution virtual scenes to satisfy the stringent quality-of-experience (QoE) of VR. Those include high frame rate and low motion-to-photon latency, which is the delay between the movement of the user's head or game controller and the change of the VR device's display reflecting the user's movement. The new approach enables 4K-resolution VR on commodity mobile devices and allows up to 10 players to interact in the same VR application simultaneously.

"We have worked to create VR technology that someone can use on a typical smartphone with a Wi-Fi connection," said Professor Y. Charlie Hu, who led the Purdue team. "Our solution not only allows multiple players to participate in a VR game at the same time, but also provides a better and more cost-effective option for single-player use."

The technology is detailed in a paper published in ASPLOS 2020, an international conference for interdisciplinary systems research, intersecting computer architecture, hardware and emerging technologies, programming languages and compilers, operating systems and networking.

One reason for the heavy computational workload of high-resolution VR apps is the constant need to render updates to both the foreground interactions with the players and the background environment in the virtual world.

"The heavy load simply cannot be handled by even high-end smartphones alone," Hu said. VR apps using Coterie split up this heavy rendering task between the smartphone and an edge server over WiFi in a way that drastically reduces the load on the smartphone while allowing the sub frames rendered on both to be merged into the final frame within 16ms, satisfying the VR QoE.

Hu said this approach not only reduces the network requirement so multiple players can share the same WiFi, but also reduces the power draw and computation demand on each mobile device and provides a better user experience.

"Our technology opens the door for enterprise applications such as employee training, collaboration and operations, health care applications such as surgical training, as well as education and military applications," Hu said. "You could have multiple doctors and health care professionals interacting in a VR operating room."