Bright spot for depth perception

A new depth-perception cameras which works in a similar way to the widespread Microsoft Kinect video gaming controller but which operates in bright light or outdoors had been developed by researchers from Carnegie Mellon University (CMU) and the University of Toronto (UoT) .

The camera could be used in medicine, on shiny or mirrored structures and, because it consumes very small amounts of energy, on extraterrestrial exploration rovers.

Kinect cameras work by projecting a pattern onto their object and detecting the reflections; the processing equipment detects how long reflections take to return and how distorted they are, then use this information to determine how far away from the camera the reflections originated. This allows them to build up a 3D image of the object. But in bright illumination or outdoors, the reflections are ‘washed out’ by ambient light, reducing the accuracy of the system.

Srinivasa Narasimhan of CMU and Kyros Kutulakos of UoT, respectively robotics and computer science researchers, devised a system that combines a laser projector with a rolling-shutter camera — the type of camera commonly used in smartphones — which detects only the laser light as it illuminates the object, rather than the light generally reflected.

“We have a way of choosing the light rays we want to capture and only those rays,” said Narasimhan, “We don’t need new image-processing algorithms and we don’t need extra processing to eliminate the noise, because we don’t collect the noise. This is all done by the sensor.”

As the laser scans over the object, the relected points of light are brighter than the surroundings, but only briefly.

“Even though we’re not sending a huge amount of photons, at short time scales, we’re sending a lot more energy to that spot than the energy sent by the sun,” explained Kutulakos.

The trick is to be able to record only the light from that spot as it is illuminated, rather than try to pick out the spot from the entire bright scene. The prototype system synchronises the rolling shutter and the projector so that as the laser scans a particular plane, the camera only accepts light from that plane and ignores all other illumination.

Among the possible applications are medical imaging, where the camera could detect skin structures that would be obscured by light diffusing through the skin. In extraterrestrial environments, the cameras could be useful in craters; the lasers actively illuminate the scene, so they can be used in shadow, and because the ignore all light other than the laser reflections, they could be used in polar regions where the low angle of the sun could cause glare problems for conventional cameras.

The system’s low power consumption makes it particularly attractive for these applications, the researchers state. Equally they could be used to detect anomalies in shiny or mirrored surfaces where reflections would dazzle and confuse other systems. According to Narasimhan, they could also be useful in automotive applications, helping in-car systems detect the distance to the car in front and maintaining distances in autonomous convoys, for example.

The team is presenting its research at Siggraph2015, the International Conference on Computer Graphics and Interactive Techniques, in Los Angeles