How to use any smartphone as a ruler

Engineers are increasingly using the video capabilities of smartphones to create 3D images. Researchers from Carnegie-Mellon University in Pittsburgh have now found a way to use a smartphone component to provide accurate measurements of these models.

All smartphones today are equipped with devices called IMUs (inertial measurement units) which switch the display between portrait and landscape formats depending on how the phone is being held. Researcher Simon Lucey at Carnegie-Mellon’s Robotics Institute has found that IMUs can also calibrate 3D models as they are created and enable the phone to obtain accurate measurements from them.

Working on a system to help people obtain the right size glasses by using their phone camera to take pictures of their faces, Lucey and his team, working with colleagues at the University of Queensland, found that IMUs can be used to track the camera’s motion as the user moves the phone around to capture various facial angles.

Other teams have tried to use IMUs for this sort of application, but they used more expensive systems than the standard cheap IMUs found in most phones, sometimes using navigation or additional motion sensors. Cheap IMUs are not very accurate, but Lucey says that this doesn’t matter when their data is combined with accurate images from the more advanced cameras now used as standard. ‘The trajectory we create with these cheap IMUs will “drift” over time, but the vision element we create is very accurate,” he said in a statement. ‘So we can use the 3D model to correct for the errors caused by the IMU, even as we use the IMU to estimate the dimensions of the model.’

The system used a technique called ‘structure from motion’, pioneered by a colleague of Lucey’s from the Robotics Institute in the 1980s. This creates  accurate 3D models from a series of 2D images captured by a moving camera. Lucey’s innovation adds accurate scale to these models. ‘The amazing thing is that we can turn any smartphone into a ruler — no special hardware, no depth sensors, just your regular smart device,’ he said. Better cameras with a higher frame rate will improve accuracy even further; ‘we can excite the IMU by moving the phone faster, without corrupting the images,’ Lucey explained.