Gesture technique points to defects

If a quality control inspector finds a defect in a car’s paintwork, for example, then a point of the finger sends the defect to the QS inspection system, store it and document it. The employee obtains visual feedback through a monitor that displays a 3D reconstruction of the bumper.

Researchers at the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe engineered the intelligent gesture control system on behalf of the BMW Group.

‘Previously, the inspector had to note all defects that were detected, leave his workstation, go to the PC terminal, operate multiple input screens and then label the position of the defect and the defect type. That approach is laborious, time-intensive and prone to error,’ said Alexander Schick, a scientist at IOSB.

The gesture control system, by contrast, is said to improve the inspector’s working conditions considerably, and deliver substantial time savings as the employee can remain at a workstation and interact directly with the test object.

‘If the bumper is fine, then he swipes over it from left to right. In the event of damage, he points to the location of the defect,’ Schick said in a statement.

This non-contact gesture-detection system is based on 3D data; the entire workstation must first be reconstructed in 3D, including the individual as well as the object with which he or she is working.

‘What does the inspector look like? Where is he situated? How does he move? What is he doing? Where is the object? – all of these data are required so that the pointing gesture can properly link to the bumper,’ said Schick.

In order to enable gesture control, the experts apply 3D-body tracking, which records the individual’s posture in real time. Car body parts are ‘tracked’ also

Hardware requirements include a standard PC and two Microsoft Kinect systems – consisting of camera and 3D sensors – in order to realize the reconstruction.

Schick and his team developed the corresponding algorithms, which fuse multiple 2D and 3D images together, specifically for this kind of application, and adapted them to the standards of the BMW Group.

Schick said, ‘The breeding ground for this technology is our Smart Control Room, where people can interact with the room quite naturally. They can use pointing gestures to operate remote displays – without any additional equipment.

‘The room recognizes what actions are taking place at that moment, and offers the appropriate information and tools. Since gesture detection does not depend on display screens, this means we can implement applications that use no monitors, like the gesture interaction here with real objects.

‘It makes no difference what kind of object we are dealing with. Instead of a bumper, we could also track a different part.’

The technology can be subsequently integrated into existing production systems at little expense. Scientists could incorporate their effective process into the BMW Group’s system through a specialized interface module.

The gesture detection system will be presented at the 2013 Hannover Messe, from 8 to 12 April.