Hands-on approach

A research team has unveiled gesture control technology claimed to be cheap and simple enough to be integrated into a wide range of everyday consumer electronics.

team has developed pattern recognition software that helps computers understand human gestures. Project manager Valiantsin Hardzeyeu said the software, GestureID, can be easily integrated into any type of video camera and installed inside such devices as PCs, TVs and satnav devices.

'Our work is based on optical pattern recognition,' he said. 'This technique mimics the way humans see things.'

His team modelled vision processes from the point where the photons hit the retina to the stage at which they are processed in the visual cortex.

The system works with a standard webcam that records the surrounding environment. The GestureID software detects if there are any human hand or hands in the frame. If there are, the system filters out all unnecessary information, extracting only the human hand/hands from the initial image.

The system then performs the recognition process, which is based on the gesture's shape description.

'At the moment we use five different types of hand signal to classify a certain gesture,' said Hardzeyeu. 'However, it is possible to extend this number to 12 easily.'

At the recognition process the system performs an action, which is defined for each gesture class. With a PC running Windows XP, for example, the system can perform up to 50 gesture detections/sec. 'This means that even really fast pointing gestures could be easily detected by the system,' he said.

The Fraunhofer team recently demonstrated how its technology could be used to control a video on a computer using an application such as Windows Media Player. Their computer, using a standard webcam, tracked the movement of a user's finger.

When it pointed up, the video began streaming. When it slid to the right, it started to fast-forward. Similarly, when a finger slid to the left it rewound. The user can stop the video by pointing down.

Other technologies, such as Microsoft Surface, rely on more advanced versions of pattern recognition software. Microsoft's coffee table surface computer allows multiple users to simultaneously manipulate digital content by natural motions, hand gestures or physical objects. Five cameras, incorporated in the machine's housing, record movements on the table's surface.

The Fraunhofer technology can only recognise simple human gestures, but it relies on only one standard camera so the entire system would be simple and cost-efficient to install in many everyday electronic devices.

One example, said Hardzeyeu, is when you wake up in the morning you could gesture 'up' and your coffee machine will prepare your drink.

It could also be used to simplify the way household electronics are operated. 'You wouldn't need a remote controllers for the VCR player, DVD player or TV set,' said Hardzeyeu. 'With gestures it would be possible to control all these devices.'

Another possibility would be to integrate GestureID in satnav devices. Hardzeyeu suggested the systems could be equipped with both gesture and voice recognition. 'You'll no longer have to fiddle with buttons,' he said. 'It will bring more functionality to the cars themselves.'

The team is currently working on designing a system that could easily be installed inside TV sets and other electronic equipment. The system would integrate GestureID with a field-programmable gate array or digital signal processor. Hardzeyeu said the most challenging part of the project was developing the software, and the rest should fall into place quite easily. He expects to have a product ready for commercialisation within a year.

'We think it's possible to integrate such a technology in every kind of electronic device such as TV sets and stereos,' he said.