Moving pictures

A project combining 3D imagery with hand-tracking and speech recognition technology could change the way in which the next generation of computers presents information — as well as how we interact with it.

One of the key aims of the EU-funded Interact project is to develop a system that provides interaction with 3D images without the need for any additional hardware such as gloves or virtual reality goggles. According to Tomas Rodriguez of Spanish software firm Eptron, which is leading the project, it will bring together the very latest developments in the field.

'We think there's a real need to develop a completely new form of human-computer interaction,' said Rodriguez.

The idea is that a user will be able to manipulate a 3D projection in real-time. The project is working on two different methods for interacting with the images.

The first and more groundbreaking of the two is a form of gesture recognition technology that tracks natural hand movements and uses the information to manipulate images. Rodriguez said one of the key challenges is the development of a hand-tracker that requires no additional hardware, the first time this has been attempted in a project like Interact.

'There are already technologies available which use spots or markers for gesture recognition,' he said. 'However, the real problem is developing extremely precise hand-tracking without any other hardware which can also operate under common office lighting conditions. Developing a sensor within these constraints is a completely different story.'

The second form of interaction uses speech recognition to enable the user to turn, flip or resize 3D objects with simple verbal commands. The user will also be able to call up contextual information about the product from the system using speech.

The core part of the project will be the 3D visualisation subsystem itself. Hungarian 3D image specialist Holografika will develop this, basing it on its existing groundbreaking HoloVizio technology (The Engineer 6 June, 2006).

Tamás Forgács, Holografika's chief tech- nology officer, said that the finished product will be a new high-end category in 3D visualisation. There are already a number of systems for creating 3D images, some of which — stereo and auto-stereo, for example — require spectacles to work, as they provide only a left and a right image. Others, such as multi-view holograms, are only visible from a certain angle.

Forgács said that the HoloVizio system, which wowed visitors at the Siggraph 2006 graphics show, is based on 'quasi-holographic' technology. While the image produced by HoloVizio is not strictly a true hologram, which can only be made at extremely high cost in certain research laboratories, it is close to a perfect 3D image.

Crucially for the project, users can view the image from different angles with the naked eye and see different perspectives of the object simply by moving around it.

'By combining these technologies, it should be possible to use it just like children shaping something from modelling clay — it will be very intuitive,' said Forgács. 'We also want 3D visualisation for multiple viewers from different angles, all in a natural environment.'

While the initial market for the finished system is likely to be design and prototyping, Rodriguez believes that technology will emerge from Interact that will have applications in many different industries.

'Because we will develop the three technologies independently of each other before we integrate them, it means the target market is not just the complete system but also the individual tools,' he said.

The £2.5m project is set to last for three years and demonstration prototypes of the system should be available by the end of the second year.