Jason Ford

Microlens array captures four-dimensional data on camera

News

A new imaging technique can adjust the depth of field and angle of view of video footage post-capture without the need for complex in-camera compression and processing.

The technology will therefore be ideal for capturing footage quickly in difficult or confined environments and, as a result, could even find a use in medical image processing and machine vision.

Researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering have been working for a number of years with lightfield or plenoptic cameras.

These cameras use a microlens array, mounted in front of the camera sensor but behind the main lens, to capture four-dimensional data on the ‘lightfield’ — which describes the position, intensity and two-way direction of a beam of light at the point it hits the sensor.

‘The microlenses basically produce their own image, which has a slightly different angle of view of the scene that’s projected through the main lens — the lightfield contains much more information than can be captured with a normal camera,’ project lead Dr Arne Nowak told The Engineer. 

In cinematic post-production, a creative producer can then decide the depth of focus and angle of view of a sequence, as well as creating so-called depth maps, which are useful when integrating CGI effects.

Nevertheless, as Nowak pointed out: ‘One of the main problems of this technology is that to achieve good-quality depth maps you need an enormous amount of samples of the scene.’

This is where he says the project’s technology has an advantage. Although plenoptic cameras have been developed in the past, they tend to require in-camera compression and processing.

By contrast, Fraunhofer’s non-regular sampling method uses a special image sensor with fewer pixels that only scans the most necessary image information to record a scene. The post-production computer then uses the scanning and signal frequency of an image to subsequently recalculate it into high-resolution quality.

The upshot is that, by shifting processing to post-production, the camera uses less power and memory consumption — potentially making the technology accessible to smaller handheld devices.