Extra eye movements could improve self-driving cars

The study, recently published in PLOS Computational Biology, focuses on the unnoticed eye movements humans make, showing that they serve a vital purpose in allowing people to stably recognise objects. 

Through this work, Andrea Benucci and colleagues at the RIKEN CBS in Japan have developed a way to create artificial neural networks that can reportedly learn to recognise objects faster and more accurately.

Despite making constant head and eye movements throughout the day, objects in the world do not blur or become unrecognisable, even though the physical information hitting human retinas changes constantly. 

What likely makes this perceptual stability possible are neural copies of the movement commands. These copies are sent throughout the brain each time humans move, and are thought to allow the brain to account for people’s own movements and keep perception stable.

In addition to stable perception, evidence suggests that eye movements, and their motor copies, might also help people to stably recognise objects in the world, but how this happens remains a mystery. 

Benucci developed a convolutional neural network (CNN) that offers a solution to this problem. The CNN was designed to optimise the classification of objects in a visual scene while the eyes are moving.

According to researchers, the network was first trained to classify 60,000 black and white images into ten categories. Although it performed well on these images, when tested with shifted images that mimicked naturally altered visual input that would occur when the eyes move, performance dropped drastically to chance level. 

However, classification improved significantly after training the network with shifted images, as long as the direction and size of the eye movements that resulted in the shift were also included, researchers confirmed.

In particular, adding the eye movements and their motor copies to the network model allowed the system to better cope with visual noise in the images.

“This advancement will help avoid dangerous mistakes in machine vision,” said Benucci. “With more efficient and robust machine vision, it is less likely that pixel alterations — also known as ‘adversarial attacks’— will cause, for example, self-driving cars to label a stop sign as a light pole, or military drones to misclassify a hospital building as an enemy target.”

Benucci added that the benefits of mimicking eye movements and their efferent copies implies that ‘forcing’ a machine-vision sensor to have controlled types of movements, while informing the vision network in charge of processing the associated images about the self-generated movements, would make machine vision more robust and akin to what is experienced in human vision.

Next steps will involve collaboration with colleagues working with neuromorphic technologies. Researchers said the idea is to implement actual silicon-based circuits based on the principles highlighted in the study, and test whether they improve machine-vision capabilities in real world applications.