Robot understands when it has gained a human's attention

Using a socially expressive robot named Simon, from assistant professor Andrea Thomaz’s Socially Intelligent Machines lab, researchers wanted to see if they could tell when he had successfully attracted the attention of a human who was engaged in a task and when he had not.

‘The primary focus was trying to give Simon the ability to understand when a human being seems to be reacting appropriately, or in some sense is interested in getting a response from Simon, and to be able to do it using a visual medium, a camera,’ said Aaron Bobick, professor and chair of the School of Interactive Computing in Georgia Tech’s College of Computing.

‘Simon would make some form of a gesture, or some form of an action when the user was present, and the computer vision task was to try to determine whether or not you had captured the attention of the human being,’ said Bobick.

With close to 80 per cent accuracy Simon was able to tell, using only the cameras as a guide, whether someone was paying attention or not.

‘We would like to bring robots into the human world. That means they have to engage with human beings, and human beings have an expectation of being engaged in a way similar to the way other human beings would engage with them,’ said Bobick. ‘Other human beings understand turn-taking. They understand that if I make some indication, they’ll turn and face someone when they want to engage with them and they won’t when they don’t want to engage with them.

‘In order for these robots to work with us effectively, they have to obey the same kinds of social conventions, which means they have to perceive the same thing humans perceive in determining how to abide by those conventions,’ he added.

Researchers plan to go further with their investigations into how Simon can read communication cues by studying whether he can tell by a person’s gaze whether they are paying attention, or using elements of language or other actions.

‘Previously people would have pre-defined notions of what the user should do in a particular context and they would look for those,’ said Bobick. ‘That only works when the person behaves exactly as expected. Our approach, which I think is the most novel element, is to use the user’s current behaviour as the baseline and observe what changes.’