Mobile phone tool tracks emotional behaviour

’Emotionsense’ uses speech-recognition software and sensors in standard smart phones to assess how people’s emotions are influenced by factors such as their surroundings, the time of day, or their relationships with others.

It was developed by a Cambridge University-led team of academics, including psychologists and computer scientists.

Early results suggest that the technology could provide psychologists with a much deeper insight into how our emotional peaks − such as periods of happiness, anger or stress − are related to where we are, what we are doing or who we are with.

’Everyone has a mobile phone, so potentially they are a perfect tool if you want to track the behaviour or emotional condition of large numbers of people,’ said Dr. Cecilia Mascolo from Cambridge University’s Computer Laboratory.

’What we are trying to produce is a completely non-intrusive means of achieving this that also respects privacy. In time, it could have an enormous impact on the way in which we study human behaviour and give psychologists a deeper insight into what it is that makes different types of people tick.’

Emotionsense uses the recording devices that already exist in many mobile phones to analyse audio samples of the user speaking. The samples are compared with an existing speech library (known as the Emotional Prosody Speech and Transcripts Library), which is widely used in emotion and speech-processing research. The library consists of actors reading a series of dates and numbers in tones representing 14 different emotional categories.

From here, the samples are grouped into five broader categories − ’happy’ emotions (such as elation, or interest), ’sadness’, ’fear’, ’anger’ (which includes related emotions such as disgust) and ’neutral’ emotions (such as boredom or passivity).

The data can then be compared with other information that is also picked up by the phone. Built-in GPS software enables researchers to cross-refer the audio samples with the user’s location, Bluetooth technology can be used to identify who they were with and the phone also records data about who they were talking to and at what time the conversation took place.

The software is also set up so that the analysis is carried out on the phone itself. This means that data does not need to be transmitted elsewhere and can be discarded post-analysis with ease to maintain user privacy.

The research team tested the effectiveness of the system on a group of 18 volunteers at the university earlier this year.

Each subject was given a modified Nokia 6210 Navigator phone for a period of 10 days. They were also asked to keep a diary in which they recorded their emotional state according to a standard set of questions already used by social and behavioural psychologists.

The results showed that, in about 70 per cent of cases, the emotional analysis offered by the phone system agreed with the results of the survey, suggesting that with further modification this type of mobile phone technology could be an accurate means of tracking the factors influencing people’s emotions.

The pilot study also threw up some interesting suggestions about how circumstances may affect our emotional condition. Location appeared to have a pronounced effect on the users’ state of mind. ’Happy’ emotions dominated the data when they were in residential locations (45 per cent of all emotions recorded), but in workplaces ’sad’ emotions became the norm (54 per cent).

The researchers also found that users exhibited more intense emotions in the evening than in the morning and that people tended to express their emotions far more in smaller groups than in larger crowds.