Acoustical spying

Computer scientists at the

have taken several 10-minute sound recordings of users typing at computer keyboards, fed the audio into a computer, and used an algorithm to recover up to 96% of the characters entered.

"It's a form of acoustical spying that should raise red flags among computer security and privacy experts," said Doug Tygar, UC Berkeley professor of computer science and information management.

"If we were able to figure this out, it's likely that people with less honourable intentions can - or have - as well."

What makes the technique feasible is that each keystroke makes a relatively distinct sound, however subtle, when hit. Typical users type about 300 characters per minute, leaving enough time for a computer to isolate the sounds of individual keystrokes and categorize the letters based upon the statistical characteristics of English text. For example, the letters "th" will occur together more frequently than "tj," and the word "yet" is far more common than "yrg."

"Using statistical learning theory, the computer can categorise the sounds of each key as it's struck and develop a good first guess with an accuracy of 60% for characters, and 20% for words," said Li Zhuang, a UC Berkeley PhD student in computer science.

"We then use spelling and grammar checks to refine the results, which increased the character accuracy to 70% and the word accuracy to 50%. The text is somewhat readable at this point."

But that's not all. The recording is then played back repeatedly in a feedback loop to "train" the computer to increase its accuracy until no significant improvement is seen. In the UC Berkeley experiments, three feedback cycles were often enough to obtain recovery rates of 88% for words and 96% for characters.

Once the system is trained, recovering the text became more straightforward, even if the text was a password and not an English word. After just 20 attempts, the researchers were able to retrieve 90% of five-character passwords, 77% of eight-character passwords and 69% of 10-character passwords.

There are limitations to the technique, however. The researchers pointed out that they did not use the Shift, Control, Backspace or Caps Lock keys for their experiments, but say that there are approaches for training a program to account for those keystrokes as well. The ability to account for use of a computer mouse will be more challenging, the researchers said.

Nevertheless, the findings highlight a security hole that could be exploited and should be investigated, the researchers said.

The new research builds upon prior work by IBM researchers Dmitri Asonov and Rakesh Agrawal in which 80% of text was recovered from keyboard recordings. One key difference is that the experiments by Asonov and Agrawal relied upon controlled conditions in which the same typist is using the same keyboard and the algorithm is trained with known text and corresponding sound samples.

In contrast, the computer algorithm in the UC Berkeley study can "learn" and adapt to different typing patterns. To show this, the researchers experimented with multiple users on different keyboards, including so-called "quiet" keyboards, and found that their algorithm was successfully able to predict data. Moreover, recordings were taken in a variety of conditions, such as environments in which music was playing or cell phones were ringing in the background.