The group’s work – which is being presented at the Association for Computing Machinery's User Interface Software and Technology Symposium in Berlin – centres on two approaches: one which makes innovative use of the microphones present in most devices and the other employing a modern-day version of eavesdropping technology used by the KGB in the 1950s.
The first of these, a sound-based activity recognition system called Ubicoustics, uses sound-effect libraries to train existing microphones to enable devices to recognise sounds associated with places, such as bedrooms, kitchens, workshops, entrances and offices.
"The main idea here is to leverage the professional sound-effect libraries typically used in the entertainment industry," explained Gierad Laput, a PhD student in the group. "They are clean, properly labelled, well-segmented and diverse. Plus, we can transform and project them into hundreds of different variations, creating volumes of data perfect for training deep-learning models."
The researchers began with an existing model for labelling sounds and tuned it using sound effects from the professional libraries, such as kitchen appliances, power tools, hair dryers, keyboards and other context-specific sounds. They then synthetically altered the sounds to create hundreds of variations.
Potential applications of the technology including alerting the user when someone knocks on the front door, for instance, or moves to the next step in a recipe when it detects an activity, such as running a blender or chopping.
In their tests, Ubicoustics had an accuracy of about 80 per cent, competitive with human accuracy, but not yet good enough to support user applications. Better microphones, higher sampling rates and different model architectures all might increase accuracy with further research.
In a separate paper, HCII PhD student Yang Zhang, along with Laput and Chris Harrison, assistant professor in CMU's Human-Computer Interaction Institute (HCII), describe what they call Vibrosight, which can detect vibrations in specific locations in a room using laser vibrometry. It is similar to the light-based devices the KGB once used to detect vibrations on reflective surfaces such as windows, allowing them to listen in on the conversations that generated the vibrations.
https://www.youtube.com/watch?v=51XaZDki6yg&feature=youtu.be
This method requires a special sensor, a low-power laser combined with a motorised, steerable mirror. Reflective tags - the same material used to make bikes and pedestrians more visible at night - are applied to the objects to be monitored. The sensor can be mounted in a corner of a room and can monitor vibrations for multiple objects.
Zhang said the sensor can detect whether a device is on or off with 98 per cent accuracy and identify the device with 92 per cent accuracy, based on the object's vibration profile. It can also detect movement, such as that of a chair when someone sits in it, and it knows when someone has blocked the sensor's view of a tag, such as when someone is using a sink or an eyewash station.
Babcock marks next stage in submarine dismantling project
Surely on a national security project all contractors ought to be UK owned? This is similar to the life enhancement of our nuclear stations which has...