Smart homes boost with VibroSense home appliance tracker

Developed by a team at Cornell University, VibroSense can track 17 types of appliances with lasers that capture subtle vibrations in walls, ceilings and floors, and a deep learning network that models the vibrometer's data to create different signatures for each appliance.

Smart speakers detect signs of cardiac arrest

According to Cornell, this could increase efficiency in typical households where people forget to take wet clothes out of washing machines, retrieve hot food from microwaves and turn off dripping taps.

"Recognising home activities can help computers better understand human behaviours and needs, with the hope of developing a better human-machine interface," said Cheng Zhang, assistant professor of information science and senior author of a paper detailing the research in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. The paper is due to be presented at the ACM International Joint Conference on Pervasive and Ubiquitous Computing, which will be held virtually between September 12-17.

"In order to have a smart home at this point, you'd need each device to be smart, which is not realistic; or you'd need to install separate sensors on each device or in each area," Zhang said in a statement. "Our system is the first that can monitor devices across different floors, in different rooms, using one single device."

In order to detect usage across an entire house, the researchers' task was to detect tiny vibrations using a laser Doppler vibrometer; and differentiate similar vibrations created by multiple devices by identifying the paths travelled by the vibrations from room to room.

The deep learning network was trained to distinguish different activities, partly by learning path signatures - the distinctive path vibrations followed through the house - as well as their distinct noises.

The device is said to have showed nearly 96 per cent accuracy in identifying 17 different activities across five houses - including dripping tap, an extractor fan, an electric kettle, a refrigerator, and a cooker hood - in five houses over two days, according to the paper. VibroSense could also distinguish five different stages of appliance usage with an average accuracy more than 97 per cent.

In single-story houses, the laser was pointed at an interior wall at the centre of the home. It was pointed at the ceiling in two-story homes.

The device is primarily useful in single-family houses, said Zhang, director of Cornell's SciFi Lab, because in buildings it could pick up activities in neighbouring apartments, presenting a potential privacy risk.

"It would definitely require collaboration between researchers, industry practitioners and government to make sure this was used for the right purposes," Zhang said.