Drones and AI help Harvard team eavesdrop on sperm whales

Described in Science Robotics, the new technology was developed to support Project CETI (Cetacean Translation Initiative), a major international programme aiming to capture millions of sperm whale vocalisations and ultimately understand how whales communicate. To do that, the CETI team must first identify spots where the tagged whales are likely to surface, so that the drones can maximise the number of times they rendezvous with the whales.

The CETI drones are equipped with very high frequency (VHF) signal sensing that uses signal phase and the drone’s motion to emulate an ‘antenna array in air’. Combining this with acoustic data from underwater sensors and pre-existing sperm whale motion models, the Harvard team has developed the AVATARS (Autonomous Vehicles for whAle Tracking And Rendezvous by remote Sensing) framework. AVATARS is a decision-making algorithm designed to increase the number of drone-whale interactions, ultimately helping Project CETI reach its goal quicker.  

“I’m excited to contribute to this breakthrough for Project CETI,” said research lead Stephanie Gil, Assistant Professor of Computer Science at the Harvard John A Paulson School of Engineering and Applied Sciences (SEAS).

“By leveraging autonomous systems and advanced sensor integration, we’re able to solve key challenges in tracking and studying whales in their natural habitats. This is not only a technological advancement, but also a critical step in helping us understand the complex communications and behaviours of these creatures.”

According to the Harvard team, the same principles of time rendezvous used to develop AVATARS are also used by rideshare companies to assign drivers to passengers more efficiently. Rideshare apps such as Uber use real-time sensing to note the dynamic paths and positions of drivers and potential riders. Project CETI’s case is similar, real-time tracking the whale with the goal of coordinating the drone’s rendezvous to meet the whale at the surface. 

“'This research was an amazing opportunity to test our systems and algorithms in a challenging marine environment,” said first author Ninad Jadhav, a Harvard University PhD candidate.

“This interdisciplinary work, that combines wireless sensing, artificial intelligence and marine biology, is a prime example of how robotics can be part of the solution for further deciphering the social behaviour of sperm whales.”