Comment: Why wearable robots need machine vision

The World Health Organisation estimates that soon many of the 1 billion people who live with disabilities could go about their daily lives without impairment thanks to wearable robots.

From picking up a cup of coffee to climbing the stairs, robots could help those who have lost limbs do what was previously impossible to them. Wearable devices such as rigid exoskeletons, soft robotic gloves, and prostheses already exist, but they’re far from perfect.

These mechanical instruments, while useful for basic grasping and walking on level ground, offer limited ranges of motion that stymies people’s ability to handle the varied tasks of everyday life.

MORE FROM OPINION

This has resulted in rates of rejection for wearable robotics systems, such as prosthetic arms and upper limb devices,  as high as 44%.

Computer vision will be key to overcoming these challenges. As a human looks around a room, they pick up rich and interpretable data about the world around them. The shape of objects, the incline of the floor – each piece of information impacting how someone intends to interact with their environment.

The key to augmenting the existing capabilities of those with limb difference is by dynamically responding to this intention, and then shaping the behaviour of robotics to complete a task. To match the speed of the modern world, assistive technology needs to infer someone’s intent as it changes –  they need to be able to see.

The history of intent and the need for vision

When reaching out to grab something, the motion of an individual’s hand will change depending on whether they see apples or oranges in front of them.

Similarly, an assistive glove needs to know about the type of object the user intends to grasp to perform specific tasks, like adapting grasp type and finger span. For leg exoskeletons or prosthetics, robotic systems need to detect when a user plans to ascend stairs or traverse a slippery walkway so that joint torques can be adjusted to maximise assistance and stability.

Using cameras to simulate vision is one of the most promising ways to collect information about the environment and measure user intent, but not the only one. Electromyography (EMG) measures electrical signals in the body to infer how a muscle might move.  EMG signals from a user’s wrist might tell a prosthetic hand to close, but these biological signals can only be interpreted simply. EMG may tell a prosthetic to close around a mug, but it won’t be able to tell it how to avoid crushing a porcelain teacup – a failure to read the environment.

By combining wearable EMG sensors with reinforcement learning for task completion and cameras to pick up visual environments, engineers are starting to create a more complete picture of the surrounding environment, improving on the classification of EMG systems by up to four times. This will allow robots to complete more complex tasks and assist those for whom EMG doesn’t work, like people with spinal cord injuries.

However, this is still a new field and a camera sensor with a machine learning algorithm could pose a large computational load for an embedded device, a problem if processing power must match the speed of human intention. Frame-based sensors like RGB cameras face a bandwidth-latency trade-off: higher framerates reduce perceptual latency but increase bandwidth demands, while lower framerates save bandwidth at the cost of missing vital data from the scene. Neuromorphic computing, which mimics the activity of the brain to increase processing in low-power devices, could help address this issue.

Adaptability, acceptability, and privacy

Intention is vital not only to help users complete more complex actions, but to also augment the ability of users with different levels of handicap.

Unlike passive prosthetics, robots and humans are both actors. Robots must read the intent of human users or risk moving against their wishes, resulting in injury. Similarly, an individual with their arm amputated above the elbow may use a robotic hand differently to someone with an amputation at the wrist.

Collecting visual data on how users with different disabilities use devices will be an important first step in training an adaptable controller that can modulate its behaviour depending on the needs of a user within different environments and their preference for use. By designing systems that augment rather than replace function, wearable robotics can build trust.

Privacy remains a concern, especially when dealing with user-generated health data to train AI models. However, vision-based home rehabilitation systems that comply with regulations like GDPR and the Health Insurance Portability and Accountability Act (1996) can offer a precedent for private use.

For too long, mechanical engineering has dominated assistive technology, leaving algorithmic control to lag behind. Manufacturers who have relied on these solutions in the past must embrace the computer vision research happening in places like King’s College London to create devices that are built for individual needs, rather than in a vacuum. Only then can engineers create solutions which allow communities to embrace the complexity of life.

Dr Letizia Gionfrida is a  lecturer in Computer Vision at King's College London