Point and fetch

A team of researchers led by Charlie Kemp, director of the Center for Healthcare Robotics in the Health Systems Institute at the Georgia Institute of Technology and Emory University, have found a way to instruct a robot to find and deliver an item it may have never seen before using a laser pointer.

El-E (pronounced Ellie), a robot designed to help users with limited mobility with everyday tasks, autonomously moves to an item selected with a green laser pointer, picks up the item and then delivers it to the user, another person or a selected location such as a table.

The robot can grasp and deliver several types of household items including towels, pill bottles and telephones from floors or tables.

The robot uses a custom-built camera that is omni-directional to see most of the room. After the robot detects that a selection has been made with the laser pointer, the robot moves two cameras to look at the laser spot and triangulate its position in three-dimensional space. Next, the robot estimates where the item is in relation to its body and travels to the location. If the location is above the floor, the robot finds the edge of the surface on which the object is sitting, such as the edge of a table.

Picking up the unknown object is a significant challenge. To do so, the robot uses a laser range finder to initially locate the object. Then, after moving its hand above the object, it uses a camera in its hand to visually distinguish the object from the texture of the floor or table. After refining the hand’s position and orientation, it descends upon the object while using sensors in its hand to decide when to stop moving down and start closing its gripper. Finally, it closes its gripper upon the object until it has a secure grip.

Once the robot has picked up the item, the laser pointer can be used to guide the robot to another location to deposit the item or direct the robot to take the item to a person. El-E distinguishes between these two situations by looking for a face near the selected location.

If the robot detects a face, it carefully moves toward the person and presents the item to the user so it can be taken. It uses the location of the face and legs to determine where it will present the object.

If no face is detected near the location illuminated by the laser pointer, the robot decides whether the location is on a table or the floor. If it is on a table, El-E places the object on the table. If the location is on the floor El-E moves to the selected location on the floor.

After delivering the item, the robot returns to the user’s side, ready to handle the next request.

The Georgia Tech and Emory research team is now working to help El-E expand its capabilities to include switching lights on and off when the user selects a light switch and opening and closing doors when the user selects a door knob.