New research out of Carnegie Mellon University's School of Computer Science seeks to teach robots to grasp the ungraspable by turning the challenge on its side.

Wenxuan Zhou, a Ph.D. student in the Robotics Institute working with Assistant Professor David Held, used artificial intelligence to teach a robot to pick up an item by pushing it against something and grasping it on its side. Her work enables a simple robot gripper to pick up hard-to-grasp objects and to manipulate them in ways that only more complex — and expensive — robotic hands could.

"We wanted to find a way to do more complex tasks with a simple hand," Zhou said. "This research could make a simple gripper useful in many more tasks. It could allow robots to work better in warehouses or the home."

In many cases, robots are trained to pick up an item by grasping the top of it, which poses a problem when working with large objects and items not in or shaped like boxes. Through Zhou's system, a robot learns to push the object against a wall or other vertical surface, rotate it and grasp its side.

To grasp the ungraspable, Zhou turned to extrinsic dexterity — manipulating an object in a hand using resources other than the hand itself. She first used reinforcement learning to train a neural network, which in turn taught the robot to use extrinsic dexterity to grasp objects in its surrounding environment. In this case, the robot manipulates the object in its gripper with the help of the wall or other vertical surface.

Previous research on extrinsic dexterity and robotics often designed systems for grasping objects based on assumptions about how a robot would make contact with an item. Sometimes these systems also required more complicated hands or robots. Zhou used AI to develop a system with fewer hardware restrictions that could work for many different objects.

Zhou tested the system with eight items, including boxes of different sizes and textures, a plastic bottle, a food container, and a purse. The robot trained under the system successfully grasped and lifted the object 78% of the time.

"We put the item in a bin with walls but didn't instruct the robot to use the wall to grasp the item," Zhou said. "The algorithm knew its task was to grasp the object on its side, and through trial and error in simulation, it determined using the wall was the best course of action."

Zhou presented her work, "Learning to Grasp the Ungraspable With Emergent Extrinsic Dexterity" at the Conference on Robot Learning this past December in Auckland, New Zealand.