A four-legged robot like the Unitree A1 — with its imprecise movements and the use of a single, jittery depth camera — wouldn't seem like a great candidate for conquering obstacles of parkour. But that didn't stop researchers in Carnegie Mellon University's School of Computer Science.

Using a single neural network, the researchers enabled the sometimes clumsy robot to perform some pretty hardcore parkour stunts, from handstands to running across tilted surfaces while leaping gaps twice its length and jumping over obstacles twice its height.

"Parkour really pushes the limits of the hardware," said Deepak Pathak, an assistant professor in CMU's Robotics Institute.

Parkour, considered a sport by some and a physical discipline akin to a martial art by others, challenges people to move over any terrain and through natural and constructed obstacles by running, jumping, climbing, crawling and more. Also called freerunning, parkour requires strength, balance, awareness, coordination, agility and precise vision.

Just as humans can develop the skills and eye-muscle coordination needed to vault gaps or run across tricky terrain with the right training, the commercially available Unitree robot can adapt its locomotion to overcome obstacles despite its underlying limitations, Pathak said. Like a human, it relies on its own perception to navigate and to adapt to the conditions it encounters.

This isn't the first time a robot has mimicked practitioners of parkour. Other research groups hand-coded the movements needed for the robots to complete a parkour course and used detailed maps based on precise measurements of the obstacles to guide the robots.

The CMU researchers wanted to take that even further.

"People don't need a map to walk, so we likewise wanted our robot to depend on its own perception to complete the course," said Xuxin Cheng, who shared first authorship on the project with fellow student Kexin Shi.

For the Unitree robot, that meant relying on a single front-facing depth camera, an instrument that measures distances. The researchers used a single neural network — a type of artificial intelligence inspired by the human brain — to direct the robot's movements based solely on the camera images.

The team initially used simulations to train the neural net via reinforcement learning, a machine learning technique that rewards desired behaviors and punishes unwanted ones, Shi said. This training required intensive computing, but once the basic behavior was learned, subsequent computation was minimal and could run efficiently on the robot's onboard computer.

"The robot has to compute its decisions efficiently otherwise parkour is impossible," Shi said.

While running an actual course, the robot rapidly adapted to inconsistent perception and slow or imprecise leg movements. It modified its gait if a leg slipped. If it failed to clear an obstacle, it quickly recovered. It learned to choose its own direction to successfully make it over obstacles such as ramps. In a sense, the robot developed its own version of athletic intelligence.

"The robot was tasked with minimizing consumed energy and making forward progress without falling, leading to complex emergent behaviors," said Ananye Agarwal, a Ph.D. student in robotics who is a co-author on the project.

The research team has posted a paper and videos of the robot dog performing parkour, and has released the project's source code for use by other developers on the open access repository website ArXiv.

"If a person learns to walk in San Diego, they don't have to learn to walk again when they move to Pittsburgh and first encounter snow," Pathak said. "They adapt."