One lesson to be drawn from the stage debut of HERB, an acronym for Home Exploring Robot Butler, might be summarized by paraphrasing a legendary theatrical quote:

“A system crash is easy. Comedy is hard.”

The May 1 performance of David Ives’ “Sure Thing” by HERB and his human co-star, drama major Olivia Brown (A’15), was well-received by the audience. If not flawless, it at least generated laughs in all of the right places. But creating those 12 minutes of thespian magic in the Helen Wayne Rauh Studio Theatre required months of preparation by an ad hoc team of researchers and students from the Robotics Institute and the School of Drama.

“Live theater is one of the hardest demos we have around here,” said Garth Zeglin (CS’99), a project scientist in the Robotics Institute. “It happens at a set time and everything has to work then and there. ‘Theater quality’ is ‘industrial quality’ in many ways.”

(Check out the video of the performance below.)

HERB is a two-armed mobile robot developed by Siddhartha Srinivasa, associate professor of robotics. The robot serves as a testbed for the technology and software that someday will enable robots to work in people’s homes — fetching the TV remote, microwaving dinner and performing other household tasks. Though it was featured in a humorous promotion for Oreo cookies last year, HERB was never meant to be an actor.

So when Entertainment Technology Center student Katie Correll (MET’14) got the idea to cast HERB in the short, two-character comedy, Zeglin, director Sam French (A’14) and a group of drama and robotics students set about to give the robot some dramatic chops.

The structure of the play suited a robot lead performer. HERB played a man trying to pick up a woman at a café. Each time one of his seduction lines landed with a thud, a bell would ring, time would reverse and he would try a new approach until successful, at which point the action progressed.

“It sort of models a search computation process,” Zeglin said.

Still, HERB needed to learn what to do with his hands. “Currently, most robot motion is very functional,” Zeglin explained, focusing on moving limbs and hands so the robot can pick up or manipulate an object. As an actor, HERB needed to be more expressive, moving arms and its head in concert and in ways that might not be human-like, but which conveyed feeling.

As for his voice, the team turned to an audio version of motion capture: an actor would record all of the lines in advance, providing inflections and accentuations that could drive HERB’s synthesized voice. When the play required him to repeat some lines, the playback could be altered accordingly.

Emphasis was placed on integrating HERB into the rehearsal process from the beginning, both in the Personal Robotics Lab and the theater.

“It was good that we had those rehearsals because there was a lot of potential for failure during the performance,” said Don Zheng (E’17), an electrical and computer engineering major responsible for operating HERB.

“There was a noticeable lag between when I cued an animation and when the robot would execute it,” Zheng said, “so for most of the lines I had to lead in the cues. But if I cued the animation too early, while other animation was still running, the robot would hang for a few seconds as it tried to plan the next starting pose. This happened often during rehearsal but thankfully only once during the performance.”

HERB was more of a puppet than an actual performer, Zheng said, and several audience members inquired about robot autonomy during the on-stage discussion following the performance.

“Full autonomy would require human-level intelligence,” Zeglin said. Even getting the robot to react to the audience would be a challenge, given the buggy speech recognition programs and imprecise computer vision programs now available. “But there’s a will in the lab to continue to use HERB in more theatrical productions,” he added.