Calypso Module 8: Recognizing Faces

Learning Goal: In this module you will learn how to make Cozmo recognize faces and detect facial expressions.

Face Detection

Tilt Cozmo's head up and position him so he can see your face. Notice the green box that appears around your face in the camera viewer. The box will be labeled with your name if Cozmo recognizes you; otherwise it will say "Unknown face".

Notice that when a face is detected it is added to the world map. This way Cozmo can keep track of where his humans are.

We can write rules to respond to the presence of a face. Let's make Cozmo greet anyone he meets by saying "Hello, human!". We'll require that the face be visible in the current camera frame, so if a face goes away and comes back later, Cozmo will greet it again. But to prevent excessive greeting, we'll use the "once" tile.

We can also have Cozmo say something when he does not see any faces in the camera frame. This rule uses the "not" modifier. To make a "not see" tile, select "not" from the petal menu, then select "see".

Try running the program and moving your face into and out of the camera frame.

Teaching Cozmo to Recognize Your Face

Face detection allows Cozmo to know when he's looking at a face. Face recognition tells Cozmo whose face he's looking at. You must teach Cozmo your face if you want him to recognize it. This is easy to do:

Go to the Home menu, select Settings, then select Faces.
Press the A button add a new face.
Type in your name and press the A button.
Now position your face so Cozmo can see it and press the A button one more time.
Cozmo will move his head and body to adjust his view, and then memorize your face.

Now when you see your face in the camera viewer, your name should appear in the green box. Cozmo knows who you are.

Normally "see face" matches any face. But you can set it to match a specific face by telling it the name. To do this, put the pencil on the "face" tile and press the Y button. This pops up a menu where you can type the name of the face you want to recognize.

With the rule below, whenever Cozmo sees Chris's face he will drive forward:

We can also write a rule that looks for anything that is not Chris. We'll make Cozmo play a bark sound if he sees something. And we'll omit the "once" tile so that like any good guard dog Cozmo keeps barking until the thing goes away.

Facial Expressions

Cozmo can recognize several facial expressions: happy, angry, sad, surprised, or neutral. How does he do this?

Experiment: Try making faces at the camera and see how the emotion label changes.

Notice that the face icon in the world map changes color to indicate the currently perceived expression: magenta for happy, blue for sad, red for angry, orange for surprised, and green for neutral. If Cozmo cannot recognize what the face is expressing then the icon will be gray, for "unknown".

Calypso provides tiles for each of the recognized facial expressions:

Happy
Sad
Angry
Surprised
Neutral

We can use these tiles to write rules that respond to particular expressions. Here's a rule that tells Cozmo to act playful whenever he sees a human face that appears surprised. If you change your expression from surprised to something else, and then change it back to surprised, Cozmo will act playful again:

How Does Cozmo Describe A Face?

You can get an idea of how Cozmo describes a face by looking closely in the camera viewer, or by looking at the details of the expression tiles above. A face is composed of four elements: two eyes, a nose, and a mouth.

Each eye is represented by three points, forming a triangle.
The nose is represented by two points, forming a line.
The mouth is represented by four points. These can form a quadrilateral (as in "surprised"), a triangle, a curved line, or a straight line.

Thus, a facial expression is represented by 12 points, or 24 numbers, since each point has both an x and a y coordinate. There is a complex mathematical function that maps these numbers into expressions like "happy" or "sad".

Review and Discovery

Do the Module 8 Review and Discovery activity to review what you've learned.

Next Module

In the next module you'll learn how to make Cozmo do two things when a WHEN condition is true.

Back to Calypso Curriculum overview.