15-494/694 Cognitive Robotics Lab 7:
OpenCV

I. Software Update and Initial Setup

1. At the beginning of every lab you should update your copy of the cozmo-tools package. Do this:

   $ cd ~/cozmo-tools
   $ git pull
   

2. Note: new simple_cli commands "show pose" and "show landmarks".

3. For this lab you will need a robot, a charger, some chips, and some paper.

4. Log in to the workstation.

5. Make a lab7 directory.

6. Connect the Kindle to the robot and start simple_cli.

II. Chip Detection

1. Download and run Chipper.fsm, which detects red chips.

2. Read the code.

3. The code converts the color image from RGB space to HSV (hue, saturation, and intensity) space. Read this tutorial to learn about color spaces and why HSV is useful.

   
   

4. Note that while most implementations of HSV use hue values from 0 to 360 degrees, in OpenCV the range is 0 to 180 degrees. Red is at 0 degrees, so to get the full range of red values we must check for two ranges: 0-10 degrees, and 170-180 degrees.
   

5. Modify the Chipper program to display the image contours in a separate window. Look at the CV_Contours demo in cozmo-tools/cozmo_fsm/examples to learn how to do this.

6. Color image segmentation is sometimes unreliable because of lighting variations. An alternative approach to chip detection is to use a bullseye pattern and look for nested contours. Write a new demo BullseyeChipper that does this.

III. Shape Matching

1. Read the OpenCV tutorial on shape matching.

2. Run the match_shapes.py program, giving it the files star-a.jpg, star-b.jpg, and star-c.png.

3. Review the findContours documentation.

   c

4. Make up your own contour for a shape like the four-pointed star in the example.

5. Use shape matching to have Cozmo detect your shape when it's pasted to the wall. Create a display in OpenCV to show your results.

IV. Optical Flow

1. Run the CV_OpticalFlow demo and move a textured object in front of the robot to see how the system detects optical flow.

2. Write your own program that uses optical flow to make Cozmo track a moving object by turning towards it.

Hand In

Collect all your code and some relevant screenshots in a zip file.

Hand in your work through AutoLab by the date indicated in AutoLab.