15-494/694 Cognitive Robotics: Lab 4

Learning Goal: This lab will give you experience with shape extraction and the MapBuilder. In addition, you will assemble the camera pan/tilt for your Calliope3 robot.

Part I: Ellipses in Camera and World Space

  1. Compose a scene of several easter egg halves for the robot to look at. Write a behavior that uses a MapBuilderNode to look at the scene and extract ellipses into the world shape space.

  2. Examine the camera shape space by clicking on the "C" button in the ControllerGUI. Click on rawY to superimpose the camera image on top of the extracted ellipse shapes.

  3. Examine the world shape space by clicking on the "W" button in the ControllerGUI. How does the distortion of the ellipse shapes vary with distance?

  4. Add another node to your behavior to look in the camera shape space and report how many ellipses the robot sees.

  5. What happens if two easter eggs touch? Does the robot still see them as two separate objects, or does it see them as one large ellipse? Experiment and see.

  6. Modify your behavior so that for every ellipse it finds in the camera image, it constructs another ellipse, centered at the same spot, but with axes that are 50% larger than the original ellipse. The new ellipse should be the same color as the extracted ellipse. When you look in camera space after your behavior has run, and select the rawY image plus all shapes, you should see a collection of ellipse pairs. Take a screenshot to hand in.

Part II: Building the Pan/Tilt

Follow the printed instructions handed out in lab to assemble the pan/tilt for your robot.

Part III: Working with Lines

  1. Use a strip of colored tape to make a roughly vertical line. Arrange easter egg halves on either side of the line. Verify that you can use the MapBuilder to detect both the line and the easter eggs (as ellipses).

  2. Using the online reference pages, look up the pointIsLeftOf() method of the LineData class. Remember to first select the DualCoding name space from the main Reference page before trying a search.

  3. Also in the online reference pages, look up the getCentroid() method of EllipseData. What type of object does this method return?

  4. Modify your behavior to report how many ellipses appear on each side of the line. If there is no line visible, the behavior should report that instead. If multiple lines are detected, just use the first line. Use the setInfinite() method to convert the line shape from a line segment to an infinite line, and notice how this affects the rendering of the line in the SketchGUI.

Part IV: Polygons

  1. Read the documentation for the PolygonData class, focusing on the constructor and the isInside() method.

  2. Write a behavior that looks for three ellipses of a given color (your choice) and forms a closed polygon joining their centroids.

  3. Extend your behavior to look for a fourth ellipse, which will be of a different color, and report whether that ellipse appears inside or outside the polygon.

Part V: Virtual Reality

  1. Point the camera at some ellipses and run the DrawShapes Demo, which you can find at Root Control > Framework Demos > Vision Demos > DrawShapes.

  2. Look in the RawCam viewer and you will see the ellipse shapes superimposed on the raw camera image. Note: this only applies to RawCam, not SegCam.

  3. Now use the Head Control to move the camera, and notice that the shapes stay registered with the ellipses as the camera image changes. Tekkotsu is translating from world coordinates back to camera coordinates in order to draw the ellipses correctly in the current camera image. Because the shapes are in world space, you can also use the Walk Control to move the robot's body, and the shapes will continue to display correctly, modulo any odometry error.

  4. Look at the source code for the DrawShapes demo to see how it works. Essentially, you simply push a shape onto the VRmixin::drawShapes vector and it will automatically be drawn in the camera image.

  5. Write your own behavior that looks for a line, then constructs two small ellipses (in worldShS) that are centered on the endpoints of the line, and causes these ellipses to be drawn in the raw camera image. Include a screenshot of the result.

Homework

  • Hand in your solutions to parts I, III, IV, and V.

  • Include screenshots of the SketchGUI and any results printed on the Tekkotsu console to document what your code is doing.

  • Your answers are due by Friday, February 12.