This project will use a convoluational neural network to detect whether a cube is present but partially out of the camera frame, so it is not recognizable by Cozmo's regular vision system.

Simplifying assumptions:

• We only need to train the network on cubes at the right edge of the camera frame. We can run the trained network on flipped camera images to detect cubes at the left edge of the camera frame.

• We only need to examine the right half of the camera image to detect rubes at the right edge of the camera frame. So the training images will be 160x240 instead of 320x240.

• We should only be concerned with cubes that are close enough to the robot that if they were fully in frame, the vision system would recognize them. This distance is 500 mm.

Data collection:

• Choose a variety of backgrounds, e.g., kitchen table, floor, desktop, etc.

• For each background, use a variety of cube distances from 100 to 500 mm.

• For each distance, use a variety of cube positions. In some cases the cube should be fully in frame (but in the right half of the image), and in other cases it should be partially out of frame (on the right side).

• Use a variety of cube orientations, so that the cube is seen both straight on and in diagonal views.

• Use a modest range of head angles.

• Collect these images using all three of the cubes.

• Collect an equal number of images where there is no cube present. To increase the variety in these images, translate or rotate the robot slightly, or change the head tilt angle, or do all of those things.

• Your dataset will consist of two classes. Class 1 images contain a cube. Class 0 images do not contain a cube.

Measure how well the network does on the training set. Look at the cases it gets wrong and see if there is any systematic pattern to the errors.