15-494/694 Cognitive Robotics Lab 8:
ALVINN

I. Experiment with Classic ALVINN

  1. Make a lab8 directory.

  2. Download the file data.zip into your lab8 directory and unzip it.

  3. Make a lab8/python subdirectory and download the file alvinn1.py into it.

  4. Read the alvinn1.py source code.

  5. Run the model by typing "python3 -i alvinn1.py". The "-i" switch tells python not to exit after running the program. Type train_alvinn() to train the network.

  6. Examine the hidden unit weights by typing show_hiddens(). You can examine an individual hidden unit, e.g., unit 2, by typing show_hidden(2).

  7. Retrieve the model paramters:
    p = list(model.parameters())

  8. What do the parameters look like?
    [param.shape for param in p]

  9. What do the output unit bias connections look like?

  10. Turn off weight decay; how does this affect the loss? How does it affect the weights?

  11. Type test_alvinn() to run the network on a test set of 97 similar road images. How well does it do?

  12. Continue the training by typing train_alvinn() again. Have we reached aymptote? How well do we do on the test set now?

  13. Write a function test_alvinn2() to test the network on two lane roads, which are also supplied in the ALVINN dataset.

  14. Use the supplied function closest_gaussian to compare the shapes of the gaussians produced for two-land roads to the ideal gaussians supplied to you in the variable gaussians by plotting one against the other. Make a similar comparison for the output patterns produced on the test set. This difference from an idealized gaussian is what Pomerleau called Output Appearance Reliability Estimation.

II. Convolutional ALVINN

In this section we're going to try using a convolutional neural net like the mnist3 model to see how it handles the ALVINN data. We'll use five kernels, each 5x5 pixels, and 2x2 max pooling.

  1. Download the file alvinn2.py into your lab8/python directory.

  2. Type train_alvinn() to train the model.

  3. Write the show_kernel(n) function to display the learned kernels.

  4. Write the test_alvinn() and test_alvinn2() functions to run the trained model on the test set and two-lane road dataset.

  5. How well does this model do on the test sets, compared to alvinn1?

III. Calypso

In this exercise we're going to use the cloud version of Calypso at calypso-robotics.com.

  1. Using the map layout editor, make a structure with two rooms separated by some walls with doorways.

  2. Place all three light cubes in one of the rooms.

  3. Write a state machine to move all the cubes from one room to the other. To do this, the robot must move to a cube, grab it, and then move to the other room and drop the cube.

Hand In

Hand in the following in a file called handin.zip:

  1. Your modified alvinn1.py and alvinn2.py files.

  2. A brief writeup describing your observations about performance of the network:
    1. Compare their mean loss on the training set.
    2. Compare their mean loss on the test set.
    3. Compare their mean loss on two-lane roads.
    4. How does OARE differ between the training set, test set, and two lane roads?

  3. Screenshots of your Calypso rules and world map.

Dave Touretzky