15-883 Homework #3
Computational Models of Neural Systems

Issued: September 23, 2013. Due: September 30, 2013

This assignment makes use of files in the course's matlab subdirectory, which is linked from the course home page. From this directory you can download zip files containing any of the demo programs used in the course.

Part I: Run the Simple Matrix Demo

This Matlab script demonstrates creation of a matrix memory using the outer product rule. The memory stores three items correctly using orthogonal keys; it fails catastrophically when a fourth item is added whose key is not orthogonal to the others. However, the four keys are linearly independent, so a new memory is constructed using the matrix inverse technique instead of outer product, and it is verified to work correctly.
  1. Go to the matlab/simplemat directory.
  2. Execute the file script1.
  3. Each time you see the word pause, press the space bar to proceed.
The file script1.log shows the output you should see if you execute the script correctly. You do not have to hand anything in for this part. But make sure you understand what the script is doing, because you will be performing similar computations later in this assignment.

Part II: Run the Graphical Matrix Memory Demo

You should cd to the directory matlab/matmem, or download the file matmem.zip and unzip it. Read the README file before proceeding further.

When you're ready to begin, type "matlab" to start up Matlab. Then type "matmem" to start the matrix memory demo.

Experiment with the matmem demo:

  1. Click on the Random button to generate a set of random memory items. Then start clicking on memory item buttons. As you click on a button, it turns pink, and the result box to its right should also turn pink to show you that the stored item is successfully retrieved. As you click on additional memory items, the memory fills up, and eventually one of the corresponding result boxes will turn blue, indicating that retrieval of that item is not completely correct.

    Questions: How many items can you store before the first retrieval error occurs? What percentage of memory bits are active when the first retrieval error occurs? Hit the New button and then Random to generate another set of data items. Try the experiment several times and report average values.

  2. Switch from a dense to a sparse representation by clicking on the Dense button. The button label will change from "Dense" to "Sparse". Now instead of using a 5-bit binary code, each letter is stored using a 2-of-8 bit code. This requires 24 input units instead of 15. Re-run the previous experiment a few times, but using sparse representations.

    Questions: How many items can you store using the sparse representation? What percentage of memory bits are turned on when the first retrieval error occurs?

Hand in your answers to the questions above.

Part III: Building Your Own Associative Memory

Consider an associative memory using binary keys with a randomly chosen 3 out of 10 bits true. Here is a function for generating such keys: 

function Z = keyvec(N,B)
 % Generate an N-bit binary vector with a random B bits true
  Z=zeros(N,1);
  P = randperm(N);
  Z(P(1:B)) = 1;

Complete this Matlab program to fill a memory with key-value pairs until the memory capacity is exceeded. The keys will be random. For the values to be stored, we'll use 1 for the first pattern, 2 for the second, and so on. Since the keys are not orthogonal, we will use the pseudo-inverse method to calculate the weight matrix. Here is the program; you need to fill in some missing parts. 


N = 10;           % length of a key
B = 3;            % number of 1 bits in a key
Keys = [];
Values = 0;       % values to store
Results = 0;      % results of retrieval using Keys

M = zeros(1,N);   % weight matrix

NumPatterns = 0;
while max(abs(Values-Results)) < 1e-10
  NumPatterns = NumPatterns + 1;
  k = keyvec(N,B) / B;   % normalized random key
  Keys = [Keys k]
  Values = 1:NumPatterns;
  M = ... fill this in to construct the weight matrix
  Results = ... fill this in to retrieve the values given the keys
end

fprintf('Stored %d patterns without error.\n',NumPatterns-1);
Questions:
  1. Run the program a bunch of times. How many patterns can you typically store in the memory?
  2. Occasionally you'll have a "bad" run where only 2-4 patterns are stored before an error occurs. What is the cause of this?
  3. If you change the value of B from 3 to 7, the keys are less sparse and less nearly orthogonal. What effect does this have on the memory capacity? (Don't just guess; try it and see.) Explain the reason for your result. Hand in your source code plus the answers to all questions.
    Dave Touretzky
    Last modified: Sun Oct 6 21:11:22 EDT 2013