15-883 Homework #6
Computational Models of Neural Systems
Issued: November 11, 2015. Due: November 18, 2015.
How to Run the Rescorla-Wagner Simulator
You should cd to the directory matlab/rescwag, or download the file rescwag.zip and unzip it.
When you're ready to begin, type "matlab" to start up Matlab. Then
type "run" to start the simulator.
What the Simulator Does
The simulator offers three conditioned stimuli: cs1,
cs2, and cs3. In addition, there is a background
stimulus bkg that is always on. A "trial" is defined as a set
of zero or more conditioned stimuli paired with either a US or no US.
By setting the pushbuttons you can define trials as you wish.
Clicking on the associated "train" button runs the trial and updates
the weights according to the Rescorla-Wagner learning rule. The
buttons are organized as two groups of five trials each, and each
group has an "all" button that will train on all the trials in that
group, and a "x10" button that will train all the trials 10 times.
When you construct an experiment that involves two phases of training,
use the first group of buttons for phase 1, and the second group for
phase 2. That way the entire experiment setup is visible when you
print the contents of the window. When moving from phase 1 to phase
2, press the "Mark" button to add a vertical line to the plot, marking
the transition.
Problems
For problems 1-5 below, create an appropriate set of trials, train the
model, and then print out the window. You can take a screen capture
any way you like (using gimp on Linux; hitting Control-Print Screen on
Windows and then pasting the image into MS Paint; etc.), or you can
print the screen to a file from within Matlab by typing print -deps
-r300 myfile.ps. Make a separate printout for each problem; these
are what you will hand in. Press the "Clear" button when you move on
to the next problem. For problem 6, a one paragraph answer should
suffice.
- Demonstrate blocking. Use one line in the first group of five
trials to train on CS1 paired with the US. Note that if you just
click on the "train" button for that line, both the CS1 and BKG
weights grow. You will need to mix in some trials with no stimuli
(but of course BKG will still be present) in order to train the model
correctly. (If you use the "all" or "x10" button for this group you
will include some of these empty trials.) Train the model for 30
epochs, i.e., 30 sets of trials. You should end up with a positive
weight for CS1 but you should not reach asymptote, i.e., the CR value
(shown in the gray box to the right of the trial) will not be 1.0.
Then, use the second group of five trials to train on CS1 and CS2
paired with the US. Train to asymptote and see what the weights look
like.
- Demonstrate overshadowing. You will need to modify an alpha
variable to make one stimulus more salient than the other.
- Demonstrate extinction of a conditioned response.
- Demonstrate conditioned inhibition, and failure to extinguish a
conditioned inhibitor.
- Demonstrate the effect of partial reinforcement, where the CS is
paired with a US half of the time, and no US the other half of the
time.
- Set up a trial where CS1 and CS2 are together paired with the US.
Set up a second trial where CS1 and CS2 are not paired with the US.
Train on all 5 trials (the extra three should be blank) to asymptote.
Notice that the CR for the CS1-CS2 trials, shown in the gray box to
the right of the trial, is 0.41 rather than 0.5
Now reduce the Beta parameter from 0.1 to 0.02, click on "Start", and
retrain. Notice that the new asymptote is 0.48. Why aren't we
reaching the correct asymptotic value, and why does reducing Beta
bring us closer to it? Explain what's going on here, paying attention
to the form of the weight update rule and the actual values you're
seeing in the simulation. (Hint: try running one trial at a time once
the weights reach their steady-state value. And try calculating the
value of ΔVi by hand after a rewarded trial and after
an unrewarded trial.)
Dave Touretzky
Last modified: Sun Nov 29 15:30:58 EST 2015