This homework assignment will be made available in 2 parts. Details of the first part are given below and will be due by Friday, Oct 4th.
Details of the second part will be made available after that.
All data for this project are available from the links in the Downloads section below.
NEW HINTS ON SOLVING THE HOMEWORK
You are a corpus of facial images. You must learn a "typical" (i.e. Eigen) face from one of them.
You are also given four group photographs with many faces. You must use the Eigen face you have learnt to detect all faces in the photos.
The faces in the group photo may have different sizes. You must account for these variations.
Use matlab, if you can. Other similar tools, such as “Octave” or “Python” (which comes with some very nice scientific and visualization libraries) are also reasonable alternatives. The machochistic among you may want to do it all in “C”.
Unlike HW1, make sure the code you write for this problem is generic, since you will need a modified version of it for Problem 2 in this homework assignment. We will also ask you to experiment with different corpora for training your system.
You are given a collection of images of faces (see the downloads section) from image database 1. These images are all dimension 64x64. From these you must compute the “best” Eigen faces.
Some hints on how to read files into matlab can be found here
You must compute the first Eigen face from this data. To do so, you will have to read all images into a matrix. Here are instructions for building a matrix of images in matlab. You must then compute the first Eigen vector for this matrix. Information on computing Eigen faces from an image matrix can be found here.
To detect faces in the image, you must scan the group photo and identify all regions in it that “match” the patterns in Eigen face most. To “Scan” the image to find matches against an N x M Eigen face, you must match every N x M region of the photo against the Eigen face.
The “match” between any N x M region of an image and an Eigen face is given by the normalized dot product between the Eigen face and the region of the image being evaluated. The normalized dot product between an N x M Eigen face and a corresponding N x M segment of the image is given by E.P / norm(P), where E is the vector (unrolled) representation of the Eigen face, and P is the unrolled vector form of the N x M patch.
A simple matlab loop that scans an image for an Eigen vector is given here
The locations of faces are likely to be where the match score peaks.
Some tricks may be useful to get better results.
The Eigen face is fixed in size and can only be used to detect faces of approximately the same size as the Eigen face itself. On the other hand faces in the group photos are of different sizes -- they get smaller as the subject gets farther away from the camera.
The solution to this is to make many copies of the eigen face and match them all.
In order to make your detection system robust, resize the Eigen faces from 64 pixels to 32x32, 48x48, 96x96, and 128x128 pixels in size. You can use the scaling techniques we discussed in the linear algebra lecture. Matlab also provides some easy tools for scaling images. You can find information on scaling images in matlab here. Once you've scaled your eigen face, you will have a total of five “typical” faces, one at each level of scaling. You must scan the group pictures with all of the five eigen faces. Each of them will give you a “match” score for each position on the image. If you simply locate the peaks in each of them, you may find all the faces. Sometimes multiple peaks will occur at the same position, or within a few pixels of one another. In these cases, you can merge all of these, they probably all represent the same face.
Additional heuristics may also be required (appropriate setting of thresholds, comparison of peak values from different scaling factors, addiitonal scaling etc.). These are for you to investigate.
Exact details will be given on the date Problem 1 is due.
In this problem we will use multiple Eigen faces to detect faces in an image.
You will also be given a collection of non-face images.
You must express all images, both face and non-face, in terms of the top N Eigenfaces. The N weights you obtain in this manner will be used to represent the images. You must then learn a boosted classifier to distinguish between faces and non-faces.
You will have to use the classifier from problem 2 to detect faces in the given images.
Download the training database of faces from here. Each image in this corpus should be 64X64 and in grayscale. The corpus was obtained from the LFWcrop Database.
Download the group photos in which you must detect faces are here, here, here and here. You may recognize these pictures.
Problem 1 is due on Oct 4, 2009. What to submit:
1. a brief write-up of what you did
2. the segments that your detector found to be faces. You may either copy those segments into individual files into a folder, or mark them on the group photograph. Make sure we can understand which part(s) of the image was detected as a face.
3. You do not need to submit code. However, we might ask you for the code at a later point, so try to document the code well.
How to submit:
Put the above in a zipfile and name it "MLSP-HW2-Prob1-FirstnameLastname.zip".
Send the zip file by email to Sourish (sourishc@cs.cmu.edu) with "MLSP HW 2 Problem 1 Submission" as the subject line.
You may include any other information if necessary in a "Readme.txt" in the zip file that you submit. Do not put important information in the body of the email.
