Project 2 - Face Morphing

by Faustinus Kevin Gozali (fkg)

Morphing my face to Jing Jin's

My original face
 Jing Jin's

My face points is here, Jing jin's is here. Here are the morphing frames (from top left to bottom right). All of them can be found here.


The Mean Face

Using the class data, we compute the mean shape, i.e. by averaging the feature points of all faces. We then morph each face towards this mean shape and average the result. Here, I only used 15 faces as some of them do not have point sets specified. The data I used can be found here.
Mean Face
My Face morphed to mean shape
 Mean Face to my face shape

Caricature by extrapolation

By having the mean shape computed, we can produce the extrapolation versions of my face as follow:
My face away from meanface, warp_frac = -0.6
(big head, small chin)
 My face towards meanface, warp_frac = 1.2
(ugh, so fat...)

Fun with PCA

Using the class data, we compute the U,S,V matrices for both the shape vectors and the texture vectors. For texture vectors, I computed one from the original pictures, and one for all faces morphed into the meanshape computed before. This is simply vectorizing both point sets and textures. Using SVD, we obtain U-S-V matrix for shape, and U-S-V matrix for texture. Here are some displays for the first 10 (weighted) texture eigen vectors. Note that the images shown here are the absolute values of the actual values, and scaled to [0,1] range, since obviously imagesc complains if it is given a 3D image with values outside [0,1].

First 10 eigen vectors in texture space (without warping to meanface)

First 10 eigen vectors in texture space (after warping to meanface)


Once these eigen vectors for shape and texture are computed, we can play around with their weights, producing various new shapes and textures.

Using less basis VS more basis

The first picture on the top left has the least basis used. The last picture on the bottom right has most basis used. Notice that the more basis we use, the image we get is more toward the original face.

A. using the texture vectors being warped to meanface

B. using the original texture vectors without warping to meanface

Messing the weight of each basis

For this experiment, I messed the weight of each basis shape and texture (one at a time), by multiplying them by -2 and 2. Every two images is produced by messing the same basis, just by different factor.


Applying random scales to all weights of the eigen basis

By multiplying each weight from my original face by a random scale, we can produce even more caricatures. Different weights give different effects on different basis. Note that the random scales are applied to both texture and shape spaces, producing more varying results. Note that the less important the eigen vector is, the less effect it gives to the result. Some images do not really change due to the small weight/importance

Face Generation by randomly selecting basis

Using the basis that we have, I assigned some random weights on the shape space, such that the absolutely new shape are produced. I morphed my face towards this new face and here are the results (some ugly results are not shown, though they are fun):

Using PCA we could generate a lot of different faces/caricatures, much better than the normal basis.

All of these images can be found here.


October 3, 2006