Programming Assignment 2 from 15-869: Single View Modeling

The original image:

This image is the scan of the painting "Birth of the Virgin" (Juan de Borgona). I found it from the book "Painting in Spain"(pp. 25). I think this image is good for this assignment since it is a beautiful painting and we can find parallel lines in X, Y, and Z direnctions very easily. But the serious occlusion makes modeling more difficult.

Computing vanishing points and vanishing line:

Here I used the method described by Collins. The white, green and red lines respectively represent the parallel lines in X, Y and Z directions.

A view of the 3D reconstructed model of the original image:

I used 14 planar patches to get this 3D reconstructed model. The following are some of the texture maps for these patches.

The front floor:

The back floor:

The old man:

The boy:

The middle wall:

The right wall:

This patch consists of several objects, such as the mother, the maid, the bed and the wall. Since this patch is too complex, I can't devide it into individual patches. I used the code for Assignment 1 to do texture mapping.

The girl and the baby:

I can't find the reference points for the baby, so I combined it with the girl into one patch.

One column:

The back wall:

Here I filled the occluded parts by Paintshop.

The VRML file:

birth.wrl

Some problems about this model:

In this model, some relative scales are a bit off--the women on the right look huge compared to the two men off to the left. Also, some of the walls don't seem to meet at right angles and the arch in the background is skewed. (Steve mentioned them to me.)
I think these things may be caused by the following reasons:
1. The mistake of the painter. I do find it in this painting, especially in its back part. Some parts of this painting do not satisfy the projective thoery strictly.
2. The surface of a people is not a plane, but I processed it as a plane. So the corresponding points may be inaccurate. In addition, it is difficult to find corresponding points for some objects, such as the girl with the baby in hands, and so on. In this image, the right scene (after the bed) is very complex, and I can't find the corresponding points for almost all the objects there, so I process the entire scene as a plane and the mother was stretched by image warping.
3. Unfit view points. We can get different views of this model from different view points. Some view points gave better results than the other, since I used planar models.
4. Numerical error in the calculations. I guess it wouldn't have much effect on the final results and so it may not be the main reason of these phenomena.

Conclusion:

In this assignment, I used the technique described by Criminisi as well as our class notes to get the 3D model of a painting.
I found if I just used the image pixel coordinates directly, the result will be inaccurate, especially when I wanted to get X, Y coordinates of a point with height z and image pixel coordinates (u,v). As Collins said, I set the constant w to be equal to half of the image size in pixels, and then the results became much more accurate.
If I have more time, I can try this method on photographs and do some further work.