Novel View Modeling from a Single Image Using TIP (Tour into the Picture)

For Computational Photography, 15-463 (Project 5)

Carnegie Mellon University jmm59@pitt.edu

This project takes a single 1-point perspective image (usually a room), and builds a three-dimensional representation of the space portrayed. The camera can then be moved to achieve new viewpoints of that interior.

Our starting image:

Surface Establishment

The first part of this project is to define the surfaces that make up the interior of our room. A GUI is provided to select the upper-left and lower-right corners of the back wall. Then the vanishing point is selected such that the lines that radiate from the VP to the corners of the inner rectangle correspond to the edges of the four surrounding walls. Among the inner rectangle, the radiating lines, and the border of the image itself, we now have the image divided into five quadrilaterals.

Image Rectification

Once we’ve outlined the five surfaces, we then rectify them so that the quadrilaterals are rectangular. This gives us rectangles that (somewhat) represent what the surface would look like if looked at from straight-on. Since the back wall is composed of the inner rectangle, this surface requires no rectification.

Rectified Floor, Ceiling, and Left and Right Walls:

Model Building

Now that we have the five surfaces of our model, we just need to put them in the right place. We choose an arbitrary scale and starting position for the back image, and build the other four walls around it. The only value we need to know about the scene is the focal length, in order to determine the depth of the room and of the (near) edges of the other walls. The focal length can be determined from the actual focal length used to take the picture, or it can simply be guessed at until an appropriate depth is found.

Once all this has been completed, we can reposition the camera for new viewpoints of the same surroundings:

I tried the same technique using other images as well.

Here’s a picture of my hallway (focal length: 7mm)

And new perspectives:

And here’s an exterior sidewalk (7mm):

And the new directions:

This one is taken from the perspective of the lion at the end of the corridor, looking towards the (original) camera position.

I also took a picture of the same corridor using a telephoto lens (18mm):

This is what the model looks like from outside the box ;-)

Here’s a screenshot of the windows used together: