ECCV 2010 Tutorial on
Nonrigid Structure from Motion

Slides available here.

Building a 3D model from a sequence of images is of fundamental importance for a variety of applications. A number of approaches have been proposed in the last three decades of research which utilize stereoscopy; these make the assumption that the scene is stationary between capture of two images of a scene. As most cameras available today are monocular, and as most scenes of interest contain elements that change over time, considering the case when the stereoscopic constraint does not apply, i.e. when the 3D points move between the capture of two images, has wide application.

In this tutorial, we will review the existing corpus of work in nonrigid structure for motion, include shape and trajectory representations, estimation paradigms, fundamental ambiguity in NRSFM solutions and the additional constraints proposed by different researchers for a numerically stable solution. The tutorial will give an overview of the area and identify the key challenges and open problems that remain to be solved.

The course will cover the following topics:

1. Introduction to Structure from Motion
2. Review of Rigid Structure from Motion
   a. Factorization
   b. Multilinear/Bundle Adjustment
3. Introduction to Nonrigid Structure from Motion
4. Nonrigid Object Representation
   1. Shape Representation
   2. Trajectory Representation
   3. Duality
   4. Joint Space-time Representation
5. Ambiguity Analysis
1. Orthonormality
2. Fundamental Ambiguity
6. Optimization Strategies
1. Factorization Solutions
2. Multilinear Solutions
7. Additional Constraints
1. Orthonormal Basis
2. Gaussian Prior
3. Rigid Subset
4. Projection Pursuit
5. Metric Projections
6. Perspective Solutions
8. Open Research Directions

Yaser Sheikh is an Assistant Research Professor at the Robotics Institute, at Carnegie Mellon University. His research is in computer vision, primarily in analyzing dynamic scene including human activity analysis, dynamic scene reconstruction, mobile camera networks, and nonrigid motion estimation. He obtained his doctoral degree from the University of Central Florida in 2006. He served as Associate Editor for IAPR International Conference on Pattern Recognition 2010, Program Committee Member for Three Dimensional Information Extraction for Video Analysis and Mining, 2010 (held in conjunction with CVPR 2010), and Track Chair for International Conference on Multimedia Computing and Information Technology, 2010. He won the Honda Initiation Grant Award in 2010, the best paper award at the ICCV THEMIS workshop in 2009, and the Hillman Fellowship in 2004.

Sohaib Khan is Associate Professor of Computer Science at the LUMS School of Science and Engineering, Lahore, Pakistan and the founding director of the Computer Vision Lab (http://cvlab.lums.edu.pk) at LUMS. His research interests broadly span the areas of image and video analysis, including image registration, multiple camera surveillance systems, structure from motion and satellite and aerial image processing. Dr Khan earned his PhD degree in Computer Science in 2002 from University of Central Florida, specializing in computer vision. He received BE degree in Electronics Engineering from GIK Institute of Engineering Sciences and Technology in 1997. He is an associate editor of Machine Vision and Applications journal. He served as the technical program co-chair of International Conference on Machine Vision, 2007, local arrangements chair for 3rd International Workshop on Frontiers of Information Technology 2005, and on the program committee of IEEE Workshop on Motion and Video Computing 2002. Dr Khan was awarded Hillman Award for Excellence in PhD Research in 2001 and Best Teacher Award by LUMS Computer Science Department in 2007.