15-869: Image Based Modeling and Rendering

Chris Rodriguez

Project Progress Report

Problem:  Create a system for real-time motion capture for use as an input device for a personal computer.

Previous Work: 

1. W⁴: A Real Time System for Detecting and Tracking People (Haritaoglu, Harwood & Davis)
   • Subtraction by first 'learning' what range of values each pixel has and thresholding according to that.
   • Post processing to remove noise and correct the size of foreground objects.
   • Method for human tracking assumes 'cardboard model' base
2. Pfinder: Real-Time Tracking of the Human Body (Wren, Azarbayejani, Darrel & Pentland)
   • Subtraction takes 'learning' color distributions modeled with a Gaussian, updated with a adaptive filter.
   • New Pixels are compared to original model to see if they are statistically different
   • Humans identified by stored known human positions.
   • Some of it's applications include gesture control of applications
3. Real-time human motion analysis by image skeletonization (Fujiyoshi & Lipton)
   • Star skeletonization to identify extremal points.

Algorithm:

1. Image Subtraction

   • Use the thresholding method to gain an initial guess as to what parts of the image are moving, because it is computationally simple to perform.
   • From this do a small amount of image processing to gain a bounding silhouette and eliminate noise.
   • Use the Gaussian on this image silhouette to more precisely eliminate background pixels.  (the method described in the paper about Pfinder handles removing shadows).

2. Image Skeletonization

   • After obtaining this removed image, skeletonization could be preformed
   • Using the method in [3], find border positions that are the farthest away and connect these to a centroid of the silhouette.
   • Track the position of the object using the centroid, and any gestures with the skeleton

Novel:

   I describe using a combination of algorithm to do the image subtraction and skeletonization.  Also, these methods were used to identify human motion.  I plan to use this algorithm as a hand gesture tracker for use as an input to a personal computer. 

Testing:

    After initially calibrating the system with a background, check for new objects in the scene.  The backgrounds I would use would be simple, as well as have very few lighting changes.  I will then use add another object in the scene.  My test case so far is a cluttered desk with a drop cloth behind it and a ping pong ball moving through it.  Once I have the ability to track a new object in the scene, it will be easy to test the image skeletonization by visualizing it on the computer screen.  I will attempt to get direct input to the computer of a hand and attempt to track the 5 most extreme points - which should be the fingers.  Then test if I can save or match motions.