Representation and Matching of
Articulated Shapes

Jiayong Zhang   Robert Collins   Yanxi Liu

Abstract
We consider the problem of localizing the articulated and deformable shape of a walking person in a single view. We represent the non-rigid 2D body contour by a Bayesian graphical model whose nodes correspond to point positions along the contour. The deformability of the model is constrained by learned priors corresponding to two basic mechanisms: local non-rigid deformation, and rotation motion of the joints. Four types of image cues are combined to relate the model configuration to the observed image, including edge gradient map, foreground/background mask, skin color mask, and appearance consistency constraints. The constructed Bayes network is sparse and chain-like, enabling efficient spatial inference through Sequential Monte Carlo sampling methods. We evaluate the performance of the model on images taken in cluttered, outdoor scenes. The utility of each image cue is also empirically explored.

Figure 1. Overview of our approach. An articulated non-rigid 2D body contour model (left) and local image cues (middle) are combined via Bayesian graphical modeling. The model is fit using sequential Monte Carlo to a sample image (right) taken in a cluttered, outdoor scene.

Publication
The paper at CVPR'04: Presentation slides at CVPR'04:

Results
Training

Sample results on fitting the indoor training set, using a uniform shape prior. Plotted are the posterior means.
  • example 1 MOV (0.5MB)
  • example 2 MOV (0.4MB)
Test

Quantitative evaluation on the outdoor test set. Plotted are the posterior means, with symmetric chamfer distance scores shown in the top corners (left-body, right-arm).
  • 50 selected frames MOV (0.5MB)
Visualizing SMC Inference

Demonstration of the inference process of Sequential Monte Carlo, with the distribution of each vertex summarized by the shape of its covariance ellipse.
  • example 1 MOV (1.3MB)
  • example 2 MOV (1.4MB)
Performance on Video Sequences

Plotted are the posterior means. Each frame is matched independently.
  • example 1 MOV (6.8MB)
  • example 2 MOV (8.0MB)
  • example 3 MOV (5.6MB)
  • example 4 MOV (6.8MB)

Last update: May 15, 2004