Dense Estimation of Motion and Appearance



Separating images into background/ foreground layers and estimating.
Top row: (1) Image sequence, (2) foreground appearance, (3) foreground ownership weight, (4) foreground mixing probability, Bottom row: (1) Horizontal and (2) vertical motion components for foreground layer. (3) Horizontal and (4) vertical motion components found by Black&Anandan approach,


Modeling or explaining appearance change in image sequences is one of the challenging problems of computer vision. A great deal of research has been done to understand what goes with what in the scene, separate image sequences into depth layers, and model background/foreground using temporal consistency, spatial coherence or layers. However, few previous approaches benefit from the motion information or fuse different sources of information.

To address above problem and explain occlusions and disocclusions, we develop a probabilistic model of optical flow. Our objective is to separate image sequences into layers (e.g. background or foreground for a simple two-layer case), model the appearance of each layer and characterize the motion in different layers.




Separating images into background/ foreground layers and improving the stability of optical flow.



In preliminary research to cope with above problem, we worked on learning and adapting information in image sequences by formulating the optical flow problem in a Bayesian framework. We investigated the improvement of the stability of optical flow estimates by estimating an appearance model that captures the structure of the image texture being tracked. Experimental results of this work motivated us to use motion information in estimating appearance models. We develop a probabilistic model of optical flow that can learn and adapt appearance models combining motion information with temporal consistency, spatial coherence and layers.

More experimental results on background/boreground separation ...

Experimental results on fitting mixture models to both motion and appearance for a static sequence...

Experimental results on crude computation of dominant motion...




[Research]
        [Tracking Vehicles In Airborne Video Imagery]
        [Dense Motion and Appearance Estimation]
        [Implicitization by Matrix Annihilation]
        [Modeling and Measurement Using IPs]
        [Automated Sorting of Remote Controllers ]
        [Shape Assessment by Selective Fixations ]



Related Publications
  • Hulya Yalcin, Michael Black, Ronan Fablet,
    "The Dense Estimation of Motion and Appearance,"
    Second IEEE Workshop on Image and Video Registration (IVR'04) in conjunction with CVPR'04. [Abstract] [pdf]

    • Resume | Research | Main Page
    Carnegie Mellon University, Robotics Institute
    5000 Forbes Av., Pittsburgh, PA, 15213
    hulyayalcin@gmail.com