From 3D Scene Geometry to Human Workspace

"the meaning or value of a thing consists of what it affords."

-JJ Gibson (1979)

Idea

We present a human-centric paradigm for scene understanding. Our approach goes beyond estimating 3D scene geometry and predicts the "workspace" of a human which is represented by a data-driven vocabulary of human interactions. Our method builds upon the recent work in indoor scene understanding and the availability of motion capture data to create a joint space of human poses and scene geometry by modeling the physical interactions between the two. This joint space can then be used to predict potential human poses and joint locations from a single image. In a way, this work revisits the principle of Gibsonian affordances, reinterpreting it for the modern, data-driven era.

Results

The results produced by our system can be downloaded from here.

Downloads

Dataset

We used the Indoor Scene dataset from Hedau et al(ICCV'09). This dataset can be downloaded from here .

Code

Code for Computing Affordances (Given 3D Geometry) Available Soon

The main code for computing joint human-scene interaction space can be downloaded from here.

Code for Computing Geometry Estimates (Using Lee et al. and Hedau et al.) Available Below

Above code requires computing the 3D scene geometry from a given image. Our code which combines geometry from Hedau et al. (ICCV'09) and Lee et al. (NIPS'10) can be downloaded here.
For computing the 3D scene geometry please download the original code from Hedau et al. (ICCV'09) and Lee et al. (NIPS'10); and use the matfiles produced by their approaches with our code.

Please cite the paper if you are using the code.

Citation

Abhinav Gupta, Scott Satkin, Alexei A. Efros and Martial Hebert, From 3D Scene Geometry to Human Workspace, Computer Vision and Pattern Recognition, 2011. (PDF)

Bibtex Reference

@inproceedings{GuptaSatkinEfrosHebert_CVPR11,
   author="Abhinav Gupta and Scott Satkin and Alexei A. Efros and Martial Hebert",
   title="From 3D Scene Geometry to Human Workspace",
   booktitle="Computer Vision and Pattern Recognition(CVPR)",
   year="2011",
}

Acknowledgement

This research is supported by MURI Grant N000141010934. The authors would like to thank Varsha Hedau and David Lee for providing their results on the indoor scene dataset.

Useful Links

Interesting !

Scott demonstrating affordances of shelves in Martial's Office !