Zoombie Dataset

(Hands under varying illumination)

Masks

Videos

Code
Raw C++ code
Simplified Version of CVPR 2013 paper (Per-Pixel Regression)
Labeling Tool (GrabCut Labeling Tool)

Updated Code from the CMU Art Fab:
https://github.com/cmuartfab/grabcut
https://github.com/cmuartfab/handtrack

Citation
Cheng Li and Kris M. Kitani.
Model Recommendation with Virtual Probes for Ego-Centric Hand Detection.
International Conference on Computer Vision (ICCV 2013). Dec 2013.

Cheng Li and Kris M. Kitani.
Pixel-level Hand Detection for Ego-centric Videos.
Conference on Computer Vision and Pattern Recognition (CVPR 2013). Jun 2013.

Activity Forecasting Dataset

Optimal Control (OC) Demo:
Main function main.cpp
Optimal control class header oc.hpp
Optimal control class methods oc.cpp
Input files (image, feature maps, reward weights and terminal points) oc_demo.zip

Inverse Optimal Control (IOC) Demo:
Main function main.cpp
Inverse optimal control class header ioc.hpp
Inverse optimal control class methods ioc.cpp
Input files (basenames, images, features, trajectories, sample output) ioc_demo.zip

Trajectories:
Observed tracker trajectories and ground truth trajectories tracker_output.zip

Images:
Rectified (top-down) images topdown_images.zip

Features:
Feature maps feature_maps.zip

Misc:
Learned reward weights reward_weights.zip
Precomputed reward function for forecasting reward_function_forecasting.zip
Precomputed empirical feature counts empirical_feature_counts.zip
List of base names basenames.zip
General explanation of data content README.txt

Citation
Kris M. Kitani, Brian Ziebart, James A. Bagnell and Martial Hebert.
Activity Forecasting.
European Conference on Computer Vision (ECCV 2012), Oct 2012.

EgoAction Dataset

UEC Dataset (Choreographed Videos)

Quad sequence
QUAD.MP4.zip (254 MB)

Park sequence
PARK.MP4.zip (1.56 GB)

YouTube Dataset (Non-choreographed Videos)

Mountain biking
Snowboarding
Slope style
Horseback riding
Skiing
Longboard Surfing

Ground Truth
Video segment labels

Code
Class for online inference with Dirichlet Process Mixture Models onlineDP.zip
Class wrapper for the OpenCV sparse optical flow optflow.zip
Class for computing the motion histogram given a set of point correspondences mohist.zip

Citation Kris M. Kitani, Takahiro Okabe, Yoichi Sato, and Akihiro Sugimoto.
Fast Unsupervised Ego-Action Learning for First-person Sports Videos.
Conference on Computer Vision and Pattern Recognition (CVPR2011), June 2011.

*FAQs

1. Do results depend on my image size?
   Yes, the flow magnitude and flow variance bin thresholds are optimized for WVGA (840 x 480)

2. Does optflow require any libraries?
   Yes, you need to have OpenCV. The code should work with version 2.0 and above.

3. Do I need to calibrate for lens distortion?
   No, parameters are actually optimized for the GoPro lens with radial distortion.