Human Motion Transfer

German Cheung, Simon Baker, Takeo Kanade

Please contact German Cheung at german@ux2.sp.cs.cmu.edu for further details

Return to Homepage RealTime 3D Reconstruction Temporal SFS Human Kinematic Modeling and Motion Capture Human Motion Transfer

Objective

This project propose a system to transfer human motion through videos. Consider the following scenario: in a room equipped with multiple cameras, we record videos of two people doing (separately) two different motions. Using the recorded videos of these two people, we produce videos of each person doing the other person's motion. There are three steps to this human motion transfer system: (1) kinematic modeling (i.e. to build 3D model of each person with precise shape and accurate joint locations), (2) motion tracking (i.e. track the motion of each person from the recorded video) and finally (3) motion rendering (i.e. render images of the person performing new motion using color information from the recorded video). Details for the first two steps can be found at my other project page:3D Human Kinematic Modeling and Motion Capture. Technical details of the rendering step can be found in the documents listed below:

Problem Scenario

Rendering Algorithm

Results

KUNGFU motion transferred from SubjectG to SubjectS
Video 		THROW motion transferred from SubjectS to SubjectE
Video 		STEP-FLEX motion transferred from SubjectE to SubjectG
Video