AI Seminar sponsored by Fortive

Tali Dekel -- Learning to Retime People in Videos

Abstract: By changing the speed of frames, or the speed of objects, we can enhance the way we perceive events or actions in videos. In this talk, I will present two of my recent works on retiming videos, and more specifically, manipulating the timings of people's actions. 1) "SpeedNet" (CVPR 2020 oral): a method for adaptively speeding up videos based on their content, allowing us to gracefully watch videos faster while avoiding jerky and unnatural motions. 2) "Layered Neural Rendering for Retiming People" (SIGGRAPH Asia): a method for speeding up, slowing down, or entirely freezing certain people in videos, while automatically re-rendering properly all the scene elements that are related to those people, like shadows, reflections, and loose clothing. Both methods are based on novel deep neural networks that learn concepts of natural motion and scene decomposition just by observing ordinary videos, without requiring any manual labels. I'll show adaptively sped-up videos of sports, of boring family events (that all of us want to watch faster), and I'll demonstrate various retiming effects of people dancing, groups running, and kids jumping on trampolines.

Bio: Tali Dekel is a Senior Research Scientist at Google, Cambridge MA, developing algorithms at the intersection of computer vision, computer graphics, and machine learning. She also just joined the Mathematics and Computer Science Department at the Weizmann Institute, Israel, as a faculty member (Assistant Professor). Before Google, she was a Postdoctoral Associate at the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT. Tali completed her Ph.D. studies at the school of electrical engineering, Tel-Aviv University, Israel. Her research interests include computational photography, image/video synthesis, geometry and 3D reconstruction. Her awards and honors include the National Postdoctoral Award for Advancing Women in Science (2014), the Rothschild Postdoctoral Fellowship (2015), the SAMSON - Prime Minster's Researcher Recruitment Prize (2019), Best Paper Honorable Mention in CVPR 2019, and Best Paper Award (Marr Prize) in ICCV 2019. She served as the workshop co-chair for CVPR 2020.