16-823: Physics based Methods in Vision, Spring 2011
General Information
Time :
Tuesdays and Thursdays, 1:30 pm -- 2:50 pm
Location: GHC 4211
Credits : 12
Pre-requisites :
Any undergraduate or graduate Vision, Graphics, or Image processing or
equivalent course
Announcements
Instructor
Srinivasa Narasimhan
http://www.cs.cmu.edu/~srinivas
Email:
srinivas@cs.cmu.edu
Office: Smith Hall 223
Office Hours: By appointment
Overview
Everyday we observe an extraordinary array of light and color phenomena
around us, ranging from the dazzling effects of the atmosphere, the
complex appearances of surfaces and materials and underwater scenarios.
For a long time, artists, scientists and photographers have been
fascinated by these effects, and have focused their attention on capturing
and understanding these phenomena. In this course, we take a computational
approach to modeling and analyzing these phenomena, which we collectively
call as "visual appearance". The first half of the course focuses on the
physical fundamentals of visual appearance, while the second half of the
course focuses on algorithms and applications in a variety of fields such
as computer vision, graphics and remote sensing and technologies such as
underwater and aerial imaging.
This course unifies concepts usually learnt in physical sciences and their
application in imaging sciences. The course will also include a
photography competition in addition to analytical and practical
assignments.
List of Topics
- Fundamentals of Appearance
- Principles of Photometry
- Light Fields
- Reflection, Refraction, Polarization, Diffraction, Interference
- Surface Reflection Mechanisms
- Signal Processing framework for Reflection
- Textures and Spatially Varying BRDFs (BTF)
- Lighting and Shadows
- Interreflections
- Caustics
- Scattering and Volumetric Light Transport
- Fluids
- Algorithms and Applications
- Photometric 'Shape-from-X' algorithms
- Image and Vision-based Rendering
- Inverse Rendering
- Understanding and measuring light transport
- Appearances of Transparent, Transluscent, Wet, Woven surfaces
- Appearances of Atmospheric and Underwater scattering effects
- Appearances of Fluids - smoke, fire, water
- Vision in Bad Weather
- Applications in Aerial, Underwater, Medical and Microscopic Imaging
- Principles of Nature Photography
Optional Texts
- Light and Color in the
Outdoors, M. Minnaert.
Grading
- One Project or Field Review paper 30%
- One Take-home Exam 25%
- Photography competition 15%
- Two Paper Presentations 30%
Lecture Presentations
[Acknowledgements]
A significant part of this course is similar to the courses offered at
Stanford (Pat Hanrahan, Marc Levoy, Ron Fediw), UC San Diego (Henrik Wann
Jensen), Columbia (Shree Nayar, Peter Belhumeur, Ravi Ramamoorthi), UW
Madison (Chuck Dyer), UWash (Steve Seitz), Utah (Pete Shirley), Rutgers
(Kristin Dana), Cornell (Steve Marschner, Kavita Bala), Technion (Yoav
Schechner), Princeton (Szymon Rusinkiewicz), MIT (Ted Adelson), Drexel (Ko
Nishino), TU Berlin and Deutsch Telecom (Rahul Swaminathan) The instructor
thanks the instructors of these courses for the materials (slides,
content) used in this course. In addition, several photographs and
illustrations are borrowed from internet sources. The instructor thanks
them all.
[Permission to use/modify materials]
The instructor gladly gives permission to use and modify any of the
slides for academic and research purposes. Since a lot of the material is
borrowed from other sources, please acknowledge the original sources too.
Finally, since this is a continuously evolving course, all suggestions
and corrections (major, minor) are welcome!
WEEK 1: INTRODUCTION
- Lecture 1: Introduction + Course Administration
[PPT]
- Lecture 2: Basic Principles of Imaging and Photometry
[PPT]
WEEKS 2,3,4: SURFACE REFLECTANCE
- Lecture 3: Basic Principles of Surface Reflectance
[PPT]
- Lecture 4: BRDF Models for Rough Specular Surfaces
[PPT]
- Lecture 5: BRDF Models for Rough Diffuse Surfaces
[PPT]
- Lecture 6: BRDF Measurements
[PPT]
- Lecture 7: Signal Processing Framework for Surface Reflection in 2D
[PPT]
(See Prof. Ramamoorthi's homepage .)
- Lecture 8: Signal Processing Framework for Surface Reflection in 3D
[PPT]
WEEK 5: STUDENT PRESENTATIONS
WEEKS 6,7: LIGHTING, SHADOWS AND INTERREFLECTIONS
- Lecture 9: Lighting and Shadows I
[PPT]
- Lecture 10: Lighting and Shadows II
[PPT]
- Lecture 11: Interreflections I
[PPT]
- Lecture 12: Interreflections II
[PPT]
(Online PPTs of
Steve Seitz
and Marc Levoy).
WEEK 8: STUDENT PRESENTATIONS
WEEK 9: REFLECTION AND REFRACTION
- Lecture 15: Basic Principles of Reflection, Refraction and Caustics
[PPT]
- Lecture 16: Caustics in Imaging and Rendering
[PPT]
(Thanks to PPTs from Ko Nishino and Rahul Swaminathan)
WEEK 10: LIGHT POLARIZATION
- Lecture 17: Basic Principles of Light Polarization
[PPT]
(Thanks to PPTs from Yoav Schechner)
- Lecture 18: Applications of Light Polarization in Computer Vision
[PPT]
(Thanks to PPTs from Yoav Schechner)
WEEK 11, 12: LIGHT SCATTERING
- Lecture 19: Basic Principles of Light Scattering
[PPT]
- Lecture 20: Volumetric Light Scattering in Computer Vision
[PPT]
- Lecture 21: Volumetric Light Scattering in Computer Graphics
[PPT]
WEEK 13: FLUIDS: SMOKE, FIRE AND WATER
- Lecture 22: Basic Principles of Modeling Fluids
[PPT]
WEEK 14: STUDENT PRESENTATIONS
Relevant Papers
-
A Coaxial Optical Scanner for Synchronous Acquisition of 3D Geometry and Surface Reflectance
-
Principles of Appearance Acquisition and Representation.
-
A Perception-based Color Space for Illumination-invariant Image Processing.
-
Color Subspaces as Photometric Invariants
-
Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction
-
Light Field Transfer: Global Illumination Between Real and Synthetic Objects
-
Projection Defocus Analysis for Scene Capture and Image Display
-
Time-varying Surface Appearance: Acquisition, Modeling, and Rendering
-
Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination
-
Acquiring Scattering Properties of Participating Media by Dilution
-
Structured Light in Scattering Media
-
Polarization-Based Vision through Haze
-
Vision in Bad Weather
-
Generalization of the Lambertian Model and Implications for Machine Vision
-
Shape from Interreflections
-
Surface Reflection: Physical and Geometrical Perspectives
-
Light Field Microscopy
-
Dual Photography
-
Synthetic aperture confocal imaging
-
Light Field Rendering
-
A Dual Theory of Inverse and Forward Light Transport
-
Compressive Light Transport Sensing
-
Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object
-
A Signal-Processing Framework for Inverse Rendering
-
Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs
-
Optical Computing for Fast Light Transport Analysis
-
A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation
-
State of the Art in Transparent and Specular Object Reconstruction
-
A Theory of Inverse Light Transport
-
From Few to Many: Illumination Cone Models for Face Recognition under Variable lighting and Pose
-
The Bas-Relief Ambiguity
-
What is the Set of Images of an Object Under All Possible Lighting Conditions?
-
Fluorescent Immersion Range Scanning
-
DISCO - Acquisition of Translucent Objects.
-
Clear underwater vision
-
Separation of transparent layers using focus
-
A Multi-layered Display with Water Drops
-
Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequences
-
What Do the Sun and Sky Tell Us About the Camera?
-
Estimating Natural Illumination from a Single Outdoor Image
-
De-(Focusing) on Global Light Transport for Active Scene Recovery
-
Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-flash Imaging