Overview
Everyday we observe an extraordinary array of light and color phenomena
around us, ranging from the dazzling effects of the atmosphere, the
complex appearances of surfaces and materials and underwater scenarios.
For a long time, artists, scientists and photographers have been
fascinated by these effects, and have focused their attention on capturing
and understanding these phenomena. In this course, we take a computational
approach to modeling and analyzing these phenomena, which we collectively
call as "visual appearance". The first half of the course focuses on the
physical fundamentals of visual appearance, while the second half of the
course focuses on algorithms and applications in a variety of fields such
as computer vision, graphics and remote sensing and technologies such as
underwater and aerial imaging.
This course unifies concepts usually learnt in physical sciences and their
application in imaging sciences. The course will also include a
photography competition in addition to analytical and practical
assignments.
List of Topics
- Fundamentals of Appearance
- Principles of Photometry
- Light Fields
- Reflection, Refraction, Polarization, Diffraction, Interference
- Surface Reflection Mechanisms
- Signal Processing framework for Reflection
- Textures and Spatially Varying BRDFs (BTF)
- Lighting and Shadows
- Light Transport
- Caustics
- Scattering and Volumetric Light Transport
- Fluids
- Algorithms and Applications
- Photometric 'Shape-from-X' algorithms
- Image and Vision-based Rendering
- Inverse Rendering
- Understanding and Measuring Light Transport
- Appearances of Transparent, Transluscent, Wet, Woven Surfaces
- Appearances of Atmospheric and Underwater Scattering Effects
- Appearances of Fluids - smoke, fire, water
- Vision in Bad Weather
- Applications in Aerial, Underwater, Medical and Microscopic Imaging
- Principles of Nature Photography
Optional Texts
- Light and Color in the
Outdoors, M. Minnaert.
Grading
- One Project 60%
- Two Paper Presentations 30%
- Photography competition 10%
Lecture Presentations
[Acknowledgements]
A significant part of this course is similar to the courses offered at
Stanford (Pat Hanrahan, Marc Levoy, Ron Fediw), UC San Diego (Henrik Wann
Jensen), Columbia (Shree Nayar, Peter Belhumeur, Ravi Ramamoorthi), UW
Madison (Chuck Dyer), UWash (Steve Seitz), Utah (Pete Shirley), Rutgers
(Kristin Dana), Cornell (Steve Marschner, Kavita Bala), Technion (Yoav
Schechner), Princeton (Szymon Rusinkiewicz), MIT (Ted Adelson), Drexel (Ko
Nishino), TU Berlin and Deutsch Telecom (Rahul Swaminathan) The instructor
thanks the instructors of these courses for the materials (slides,
content) used in this course. In addition, several photographs and
illustrations are borrowed from internet sources. The instructor thanks
them all.
[Permission to use/modify materials]
The instructor gladly gives permission to use and modify any of the
slides for academic and research purposes. Since a lot of the material is
borrowed from other sources, please acknowledge the original sources too.
Finally, since this is a continuously evolving course, all suggestions
and corrections (major, minor) are welcome!
WEEK 1: INTRODUCTION
- Lecture 1: Introduction + Course Administration
[PPT]
[PDF]
- Lecture 2: Basic Principles of Imaging and Photometry
[PPT]
[PDF]
WEEK 2: BRDF MODELS AND MEASUREMENTS
- Lecture 3:
Basic Principles of Surface Reflectance
[PPT]
[PDF]
BRDF Models for Rough Specular Surfaces
[PPT]
[PDF]
- Lecture 4:
BRDF Models for Rough Diffuse Surfaces
[PPT]
[PDF]
WEEK 3: SIGNAL PROCESSING AND LIGHTING/SHADOWS
- Lecture 5: Signal Processing Framework for Surface Reflection in 2D and 3D [PPT]
[PDF] 
(See Prof. Ramamoorthi's homepage .)
- Lecture 6: Lighting and Shadows
[PPT1]
[PDF] 
[PPT2]
[PDF]
WEEK 4: STUDENT PRESENTATION I
Papers link
- Paper 1: Microgeometry Capture using an Elastomeric Sensor, Siggraph 2011
[PPT]
[PDF]
- Paper 2: A Dictionary-based Approach for Estimating Shape and Spatially-Varying Reflectance, ICCP 2015
[PPT]
[PDF]
- Paper 3: A New Perspective on Material Classification and Ink Identification, CVPR 2014
[PPT]
[PDF]
WEEK 5: STUDENT PRESENTATION II
Papers link
- Paper 1: What is the Set of Images of an Object Under All Possible Lighting Conditions, IJCV 1998
[PDF]
- Paper 2: Estimating Natural Illumination from a Single Outdoor Image, ICCV 2009
[KEYNOTE]
[PDF]
- Paper 3: Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequence, TOG 2009
[PDF]
- Paper 4: Light field rendering, TOG 1996
[PDF]
- Paper 5: Light field photography with a hand-held plenoptic camera, 2005
[PDF]
WEEK 6: LIGHT TRANSPORT I AND II
- Paper 6: Light Field Transfer: Global Illumination Between Real and Synthetic Objects, TOG 2008
[PPT]
[PDF]
- Paper 7: Compressive Light Field Photography, TOG 2013
[PPT]
[PDF]
- Lecture 7: Light transport I
[PPT]
- Lecture 8: Light transport II
[PPT]
WEEK 7: STUDENT PRESENTATION III
Papers link
- Paper 1: Dual Photography, Siggraph 2005
[PDF]
- Paper 2: Fast Separation of Direct and Global Components, Siggraph 2006
[PPTX]
[PDF]
- Paper 3: A Combined Theory of Defocused Illumination and Global Light Transport, IJCV 2012
[PPTX]
[PDF]
- Paper 4: Optical Computing for Fast Light Transport Analysis, Siggraph Asia 2010
[PPTX]
[PDF]
- Paper 5: A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus, IJCV 2012
[PPTX]
[PDF]
- Paper 6: Programmable Automotive Headlights, ECCV 2014
[PDF]
WEEK 8: LIGHT TRANSPORT III AND IV
- Lecture 9: Light transport III
[PPT]
- Lecture 10: Light transport IV
[PPT]
WEEK 9: SPRING BREAK
WEEK 10: STUDENT PRESENTATION IV
Papers link
- Paper 1: 3D Shape and Indirect Appearance by Structured Light Transport, CVPR 2014
[PPTX]
[PDF]
- Paper 2: Homogeneous Codes for Energy-Efficient Illumination and Imaging, Siggraph 2015
[PPTX]
[PDF]
- Paper 3: Recovering 3D Shape around a Corner using Ultra-Fast Time-of-Flight Imaging, Nature Communations 2012
[PPTX]
[PDF]
WEEK 11: REFLECTION AND REFRACTION
- Lecture 11: Basic Principles of Reflection, Refraction and Caustics
[PPT]
[PDF]
- Lecture 12: Caustics in Imaging and Rendering
[PPT]
[PDF]
(Thanks to PPTs from Ko Nishino and Rahul Swaminathan)
WEEK 12: STUDENT PRESENTATION V
Papers link
- Paper 1: A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation, IJCV 2008
[PPTX]
[PDF]
- Paper 2: What a single light ray reveals about a transparent object? ICIP 2015
[PDF]
- Paper 3: Seeing through Water: Image Restoration using Model-based Tracking, ICCV 2009
[PPTX]
[PDF]
WEEK 13: LIGHT POLARIZATION
- Lecture 13: Basic Principles of Light Polarization
[PPT]
[PDF]
(Thanks to PPTs from Yoav Schechner)
- Lecture 14: Applications of Light Polarization in Computer Vision
[PPT]
[PDF]
(Thanks to PPTs from Yoav Schechner)
WEEK 14: LIGHT SCATTERING (Apr.14 no class)
- Lecture 15: Basic Principles of Light Scattering
[PPT]
[PDF]
- Lecture 16: Volumetric Light Scattering in Computer Vision
[PPT]
[PDF]
and Computer Graphics
[PPT]
[PDF]
WEEK 15: STUDENT PRESENTATION VI
Papers link
WEEK 16: FINAL PROJECTS PRESENTATION
Apr. 29 12:00pm~5:00pm, project presentation
Relevant Papers
Presentation I: photometry, BRDF
- Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs, 2005
- Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction, 2002
- Specularity Removal in Images and Videos: A PDE Approach, 2006
- Color Subspaces as Photometric Invariants, 2006
- Projection Defocus Analysis for Scene Capture and Image Display, 2006
- A Coaxial Optical Scanner for Synchronous Acquisition of 3D Geometry and Surface Reflectance, 2010
- Principles of Appearance Acquisition and Representation, 2009
- Time-varying Surface Appearance: Acquisition, Modeling, and Rendering, 2006
- Generalization of the Lambertian Model and Implications for Machine Vision, 1994
- Surface Reflection: Physical and Geometrical Perspectives, 1989
- A Perception-based Color Space for Illumination-invariant Image Processing, 2008
- Microgeometry Capture using an Elastomeric Sensor, 2011
- A New Perspective on Material Classification and Ink Identification, 2014
- A Dictionary-based Approach for Estimating Shape and Spatially-Varying Reflectance, 2015
- Printing Spatially-Varying Reflectance for Reproducing HDR Images, 2012
- Fabricating BRDFs at High Spatial Resolution Using Wave Optics, 2013
- Light-Efficient Photography, 2011
- A Digital Gigapixel Large-Format Tile-Scan Camera, 2011
Presentation II: signal processing, light field, lighting, shadows
- A Signal-Processing Framework for Inverse Rendering, 2001
- Estimating Natural Illumination from a Single Outdoor Image, 2011
- Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-flash Imaging, 2004
- Light Field Transfer: Global Illumination Between Real and Synthetic Objects, 2008
- Light Field Microscopy, 2006
- Synthetic aperture confocal imaging, 2004
- Light Field Rendering, 1996
- Light Field Analysis for Modeling Image Formation, 2011
- Light field photography with a hand-held plenoptic camera, 2005
- Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing, 2007
- Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object, 2002
- From Few to Many: Illumination Cone Models for Face Recognition under Variable lighting and Pose, 2001
- Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequences, 2009
- The Bas-Relief Ambiguity, 1999
- What is the Set of Images of an Object Under All Possible Lighting Conditions?, 1998
- Light Fall-off Stereo, 2007
- Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects, 2007
- Compressive Light Field Photography, 2013
Presentation III: light transport
- Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination, 2006
- Shape from Interreflections, 1990
- Dual Photography, 2005
- Optical Computing for Fast Light Transport Analysis, 2010
- A Theory of Inverse Light Transport, 2005
- A Combined Theory of Defocused Illumination and Global Light Transport, 2011
- Shape from Second-bounce of Light Transport, 2010
- Programmable Automotive Headlights, 2014
- A Practical Approach to 3D Scanning in the Presence of Interre.ections, Subsurface Scattering and Defocus, 2012
- Compressive Light Transport Sensing, 2009
Presentation IV: light transport
- Homogeneous Codes for Energy-Efficient Illumination and Imaging, 2015
- Micron-scale Light Transport Decomposition Using Interferometry, 2015
- Interreflection Removal Using Fluorescence, 2014
- A Dual Theory of Inverse and Forward Light Transport, 2010
- Recovering ThreeDimensional Shape around a Corner using Ultra-Fast Time-of-Flight Imaging, 2012
- A Practical Analytic Model for the Radiosity of Translucent Scenes, 2013
- Frequency-Space Decomposition and Acquisition of Light Transport under Spatially Varying Illumination, 20112
- Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport, 2014
- Primal-Dual Coding to Probe Light Transport, 2012
- Frequency Analysis of Transient Light Transport with Applications in Bare Sensor Imaging, 2012
- 3D Shape and Indirect Appearance by Structured Light Transport, 2014
Presentation V: (mirror) reflection and refraction
- A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation, 2008
- State of the Art in Transparent and Specular Object Reconstruction, 2008
- Fluorescent Immersion Range Scanning, 2008
- A Multi-layered Display with Water Drops, 2010
- Reflection Removal using Ghosting Cues, 2015
- What a single light ray reveals about a transparent object? 2015
- Depth from Optical Turbulence, 2012
- STELLA MARIS: Stellar Marine Refractive Imaging Sensor, 2014
- Triangulation in Random Refractive Distortions, 2013
- Refraction wiggles for measuring fluid depth and velocity from video, 2014
- Image invariants for smooth reflective surfaces, 2010
- Specular surface reconstruction from sparse reflection correspondences, 2012
- Angular Domain Reconstruction of Dynamic 3D Fluid Surfaces, 2012
- Seeing through Water: Image Restoration using Model-based Tracking, 2009
Presentation VI: polarization, scattering
- Polarization-Based Vision through Haze, 2003
- Separation of transparent layers using focus, 2000
- Acquiring Scattering Properties of Participating Media by Dilution, 2006
- Structured Light in Scattering Media, 2005
- Vision in Bad Weather, 1999
- DISCO - Acquisition of Translucent Objects, 2004
- Clear underwater vision, 2004
- What Do the Sun and Sky Tell Us About the Camera? 2010
- 3Deflicker from Motion, 2013
- Self-Calibrating Imaging Polarimetry, 2015
- Polarized 3D: High-Quality Depth Sensing With Polarization Cues, 2015
- Single image haze removal using dark channel prior, 2009
- On the Appearance of Translucent Edges, 2015
- Airborne Three-Dimensional Cloud Tomography, 2015
- Recovering Inner Slices of Translucent Objects by Multi-frequency Illumination, 2015
- Surface Normal Deconvolution: Photometric Stereo for Optically Thick Translucent Objects, 2014
- Shape from Single Scattering for Translucent Objects, 2012
- Compressive Structured Light for Recovering Inhomogeneous Participating Media, 2008