15-463 Computational Photography Final Project

Fun with High Dynamic Range (HDR)

 

Barry (BoB) Hon

 

Premise

 

This project explores different high dynamic range research.  Some topics covered include hdr generation, and hdr compressing.  HDR imaging is attempting to display large high dynamic range images on limited devices with a limited dynamic range.  Current display technology (LCDs, CRTs, paper) can only display a limited range of the real world.  For example, in photographs only a limited range can be mapped to paper.  Anything higher than the range is mapped to white and lower to black.

 

For example the below are two photographs.  The first has a long exposure (2 seconds) and captures the dark room.  The send has a short exposure (16 seconds) and captures the hallway outside the door.  Using normal capture and camera compression methods there is no way to capture both the inside and outside.

 

 

Recovering HDR: Radiance Maps

 

Capturing HDR can be done with a camera-photometer which can measure the incoming light or the radiance of an image/environment.  Some high-end cameras do this already and store it in the TIFF format.  Another way is to take several pictures of the same environment with different exposures to attempt to recover the radiance maps.  The technique described here is from Paul E. Devebec and Jitendra Malik’s Recovering High Dynamic Range Radiance Maps from Photographs.

 

First we capture several images of the scene with different exposures.  Exposures of 2, 1, ½, ¼, 1/8, 1/16. 1/32, 1/64 were captured.

 

 

 

 

 

 

 

 

 

 

 

 

 

We then select 100 random points in the graph to choose points of reference for recovering the HDR range using Devebec’s Matlab code, which returns an array of log exposures for a pixel value.  Once we have the log exposure we can calculate the radiance since:

 

 

Where E is irradiance, Z is a pixel value (ranging from 0 to 255) and  is the exposure time.  g(Z) returns the log exposure of a pixel with value Z.  Below is sample recovered g function.  With this function and the above Equation we can calculate the radiance for each pixel.

 

 

We can also retrieve obtain HDR images online.  The Munsell Color Science Laboratory has a database of HDR images (http://www.cis.rit.edu/mcsl/icam/hdr/ ).  These images have been saved in the .hdr format which is a format created by Greg Ward.  Links to other HDR images (including Devebec and Reinhard images) can also be found at Munsell.  Munsell also has Matlab code which will read in the hdr file.

 

Compressing HDR images

 

Once we have a radiance map we need to compress the image down to RGB.  If we are to just display the image as is we obtain this:

 

 

Which if we scaled the image would show us relative intensities like so:

 

 

For the most part everything is dark and a lot of information is lost.  The outside has a much greater intensity than the inside room.

 

What we can do is first try to linearly normalize the object by dividing the image by the most intense radiance which gives us this:

 

Since the outside room is a lot more intense than the inside room the information on the outside is completely lost.

 

Reinhart suggests to use a Global Operator in which the luminance we use the log-average-luminance as a useful approximation.  Log-average-luminance has the formula:

Where  is the luminance of a pixel (for our case it is equivalent to the radiance).  Once we have the Log-average- luminance we scale the  with the following form:

Where is a key value (set to 0.18 for our purposes).  Once we have a scaled luminance we can apply Reinhart’s time mapping operation to obtain the compress displayable image that ranges from [0-1]:

This leads to the following curve mapping of the image:

Which has very low luminance values increase very fast and then as we achieve higher luminance it curves downward to display higher levels of luminance.  The Reinhard final image like this:

 

 

For this particular photo Reinhard does not do that great but let’s look at other pictures:

 

The radiance image:

 

Linear Scaled:

 

Reinhart

 

A little something different: Subband Compression.

 

The idea behind subband compression is to break apart an image into subbands and compress the subbands individually.  The following images displays.  The original subband is put through a point nonlinearilty limits range compression.  This however leads to distortions such as halos (observable even in the Reinhart implementation.)

Note the Halo around the ball

 

  

 

 

Each Activity Map is calculated by taking the absolute value and blurring the subband:

 

The gain is then calculated by “compressing” the activity map:

 

Where  ranges from 0 to 1 which is the compressive factor.

 

Finally the compressed subband is calculated by multiplying the gain map by the original subband:

 

 

Some results:

 

 

Edges are preserved in the image and there are no halos but there is a problem with using Laplacian Pyramids with this algorithm.  Namely the compression works well on high frequencies (Edges are retained).  However it cannot remove low frequencies artifacts (problems in the residue image of the laplacian pyramid).

 

Some other examples:

 

With compression: y = 0.8

 

 

 

With compression: y = 0.8

 

With compression y=0.1

 

Note as you compress more, a lot more noise pops up.  I believe this is the problem with compressing algorithm used with Laplacian, although you can “see” more ranges with this compression.  The Li et al suggested that this algorithm of subband compression is more inline with wavelet subbands.

 

Reinhart

 

Li with high compression.

 

Some other fun

Reinhard focusing.

 

Global operators like Reinhard’s method calculates a global value and uses that global value as an “average” of a scene.  However, there are times when a large portion of an image is of one luminance and a small portion of a large luminance that will be “averaged” out.  Thus I created a small program that uses a local operator on a HDR image such that if the focus is on one portion of the image the “global operator” is calculated based on a locality specified by the user.  The locality is a box of 10% of the original image in which we calculate the log-average-luminance of that box.  We use Reinhard because it is a fast and simple method:

 

Original Image

 

Reinhart global:

 

Focus on outside (Note Diamond marks where there is focus)

 

Focus on Dark area inside

 

Note: this “focusing” is only possible if we have a radiance map.  Uses for this could be to detect what a person is focusing on and generate an image based on what the person is focusing on.

 

Some Cool Mistakes

 

Images that are obvious mistakes but are cool to look at:

Compared with Reinhart: