William Wedler
15-463 Fall 2007
Sergei Mikhailovich Prokudin-Gorskii (1863-1944) was a man ahead of his time. He conceived a method for recording color images before color photography. His method was simple, but practical. He took photos of various subjects across the Russian Empire. Each subject was photographed three times. Once with a Blue filter, once with a Green filter and finally once with a Red filter. These three photographs made up a set that was placed onto plate glass. His intention was to project each image on top of each other using Blue, Green and Red light to combine them into a color photograph.
Gorskii did not finish his project due to political turmoil, but many of the plates remain. The collection was purchased by the library of congress and is available through the internet. In this project, I will use several photograph sets of black and white Gorskii photos to generate color photos of Russia.
Images from the collection were made available. The images included low resolution JPEG images and also very high resolution TIFF images. The high resolution posed a challenge and the code was modified to process the large images. Image processing was done using MATLAB.
An example of an input image is given below:
The image was read in and divided into three equal sections based on the image height. The first section was stored as the Blue channel, the second section was stored as the Green channel and the third section was stored as the Red channel. This is the order that the photos were taken by Gorskii.
An image in MATLAB is simply a 3D array of color data that maps RGB values to pixels in the image. Creating a color image from the three channel data sets simply involves concatenation of the channels into a single 3D array. This can then be displayed on the screen and saved to the disk.
However, each photograph in the set is not identical. The channels must be aligned before being combined or the result will not be a sharp colored image. A significant portion of the colorizing script involved aligning the channels.
The channel alignment portion of the script went through a few iterations. The iterations were:
The first attempt did not result in good alignment. The channels were nearly aligned, but even small examples were off by several pixels.
The second attempt solved the quality of the alignment. However, it was only practical for the low resolution images, since calculating the NCC between large matrices becomes very time consuming.
The final attempt used NCC to get a quality alignment and also an image pyramid so that the NCC can be calculated on a small portion of the image, yet still result in good alignment.
Below are 3 figures that show the NCC step at varying image scales. The left image is taken from the Red channel and it is aligned with the right image taken from the Blue channel.
The 3D plots and the contour plots show the computed Normalized Cross Correlation between the Red channel and Blue channel samples. The Red channel sample is smaller than the Blue sample, since this was found to give better results. The reason for better results, is that maximizing the NCC works best when all points in the Red channel are also in the Blue channel. By extracting a small sample from the Red channel and then a large sample from the Blue channel, both from near the center of the image, it is likely that all of the pixels in the Red channel will be able to match some of the pixels in the Blue channel. When the samples for each channel were the same size, some of the larger images were not aligned successfully.
The Figures below are ordered from coarse to fine. The first one shows samples from the most scaled down images and the scale increases with each figure. The images appear blurry because they were forced to a uniform square size across the level.
One reason why the images may not align would be if there is little variation on the sampled section of the images. This could be if the sampled section is from clear sky, for example. Another scenario that has given my script trouble is when one sample does not contain the other and so some pixels do not translate.
One of the Bells and Whistles that I implemented was an automatic color level adjustment. I noticed in the input images that parts of the plate had a faded out look appearance. That extra white may have been higher than wanted values for some of the channels that should have been. To help remove some of the extra values, I added/subtracted a constant amount from each pixel from channels that appear to be off.
Part of the automatic process was determining what channels need to be modified if any. I computed the overall average value of each channel separately, and then compared those averages to the overall average across each channel. If the difference was above/below a certain threshold, the channel was adjusted proportional to the difference.
Since the channels are shifted relative to each other, ugly color bands appear around the edges of the colorized images. To remove those bands, I used the shift information to crop off the extra pixels around the borders. This is a simple edit and it is only based on the maximum amounts that the channels are shifted in each direction.
These images were read in and processed as TIFF images. They were later scaled down and compressed into JPEG's using Photoshop for presentation on this web page.
The top image is the original result from simply combining the channel data.
The bottom image shows the result after adjusting the channels relative to the average value across all channels.
Inspecting the input image for this photo motivated me to experiment with adjusting the channel data.
The Red channel seemed to overall have high values relative to the other channels.
The adjusted image has lowered red channels, which brings out the Blue and Green shades more.
This image was shifted to the right and down. The cropped image is to the right.
This image was shifted to the left and up. The cropped image is to the right.