Sergei Mikhailovich Prokudin-Gorskii (1863-1944) was a man well ahead of his time. He took color photographs of everything he saw:  people, buildings, landscapes, railroads, bridges…thousands of color pictures! There was no way to print color photographs until much later.  Luckily, his RGB glass plate negatives, capturing the last years of the Russian Empire, survived and were purchased in 1948 by the Library of Congress.  The LoC has recently digitized the negatives and made them available on-line.

The goal of this project is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.  In order to do this, we need to extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.   We will assume that a simple x,y translation model is sufficient for proper alignment.  However, the full-size glass plate images are very large, so coarse-to-fine pyramid speedup is applied to large images. Attempts to recover the bad edge of the picture are also carried out.

Color Rendering Process

Fitst, I create Red(R), Green(G) and B(Blue) layers from the tri-part plate forming three separate “layers” from which the final color composite will be generated.

Second, to align the three parts, I found that searching over a window of possible displacements (say [-15,15] pixels or even larger window) directly using the original layers did not produce good results. Partly it's because of  the average  brightness of three layers may be very different. For example, if the R layer is averagely very bright and B layer is averagely very dark and I took SSD distance as a metric, R layer may match the brightest part of B layer rather than the correct part wanted. So normalizing three layers before aligning can make results much better. I found that histeq is an effective pre-processing function to normalize three layers.

What I do is first adjust the histogram of R, G, B layers so that the grey scale distribution are uniform in three layers.

R=histeq(R);

G=histeq(G);

B=histeq(B);

Then, I use the Sum of Absolute Difference (SAD) distance as a metric to score how well the images match.

SAD = sum(sum(abs(image1-image2)));

Of course, other metrics may also be applied, such as the Sum of Squared Differences (SSD) distance which is simply sum(sum((image1-image2).^2)) or normalized cross-correlation (NCC), which is simply a dot product between two normalized vectors: (image1./||image1|| and image2./||image2||.  I found that SAD works quite good, so I use this metric in the program.

Third, for high-resolution glass plates, exhaustive search will become prohibitively expensive if the pixel displacement is too large.  In this case, I implemented a faster search procedure - an image pyramid.  The image pyramid in our program is composed of images scaled by factors of 2,  2², 2³,..., 2¹⁰, from the original resolution to the coarsest resolution. Exhaustive search over a window of displacements [-16,16] pixels is applied only to the coarsest image. Then, for each picture on the lower level of the pyramid, it searched  over a window of displacements [-1,1], with the displacement obtained from the previous level picture as the center point. Note that applying a window of displacements [-16,16] pixels to a too small picture may result in unexpected low accuracy, so I imposed a  restriction that  height and width of the coarsest R G B images aligned must be greater than or equal to 50. Any coarser images under  this threshold in the pyramid will be over skipped.  The results show it can handle large TIFF images in a short time, providing aligned pictures of pretty good effects.

Fourth, RGB (Red, Green, Blue) color composite is cropped to eliminate all but the photographic area shared in common by all three layers.  To gain a better visual effect, the composite is also adjusted by applying imadjust function before they are put together to create the proper contrast, appropriate highlight and shadow detail, and optimal color balance.

At last, attempts to amend the edges of the recovered color picture are made for a specific image 00153v.jpg - The Emir of Bukhara, 1911. As the edges of the picture is defected with over or underexposure, development, or aging of the emulsion of Prokudin-Gorskii’s original glass plate, adjustments may be applied to specific, localized areas of the composite color image to minimize those  defects. Here, I tried to recover the left edge of the picture (the lanai) from the good part in the picture. I cut one slim strip (one column pixels) from the picture that contains good part of the lanai, then stretch it and paste it on the left. The process is repeated until the bad parts on the left edge is completely fulfilled. The stretching factor is chosen based on the overall visual effects. In the following results section, I show the image with recovered lanai on the left edge.

result-00125v(1).jpg