15-862 Computational Photography

Project 1: Images of the Russian Empire
 -- colorizing the Prokudin-Gorskii photo collection
by: Lisa Chan

Project Details:

The method used in aligning the images was the Sum of Squared Differences. Please note that the displacement values given correspond to positive when the image shifts to the right or down. In the Bells and Whistles section, auto-cropping, color balancing, and an attempt to reproduce Prokudin-Gorskii’s work was performed. Don’t forget to check out the 'Just For Fun' section too!


1. Single-Scale Alignment:

Almost all the images were perfectly aligned with the exception of 01728v.jpg. For that image, because the displacement was larger than 15, I had to expand my usual scan window from 15 to 20.

A. Assigned Images:
00017v.jpg
00056v.jpg
00084v.jpg
Displacements:
Green: x:-2 y:1
Red: x:-3 y:1
 Displacements:
Green: x:1 y:6
Red: x:1 y:13
 Displacements:
Green: x:-4 y:4
Red: x:-9 y:11

00362v.jpg
00498v.jpg
00646v.jpg
Displacements:
Green: x:0 y:5
Red: x:-5 y:11
 Displacements:
Green: x:3 y:4
Red: x:3 y:9
 Displacements:
Green: x:2 y:7
Red: x:4 y:15

00704v.jpg
00858v.jpg
00872v.jpg
Displacements:
Green: x:2 y:6
Red: x:3 y:13
 Displacements:
Green: x:3 y:6
Red: x:4 y:13
 Displacements:
Green: x:-1 y:-3
Red: x:-2 y:-2

01039v.jpg
01728v.jpg
31421v.jpg
Displacements:
Green: x:0 y:5
Red: x:1 y:11
 Displacements:
Green: x:1 y:8
Red: x:1 y:18
(expanded max displacement to 20)
 Displacements:
Green: x:0 y:8
Red: x:0 y:13

B. Chosen Images:
00185v.jpg
01489v.jpg
01352v.jpg
Displacements:
Green: x:0 y:6
Red: x:-1 y:12
 Displacements:
Green: x:0 y:5
Red: x:-1 y:11
 Displacements:
Green: x:0 y:6
Red: x:0 y:14


2. Multi-Scale Alignment:

In the multi-scale alignment of the high resolution images, the images 00153u.tif, 00797u.tif, and 01754u.tif did not properly align. In image 00153u.tif, there may have been some rotational displacement between the three colors, because as can be seen in the image, even though the person in blue is not aligned, the background is perfectly aligned. I have even tried only aligning the person’s head or the body, and the colors simply won’t perfectly match. As for 00797u.tif, and 01754u.tif, the edges of the pictures just had to be ignored to have the colors align correctly. As for the chosen images, all of them aligned perfectly with the original codes.

A. Assigned Images:
00033u.tif
00153u.tif
Displacements:
Green: x:12 y:52
Red: x:16 y:104
 Displacements:
Green: x:26 y:-60
Red: x:42 y:-128

00794u.tif
00797u.tif
Displacements:
Green: x:16 y:52
Red: x:16 y:120
 Displacements:
Green: x:16 y:64
Red: x:16 y:128

01443u.tif
01754u.tif
Displacements:
Green: x:20 y:32
Red: x:40 y:80
 Displacements:
Green: x:-8 y:36
Red: x:20 y:92

B. Chosen Images:
00115u.tif
01140u.tif
Displacements:
Green: x:0 y:52
Red: x:-24 y:112
 Displacements:
Green: x:-12 y:44
Red: x:-20 y:92

01500u.tif
Displacements:
Green: x:24 y:52
Red: x:32 y:120


3. Bells and Whistles:

A. Auto-Cropping of Images:

The method used to perform the auto-cropping of the images was with a brute force method. The program simply goes into each of the edges, isolates the row or column of pixels, and finds black borders and the average mismatch between the three color channels. The maximum allowable difference (maxdiff) between the three color channels had to changed from time to time to optimize the cropping. As seen from the results, most of the images are reasonably cropped without any optimization necessary.

As Aligned
Cropped

As Aligned
Cropped

00017v.jpg
00056v.jpg
00084v.jpg
Cropped:
Left:11 Top:20 Right:23 Bottom:25
 Cropped:
Left:20 Top:25 Right:10 Bottom:9
 Cropped:
Left:24 Top:9 Right:22 Bottom:25

00362v.jpg
00498v.jpg
00646v.jpg
Cropped:
Left:20 Top:25 Right:25 Bottom:11
(used maxdiff=0.08)
 Cropped:
Left:16 Top:16 Right:9 Bottom:22
(used maxdiff=0.28)
 Cropped:
Left:22 Top:16 Right:25 Bottom:11
(used maxdiff=0.18)

00704v.jpg
00858v.jpg
00872v.jpg
Cropped:
Left:25 Top:25 Right:25 Bottom:25
 Cropped:
Left:18 Top:17 Right:14 Bottom:12
 Cropped:
Left:10 Top:12 Right:9 Bottom:25
(used maxdiff=0.14)

01039v.jpg
01728v.jpg
31421v.jpg
Cropped:
Left:24 Top:21 Right:25 Bottom:25
(used maxdiff=0.1)
 Cropped:
Left:16 Top:23 Right:16 Bottom:0
 Cropped:
Left:15 Top:12 Right:25 Bottom:25
(used maxdiff=0.13)

B. Color Balance Based on Gray World Assumption:

The color balancing is performed based on the Gray World Assumption. The Gray World Assumption states that the average value of the red, green, and blue color channels should converge to a common gray value. Hence, the program simply calculates the average gray value of the image, and then scales each color channel according to the deviation from the gray value. As seen in the first set of images, the green tinge in the left image is removed, giving the boat a more realistic color. In the other sets of images, certain features of the images are enhanced with the color balancing.

As Aligned
Color Corrected

As Aligned
Color Corrected

As Aligned
Color Corrected

As Aligned
Color Corrected

C. Reproducing Sergei Mikhailovich Prokudin-Gorskii's Work:

In an attempt to reproduce Prokudin-Gorskii’s work, red, green, and blue cellophane sheets were put in front of the digital camera set at the ‘black and white’ option. The three color channels were compiled in Photoshop in the order of BGR so that the same single-scale alignment program can be used again here. The problem of using cellophane sheets was that the sheets are not completely translucent. The cellophane sheets actually blur the images. Also, as can be seen below, the green cellophane may have been too dark for the color images, giving it a green tinge on all the aligned images. However, by running the color balancing algorithm generated before, the image was enhanced.

Strip
As Aligned
Color Balanced
Displacements:
Green: x:0 y:1
Red: x:-2 y:0

Strip
As Aligned
Color Balanced
Displacements:
Green: x:-1 y:2
Red: x:2 y:-1


4. Just For Fun:

Just for fun, I wrote a program that scans the image and isolates one of the colors: red, green, or blue. The program filters out the high intensity points that exist in the chosen color channel, and then combines the high intensity points with the black and white version of the image. As can be seen below, the program doesn’t work that well when the colors that exist in the image are not exactly of that color channel, e.g. the green in the image is not exactly pure green.

Original Image

Red Isolation
Green Isolation
Blue Isolation