15-463 Computational Photography
"Images of the Russian Empire" - Sept. 12, 2007

Anna Michalska

Colorizing the Prokudin-Gorskii Photo Collection

Low Resolution - Exhaustive Search - Sum of Squared Differences

Image: '00228v.jpg'	Image: '00011v.jpg'

Displacement: Red: x = 5 y = 8 Green: x = 3 y = 1	Displacement: Red: x = 4 y = 8 Green: x = 2 y = 2

Image: '00385v.jpg'	Image: '00095v.jpg'

Displacement: Red: x = 0 y = 10 Green: x = 0 y = 4	Displacement: Red: x = 0 y = 14 Green: x = 0 y = 3

Image: '00234v.jpg'	Image: '00091v.jpg'

Displacement: Red: x = 1 y = 14 Green: x = 0 y = 4	Displacement: Red: x = 3 y = 15 Green: x = 2 y = 7

Image: '00206v.jpg'	Image: '00215v.jpg'

Displacement: Red: x = 1 y = 1 Green: x = 1 y = 6	Displacement: Red: x = -3 y = 9 Green: x = -2 y = 4

Image: '00032v.jpg'	Image: '00086v.jpg'

Displacement: Red: x = 1 y = 9 Green: x = 1 y = 4	Displacement: Red: x = 5 y = 13 Green: x = 3 y = 6

Image: '00498v.jpg''	Image: '00646v.jpg'

Displacement: Red: x = 3 y = 10 Green: x = 3 y = 4	Displacement: Red: x = 4 y = 14 Green: x = 2 y = 6

Image: '00165v.jpg'	Image: '00704v.jpg'

Displacement: Red: x = 1 y = 9 Green: x = 1 y = 0	Displacement: Red: x = 3 y = 13 Green: x = 2 y = 6

Image: '00858v.jpg'	Image: '00083v.jpg'

Displacement: Red: x = 4 y = 13 Green: x = 3 y = 6	Displacement: Red: x = 1 y = 6 Green: x = 2 y = 2

Image: '01728v.jpg'	Image: '01039v.jpg'

Displacement: Red: x = 1 y = 18 Green: x = 1 y = 8	Displacement: Red: x = 2 y = 11 Green: x = 1 y = 5

Image: '00084v.jpg'	Image: '31421v.jpg'

Displacement: Red: x = -9 y = 11 Green: x = -4 y = 4	Displacement: Red: x = 0 y = 13 Green: x = 0 y = 8

Image: '00017v.jpg'	Image: '00056v,jpg'

Displacement: Red: x = -3 y = 1 Green: x = -2 y = 1	Displacement: Red: x = 1 y = 13 Green: x = 1 y = 5

Image: '00362v.jpg'	Image: '00872v.jpg'

Displacement: Red: x = -5 y = 11 Green: x = 0 y = 5	Displacement: Red: x = -2 y = -2 Green: x = -1 y = -3

Called function: [Goffset Roffset] = color('image_name', window_size)
image_name - the name of the image in the Photos/ directory
window_size - the size of the window of possible displacements
Goffset - the alignment vector of the green channel
Roffset - the alignment vector of the red channel

The first value of the alignment vector is the y (vertical) component
and the second value is the x (horizonal) component.
Values of y component increase from top to bottom of the image
and values of x component increase from left to right of the image.

With the exception of 01728v.jpg where the window_size is set to 20,
in all of the remaining ones we do exhaustive search on [-15x15] area.

To remove the white/black edges that cause error in the alignment process,
after separating the RGB channels, we crop 1/15 of all images from four sides.
For images: 31421v.jpg and 00084v.jpg we only have to crop 1/25 from their top.

High Resolution - Image Pyramid - Sum of Squared Differences

Image: '01044u.tiff'

Displacement:
Red: x = 0 y = 22
Green: x = 1 y = -11

Image: '01845.tiff'

Displacement:
Red: x = 34 y = 84
Green: x = 23 y = 37

Image: '00797u.tiff'

Displacement:
Red: x = 14 y = 144
Green: x =12 y = 67

Image: '01443u.tiff'

Displacement:
Red: x = 44 y = 79
Green: x = 21 y = 32

Image: '00794u.tiff'

Displacement:
Red: x = 17 y = 123
Green: x = 16 y = 53

Image: '00033u.tiff'

Displacement:
Red: x = 14 y = 104
Green: x = 10 y = 52

Difficulties

Image: '00153u.jpg'

Displacement:
Red: x = -810 y = 196
Green: x = 24 y = 69

In this case, the algorithm implementing Gaussian pyramid returned
extremely bad alignment vectors.

Image: '00153u.jpg'

Displacement:
Red: x = 16 y = 72
Blue: x = -24 y = -69

In order to improve the alignment, a new function colorrev() was written.
Instead of aligning a red and green filter to the blue one (as in color())
it aligns red and blue filter to the green one.

Write-up

Aim:
Using image processing techniques automatically colorize images of the Russian Empire
from the Prokudin-Gorskii photo collection (both high and low resolution versions).

Process of completion:
- Extract three color channel images from one glass plate
- Place them on top of each other in the right order: red, greeen, blue
- Remove all white and black edges around the image
- Find the best alignment between the filters using
either Sum of Squared Differences (SSD) method of measurement
or Normalized Cross-Correlation (NCC) method of measurement
- For large resolution images implement the image pyramid
- Crop final image to the area shared by all three layers

Bells & Whistles

Image Cropping

Image: '00646v.jpg'	Image: '00646v.jpg'

In order to properly crop the picture, two points are found.
The first one - the lowest value in the upper left corner
and the second one - the highest value in the lower right corner.
These two numbers form the diagonal of the cropped image.

Image: '00362v.jpg'	Image: '00362v.jpg'

Cropping to the areas shared by all three layers

Image: '00084v.jpg'	Image: '00084v.jpg'

Cropping to the areas shared by all three layers

Image: '00086v.jpg'	Image: '00086v.jpg'

Cropping to the areas shared by all three layers

Color Adjustment

Image: '00498v.jpg'	Image: '00498v.jpg'

Adjusting the color of the image - imadjust()

Image: '01039v.jpg'	Image: '01039v.jpg'

Adjusting the color of the image

Image: '00872v.jpg'	Image: '00872v.jpg'

Adjusting the color of the image - imadjust()

New Digitized Plate

Self made digitized plate