By Ekapol Chuangsuwanich
Procedure
Alignment Algorithm
The Green and Red layer is slide along the x (vertical) and y (horizontal) axis. Compute the difference between each point in the overlapped area, A (the non-overlapped point is not considered). This is a modified version of the SSD mentioned in the assignment. Since each x,y slide there are different numbers of points considered, the SSD is square rooted then divided by the number of points in the overlapped part. Thus, the SSD is calculate by the formula
SSD = sqrt(sum(sum(image1(A) - image2(A)).^2)))/area(A)
The x,y pair that gives the minimum amount of SSD is the best alignment for the layers.
In this report, a negative x shift is shifting the Green/Red layer upwards. a negative y shift is shifting the Green/Red layer to the left, and vice versa.
Testing the Algorithm
Using the algorithm on 00056v.jpg, yeilds Green shift of (6,1) and Red shift of (13,1).
Unshifted version
shifted version, non-overlapped part cropped
Note that the on the bottom of the picture there is a noticeable green that should not be there. Upon checking each of the color channel (shown below), the problem is due to the dark line at the bottom of the red layer. Thus, one way to improve the image quality is to crop the dark frame lines of the 3 layers then reconstruct the image
The 3 color channels of the pictured already shifted
Frame cropping
In order to remove the dark frames of each layer, we use the fact that the difference between the dark frame and the picture is very noticeable. In order to detect the left edge, we sum along each vertical line. The maximum difference between each line will appear at the transition between the frame and the picture. In order to distinguish whether the maximum difference is a real frame or not, I test it with a given threshold. Note that this is a very simple edge detection algorithm. If the frame is not straight enough, the algorithm will fail (the difference does not pass the threshold). In this case, only the white part outside the frame is remove as much as possible. Below is the same image after the frame cropping is performed.
The green at the bottom now disappears. However, the algorithm
fails to crop the right edge due to the markings on the dark frame.
Cropping the frame also has another advantage. If we apply frame cropping before the alignment algorithm, we get better results since there are less irrelevant information on the picture. Shown below are the same part of a picture. The left one does the alignment before the frame removal. The right one does the frame removal before alignment.
Pyramid implementation
Alignment is now done recursively by trying to find the best alignment on a smaller version of the picture to the original. The size of each level of the pyramid is reduced by half and stop when the size is smaller than 30. Frame cropping does not utilize the pyramid structure, since the frame detection is very sensitive to the size of the picture. Image is resized to have the height of 1024 to crop the frame. The pyramid is created from the cropped picture.
Results
Mandatory pictures
Jpeg pictures
|
|
|
|
00017 (G 1, -2 | R 1, -3) |
00056 (G 6, 1 | R 13, 1) |
|
|
|
| 00084 (G 4, -4 | R 11, -9) | 00362 (G 5, 0 | R 11, -5) |
 |
 |
| 00498 (G 4, 3 | R 10, 3) | 00646 (G 7, 2 | R 15, 4) |
 |
 |
| 00704 (G 6, 2 | R 13, 3) | 00858 (G 6, 3 | R 13, 4) |
 |
 |
| 00872 (G -3, -1 | R -2, -2) | 01039 (G 5, 2 | R 11, 2) |
 |
 |
| 01728 (G 9, 1 | R 18, 1) | 31421 (G 8, 0 | R 13, 0) |
tif pictures (click image for real size (converted to jpeg))
|
|
|
|
00033 (G 53, 11 | R 103, 15) |
00153 (G 69, 25 | R -381, 376) |
|
|
|
| 00794 (G 53,16 | R122,18) | 00797 (G 66, 12 | R 143, 13) |
|
|
|
| 01443 (G 33, 21 | R 79, 44) | 01754(G 34, -8 | R 93, 21) |
Additional pictures (tiff)
|
|
|
|
01565 (G 87, 10 | R 172, 7) |
00153 (G 59, 16 | R 124, 14) |
|
|
|
| 01143 (G 21,21 | R 59, 41) | 00194 (G 38, 21 | R 77, 35) |
Result discussion
Most image looks properly aligned except for 00153. Upon looking closely at the output image, the only miss-aligned layer is the red channel. In the picture, the red channel is brightest (highest value). The picture is very uniform for the most part (except for the man). Thus, the incorrect shifted does not reflect to a higher SSD value than a correct one. Also due the difference in brightness, it seems that the algorithm is trying to align the ground in the red channel to the wall in the blue channel instead of aligning the man because the picture consist more of the background. The green is still correctly aligned because the difference in brightness between blue and green is lesser than blue and red.
00084 also got an undesirable effect from the frame removal. The line on the top part is mistaken for the frame and is cropped off.
Bell and Whistle
Implement simple border cropping as described above.
