15-862

Dmitriy Solomonov

Project 2: Seam Carving

The purpose of this project was to reduce or enlarge the sizes of images using seam carving. The basic idea is that using some measure of energy for every pixel, you find the seam of pixel of minimum cumulative energy. This can be done for both horizontal and vertical seams.

Choosing a proper energy function is based mostly on the image in question. For images having a very smooth and unchanging background, a simple gradient of the image serves quite nicely to remove unnecessary seams. This is one of the faster energy functions. An HoG energy function requires some more computation, but works slightly better on images with more varied composition. The best computational heuristic I found was to try the gradient energy and see if the distribution of energy in a simple histogram places more than 20% of all pixels into the highest bin. In such a case, the more complex energy function was used.

Making the sizing fast is also fairly straight forward. First of all, to make the carving faster at the expense of quality, one can only calculate energy once for the entire image, and then carve pixels based on those energies. There is a slightly noticeable improvement when energy is recomputed when every seam is removed, but this is generally not required for images large enough. To make the sizing faster and more or less real-time with user interface, I pre-computed the carving the image, and stored a 2D array that stored for any row and ith seam, which column to remove the pixel from. An array would have to be stored for both horizontal and vertical carving. Doing this made carving the image very fast.

This is a nice example of the effect of the carving. This particular image suits it well because it has much empty similar space with very specific and clear objects in the image. The original image is above. The top right is a horizontal carving of 1/3 the height. To the right is a carving of vertical seams to reduce the image width by 1/3. The image came from Google images.

This image was more difficult because it has a lot of changing texture in the bottom. The 3 images shown are the original, the horizontally carved and the vertically carved. This particular image performed better with an HoG energy function.

This image has some noticeable artifacts.

Since there are many key objects, and the background is not as uniform, the carving produces artifacts when carved at the amount.

This example shows the effect of the algorithm. Here, the reflection in the water is preserved while the unimportant information is thrown away.

Image taken from Flickr.

This one is just too easy. Notice how the shapes and sizes of each planet are identical. Taken from Google Images.

This example shows how vertical carving has better results over horizontal carving. Here, the face and body contour is more distinguishable across the columns than the rows. Hence, the image is much more distorted in the top right.

Picture is provided by Google Images.

In this case, HoG gave better results. The amount of figures in this scene is very large, and the texture is not that simple. Vertical carving is actually quite good in this one as well.

This image proved ok for horizontal carving, but the tennis court in the corner broke the algorithm did and so the grass was not cut. The vertical cutting was just horrible.

Let’s just not talk about these next sets of images