Originally, I wanted a program that would take a set of photos of a particular scene and extract objects. Then, I would be able to randomly generate new scenes based by placing randomly selected objects. This however, did not get fully implemented as I was intruiged by some other aspect of multi-image photography. Also, all the scenes I captured did not lend themselves too well with the placement of objects.
The first (and simplest) part of this project became the basis for everything else that follows. I was thinking about how I could implement part of the gigs of photos I took over the past couple years, and noticed that I really liked slow shutter photography. The first problem I decided to tackle was the use of multiple underexposed images to give me a single well-exposed image. This could be used to solve the problem of hand jitter while capturing dark scenes.
These images were of the desk in the EMS office while I was just sitting around fooling around with my camera. I found that being a simple cheap 4 year old point and shoot camera marketed for its simplicity, it didn't really have manual control, so that was a bit of a limiting factor. In total, there were 16 images, resized to 640x480px. The images were relatively stable, but there was some movement of the camera from picture to picture.
The next part was to simply use the alignment functuon from Project 1. I found that both using edge information for alignment or just simple texture information was sufficient to perform a good alignment.
Finally, we just did a simple addition of slides. I didn't really know how much -3EV was equivalent to, so I just added random images until it got the right lighting level. While this isn't as fancy as finding the lighting curves, the linear addition proves to be good for this example. That said, I don't see how it could be too difficult to implement the curve.
This effect is one of my favorite effects. This effect takes advantage of a bright particles streaking across picture against a dark background.
For example, this picture I took during Boston's 4th of July celebration in 2005:
For this experiment, I took some old footage I had from an experiment where a chunk of sodium was thrown into a body of water, with colorful results. Capturd 25 clips from the video:
As a result, we get:
Another effect that uses a slow shutter speed is the motion blur from high motion scenes. While often unavoidable, this effect can be used to enhance the sense of action in the picture.
For example, this picture I took from a icebreaking activities at a summer camp I went to a couple years back:
For this project, I used two different set of videos from a TV show "Third Watch". Both videos show some complicated motion that needs to be captured. The first one has some significant camera motion while the second has a difficult object motion.
For these images, an area of interest was also defined so that the alignment is performed with regard to the propert objects.
After alignment, the images are summed up, and displayed. My results:
For the image on the left, we get some decent motion blur while maintain the crisp image of the car. However, there is some aliasing that occurs with the motion blur that shows that we need a higher FPS to get a smooth motion blur. For the image on the right, the car is decently sharp, especially the rear half of the car. However, since the geometry of the car changes a bit, the front half is blurred. Also, we don't get the same sense of blurring because the background contains little landmarks that get blurred.
The last part of this project goes back to the original projet proposal. Using multiple images to selectively remvoe the foreground noise to extract a static background.
For this part, I took my trusty camera, put it on my windowsill, and took pictures every once in a while, capturing 52 pictures in total. For example:
The first part was to extract align the images. Then they were averaged, using a mean filter (left) and a median filter (right) to extract the static background.
As expected the median filter gives us a sharper image. However both have some barrel distortion around the edges of the picture lending to some blurring. Also, there was some wind in the image that caused some blurring in the branches. The median filter also required all the images to be loaded before the median could be calculated, which as a result, required more time and a larger buffer. Also notable is the fact that the snow has disappeard from the images.
After the background is extracted, we want to see if we can isolate different foreground objets. The difference is calculated, thresholded by value, then thresholded by size. Then a dilation is performed on the image, followed by gap filling. The result is mixed.
Each of these patches are individually captured and saved.
In general, the good objects recovered were light (high difference from dark asphalt background), unobscured, and in the center roadway.
However, there were some problems. The minivan in the front right often got categorized as a new object. In addition, the snow interfered, adding many splotches of nove objects. Also, dark vehicles did not turn up well becuase not much difference from the asphalt. The final major problem came from the two occluding objects (tree and pole), that would often split a solid object into two.
As stated before, the limited area that we had to work with reduced the possibility to randomly assign objects to be placed in the image. However, conceptually, we'd register the bottom-most point of each object to help organize the depth of each object. Then we'd tile objects from rear to front.
Some promising results that utilize what digital cameras are truely good at. Taking massive amounts of pictures with no remorse. We trade initial quality for quantity, and generate the quality from them. However...
Framerate needs to be addressed. For a smooth blur, a really high framerate needs to be used. Also, light is not linearly additive, may need to consider the HDR curve for more extreme usages. And of course, the quality of the picture is always reliant on the quality of the photographer. No amount of digital processing can replace a quality photographer.