The goal of this project is to implement the paper Schold et al, Video Textures, 2000, and be able to produce video with user defined length based on a short clip.
Implementation
All coding are done in matlab. I used dist2.m provided in project 4 to compute the SSD between all the frames. However to format the data from the video for dist2 to compute, I often ran into Out of memory error in matlab, which eventually eliminated me from reading in large videos. To compute the probability of transaction between frames, I went through three stages:
First I only computed the probability between individual frames based on the SSD result. This is the plot of the probability my code computed for the clock video example from the paper. Red color means high probability, obviously the diagonal line from upper left corner to the lower right corner is the frame path of the original clip.
Next I implemented the "preserving dynamics" part by taking the average of the SSD of 3 consecutive frames, and use that as the SSD for the frame in the middle. Then I plotted the new probability graph for the same video clip used above. You can tell it eliminated centain paths already.
Finally I added in the "avoiding dead ends" part by weighting the SSD of any transaction going into the ending frame more than others. This is the plot of the final probability graph. The right bottom corner looks darker now.
To choose the transaction frames besed on the probability matrix, I used several parameters. First only the frames with probability higher than 0.82 would be picked. Second, it has to be more than 10 frames apart from the previous frame, and I also set up a random number condition to make sure the transaction is randomized. Finally when I was rendering the final video I added the cross-fading technique to smooth the transaction.
Some similar motion clips. Thanks to Jing for providing it. Original: 150 frames. Testing Result: 200 frames.
This is the probability plot of this video:
The following clip is the reason why I wanted to do this project in the first place. Double pendulum is known for chaotic motion, so I wanted to take a clip of it and make it into a loop, just for fun ^^. The original clip is from youtube, I only used 100 frames from it and resized them to half, and the resulting clip is 130 frames.
This is the probability plot of this video:
Some problems
Some of the clips are too big for matlab to handle, some of them don't have enough pattern to repeat, so I couldn't explore more. I also had to write matlab function to crop, resize, and rotate some of them, but then I think I lost some quality there. Overall it has been really time consuming, but also fun I guess^^
