For video textures, I had to find a subject whose movement was fairly simple, repetitive, and didn't have too many degrees of freedom. So I decided on this subject.

After that, I split the movie into into individual frames and deleted all the frames where my hand was showing. That left me with 7 discontinued action segments.

Step 1
I computed the difference between each frame and stored it in a nxn matrix, diffMatrix, where n is the number of total frames in all my segments. I used simple SSD.

Step 2
I computed the probabilities that frame i would progress to frame j and stored it into matrix prob. The probability that frame i would progress to frame j if frame i+1 is very similar to frame j.
prob(i,j) = e^(-diffMatrix(i+1,j)/sigma)
where sigma= about the average value of diffMatrix entry.
Then I normalized the probabilities so that sum(prob(i,:))=1.

If we stopped here, and just used these probabilities, then it would just go straight through frame by frame and then get stuck at the end.

Step 3 - Preserving Dynamics
create a diffPrime matrix that calculates the difference between two neighborhood of frames. (two frames before, the current frame and the frame after)

let k=-2 to 1
diffMatrix'(i,j) = sum(weight(k)*diffMatrix(i+k,j+k))
calculate a new prob' based on new diffMatrix'
(plus a lot of hacking to prevent it from reaching frames that don't have adequate probability info.)
First round results - quality varied depending on the frame I told it to begin on
(start from frame 10) Decent
(start from frame 50) Still okay
(start from frame 180) Plain bad

This happens because the transitions followed the frames to a "dead end" (e.g. when the duckie turns.) The duckie only faces that direction at the end of the video sample, so there is no decent transition for continuing past it.

Also, some transitions are jumpy because whenever it's not at the end of a segment, the best match for the next frame is always just the next frame. So it keeps playing the segment until it reaches the end. But the best transitions for the end of a clip isn't all that great.

Can fix that with a lot more data, or we can try...

Step 4 - Avoiding dead ends and anticipating the future

let's make a matrix fCost, where fCost(i,j) = diffMatrix(i,j)^p + alpha * min over k(fCost(j,k))

p and alpha are constants. the greater p is, the smoother the transitions.

Algorithm is recursive, but they did some fancy math and simplified it.

(start at frame 10) Pretty good
(start at frame 50) Hmmm...
Still good

Some of it still looks a bit jumpy, but I guess that's because I only have about 190 frames of data. Since I don't have that much data, the frames that look most alike are when the duck is not moving much. So the algorithm ends up doing the work after an active segment is done (the duck stops bouncing) and just adds more frames of the duck being pretty still.

Looks like simple SSD doesn't calculate the similarity of two images too accurately. Using a different method should help.

Another goal --> get enough data, keep only frames where duckie is not at rest. So that it can construct a clip of the duckie constantly bobbing up and down.

Things I didn't account for -> When I clipped out the frames with my hand in it, I didn't clip out all the frames that hand a little bit of my hand's shadow, or my reflection in the cup and water. That affects the smoothness of the transitions and the uniformness of the data.