1. Video Textures This is a very cool paper.

2. Why video texturing? Static images are boring Video clips are finite in length We need something more interesting

3. What is video texturing? The process of extracting an infinitely varying sequence of continuous images from a finite length video clip For the purposes of creating backward compatible videos, it is sufficient to generate a single video clip with continuity between the final and beginning frames

4. The System

5. Objectives Find points in the video between which transitions can be made Smooth the transition as much as possible Rendering An infinite random play A finite video clip using a looping sequence to simulate infinite play

6. Representation Probability Matrix Each entry indicates the probability of a certain frame-to-frame transition being taken Good for dense set of possible transitions Link Set Each entry indicates a link from one frame to another and that transition Good for sparse set of possible transitions More typical

7. Techniques Splitting the video into distinct regions

8. Techniques Sprites

9. Extraction First Equalize brightness Stabilize video Enhance in any other way desired Compute the L 2 distance to determine similarity between frames Store them in the matrix: D ij = ||L i -L j || 2

10. Extraction Now transitions from frame i to frame j can be performed when D i+1,j is sufficiently small (i.e. similar) Store the probability of these transitions in a probability matrix (or analogous structure): P ij = exp(-D i+1,j /σ) Then normalize each transition according to the rule: Σ j P ij = 1

11. The Problem This creates smooth frame-to-frame transitions, but…

12. Preserving Dynamics Smooth frame-to-frame transitions are not sufficient Smooth motions must also be conveyed In other words For a transition from i to j to be “good,” the frames i±m must be similar to the frames j±m

13. Preserving Dynamics Simply take the similarity matrix D and check for similarities between frames to form the filtered matrix D’ D’ ij = Σ k (w k D i+k,j+k ) where -m≤k≤m-1 And m is the number of frames to check on either side (1 or 2 typically) Now compute probabilities based on D’

14. Preserving Dynamics Graphically:

15. Preserving Dynamics Much better, but…

16. Avoiding Dead Ends Now we have a dead end No problem… Simply calculate the anticipated cost of transitioning from one frame to another D'' ij = (D' ij ) p + α Σ k P'' jk D'' jk High p favors the fewer good transitions Lower p favors the abundant poor transitions 0.99 <= α <= 0.999 makes a good alpha Alpha controls the affect that future transitions should have P'' is now calculated with D''

17. Avoiding Dead Ends Solving the interdependencies has a simple iterative solution… However, it’s slow to converge So simplify the calculation D'' ij = (D' ij ) p + α· min k (P '' jk D'' jk ) This assumes sigma approaches 0 (i.e. P ij approaches 1) for good transitions… only the best transition is considered

18. Avoiding Dead Ends See paper for details on iterative solution

19. Sequencing Random Play Random (infinite) play Begin sequence at any transition before the last non-zero probability transition while (1) Determine transition after frame i by finding frame j such that P ij is greatest Generates video textures that never repeat exactly

20. Sequencing Video Loops Primitive loop Single transition from i to j Must satisfy i ≥ j to guarantee loop Properties Range = [j,i] Cost = D’ ij Length = i-j

21. Sequencing Video Loops Compound Loop Combination of one or more primitive loops (or compound loops) Each loop must overlap with at least one other Properties Range = U k range(T k ) Cost = Σ k cost(T k ) Length = Σ k length(T k )

22. Sequencing Video Loops Selecting Transition Sets Need to find the optimal loop for a given sequence length This is just a simple DP algorithm

23. Still Sequencing Video Loops

24. Sequencing Video Loops

25. Scheduling Now we have a set of transitions to use All we know, based on the properties of compound loops is that the set represents a continuous range of frames Must schedule the set into a coherent sequence of transitions

26. Sequencing Video Loops Algorithm for Scheduling Primitive Loops 1. Schedule the transition at the end of the range first 2. Remove transition from set – this may leave one or more disjoint sets 3. Schedule the next transition from the first disjoint set that begins after the beginning of the removed transition 4. Repeat 2-3 until no transitions remain

27. Rendering Between transitions Cross fade Morph or de-ghosting Between film regions Feathering

28. Questions? Is “cyclified” a word?