1. Filling Algorithms Pixelwise MRFsChaos Mosaics Patch segments are pasted, overlapping, across the image. Then either: Ambiguities are removed by smoothing (Chaos Mosaics-MSR). Or a least cost path through the (chosen) overlapping images are found. Efros01 uses dynamic programming, while Graphcut textures03 uses… graphcut. Pixel distributions are determined by comparision with those with similar neighbourhoods. These distributions are sampled from or heuristics are performed on them to determine how to fill them.

2. Synthesizing One Pixel Infinite sample image Generated image Assuming Markov property, what is conditional probability distribution of p, given the neighbourhood window? Instead of constructing a model, lets directly search the input image for all such neighbourhoods to produce a histogram for p To synthesize p, just pick one match at random SAMPLE p Taken from Efros original presentation

3. Really Synthesizing One Pixel finite sample image Generated image p However, since our sample image is finite, an exact neighbourhood match might not be present So we find the best match using SSD error (weighted by a Gaussian to emphasize local structure), and take all samples within some distance from that match SAMPLE Taken from Efros original presentation

4. The pixel metric doesnt matter One of the surprising things about this is that the choice pixel metric doesnt seem to matter. Efros uses ||. || 2 2 on RGB space, Ive been using ||. || 1, while Criminisi uses a metric based on the CIELab colour space. It doesnt seem to matter which you use, presumably because we are only concerned with nearest neighbours, and they are all topologically equivalent.

5. Dynamic Programming solution block B1 B2 Neighboring blocks constrained by overlap B1B2 Minimal error boundary cut Chaotic Mosaics

6. Graphcuts solution Chaotic Mosaics Takes full advantage of the power of graphcuts method, it treats the whole image as one patch and finds optimal joins along it. Pros: Finds optimal (and often seamless) matches Cons: Doesnt find anything else, the recycling the optimal matches still leaves you with tiling artefacts.

7. Graphcuts solution Chaotic Mosaics Image QuiltingGraphcuts

8. Onion skin or Outside in The first. The simplest? Works with single textures or simple convex filling regions Just picks away at the image one layer at a time Pixel choice and Filling Algorithms

9. Linear structure propagation Onion skin +pushing in on linear textures Better than Onion skin for multi textural environments When all you have is hammer, everything starts to look like a nail. ~ Artefacts from trying too hard. Pixel choice and Filling Algorithms Missing Data Correction in Still Images and Image Sequences, Bornard et al. 2002

10. Linear structure propagation Onion skin +pushing in on linear textures Better than Onion skin for multi textural environments When all you have is hammer, everything starts to look like a nail. ~ Artefacts from trying too hard. Pixel choice and Filling Algorithms Missing Data Correction in Still Images and Image Sequences, Bornard et al. 2002

11. Filling Algorithms Onion Peel Vs. Linear propagation Push In Now Onion Peel

12. Max. entropy fill Consistent with MRF assumptions. Locally convex with a minimum of occlusions at point of fill. Spirals in on simple shapes. Pixel choice and Filling Algorithms

13. Why Coarse to Fine? Standard Efros 15 pixelsStandard efros 21 pixels New metric Efros 15 pixels Efros uniform pixel weighting 15 C2f uniform pixel weightingCourse to fine 15 pixel nhood The same but quicker...

14. Structures... Why Coarse to Fine?

15. Structures As Textures Why Coarse to Fine?

16. Structures As Textures Why Coarse to Fine?

17. Texture as structure? ? Strong linear propagation comes from efros style fills naturally. Why Coarse to Fine?

18. Not readily apparent due to the onion skin fill Efros Linear propagation The Efros Algorithm with my pixel choice

19. No guarantee its any better than linear propagation The algorithm often spots at the coarser levels that it has insufficient data to complete

20. No guarantee its any better than linear propagation Annoyingly, this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the higher levels is still needed.

21. No guarantee its any better than linear propagation Annoyingly, this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the higher levels is still needed.

22. No guarantee its any better than linear propagation Annoyingly, this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the lowest resolution is still needed.

23. No guarantee its any better than linear propagation Annoyingly, this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the lowest resolution is still needed.

24. No guarantee its any better than linear propagation Annoyingly this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the lowest resolution is still needed.

25. No guarantee its any better than linear propagation Annoyingly, this problem is not solvable by more data, in the sense of higher resolution images. Information about how to propagate edges at the lowest resolution is still needed.

26. No guarantee its any better than linear propagation Compare it with a smaller fill

27. Is manifold learning possible? ? Up to a point, we dont care what colour the books are when propagating the shelf. Similar structural edge patterns are apparent everywhere. Why Coarse to Fine?

29. Problems Speed Massively slower(hours rather than seconds) than patch based synthesis even with coarse to fine reducing neighbourhood size. Can we reduce the search space via image segmentation? Alternatively turn our soft coarse to fine constraints into something harder, by only testing pixels from the neighbourhoods of the k-closest fits at a coarser level.

30. Problems Surprisingly, this could even increase robustness by preventing the growth of miss-fittings