Agenda Seam-carving: finish up “Mid-term review” (a look back) Main topic: Feature detection

Seam carving: construct an energy image

Define (vertical) seam All the pixels in the seam are removed (shift row/column) Visual artifacts are visible only along the seam k=1k=2k=M Slide credit: Andrea Tagliasacchi

Define cost function for seam Energy function: (gradient) let I be an n×m image and define a vertical seam to be: Optimal seam s* that minimizes this seam cost : 4 How to efficiently compute s* ?

Find best seam with dynamic progamming For M row by N column image: Scan every row in the image from i=2 to i=M updating with local best choice In the final bottom row we pick the smallest entry M ij (for i=M) and we backtrack a path choosing always the local minima Slide credit: Andrea Tagliasacchi i-1 M i,j = cost of best seam from top row (i=1) down to pixel i,j i jj+1j-1

(c) ariel shamir Alternative formulation: mincut on a graph p i+1,j p i,j p i+1,j+1 p i,j+1 yy xx xx yy p i,j p i,j+1 p i+1,j p i+1,j+1 yy yy xx xx S T -Each pixel is a node that’s connected to its 4 neighbors and a “source” and “terminal” -Weight edges appropriately (eg, gradient magnitude) -Find minimum cost cut that separates S and T – this is a “graphcut” problem -Graphcuts is a very common tool in pixel labeling problems -Can solve mincut/maxflow problem (cf, Algorithms textbook) -With a particular choice of of weights on edges, the min cut is equivalent to DP soln

(c) ariel shamir Extension to video: 3D graph of pixels Frame t Frame t+1 Frame t+2Time Video Cube

(c) ariel shamir 3D Graph Cut Video Cube

Video

Halfway done with course; take a breather Midway evaluation form is posted on EEE – First-time course, so please provide feedback!

A look back Part I: Introduction – Camera optics – Color – Fourier/filtering Part II: (Photography) Image enhancement – Texture synthesis Texture models: histograms of textons/filter responses Markov models: sampling from conditional probability tables – Image blending Gradient-domain editing Constrained optimization (lagrangian techniques) – Image matting Compositing Bayesian modeling Gaussian color models – Image retargeting (resizing) Dynamic programming Combinatorial optimization (graphcuts) Part III: (Vision) Visual analysis – Feature matching – Mosaicing/stitching images – Recognition (finding and recognizing faces)