Kshitij Padalkar 13 th Oct 2008

Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy Functions? Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal Multi-size Images Limitations Links

In One Line… Method for Content-Aware Image Resizing

Need

Different Displays-Different Resolutions-(Different Images??)

HTML allows automatic layout of the text, but not the images

Solutions and Other Work Crop and Scale Saliency Segmentation

Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy Functions? Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal Multi-size Images Limitations Links

Definition and Maths… Goal is to remove unnoticeable pixels. Energy Function: Remove Pixels with least energy Sort pixels by Energy, remove smallest pixels (optimum) Remove Column with smallest total Energy (column) Remove Seam with smallest total Energy (seam)

Seam Vertical Seam: an 8-connected path of pixels in the image from top to bottom, containing one, and only one, pixel in each row of the image

Seam Energy of a Seam Minimum Energy Seam

Pixel Removal Criteria Optimal

Pixel Removal Criteria Pixel

Pixel Removal Criteria Column

Pixel Removal Criteria Seam

Pixel Removal Criteria Comparison

Other Energy Functions Gradient Entropy HOG (Histogram of Gradient) Saliency

Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy Functions? Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal Multi-size Images Limitations Links

Aspect Ratio Change Change the image from (n X m) to (n X m’) (assume m’ < m) Remove m-m’=c seams successively Seam Carving

Aspect Ratio Change Change the image from (n X m) to (n X m’) (assume m’ < m) Remove m-m’=c seams successively Scaling

Optimal Seam Ordering Change the image from (n X m) to (n’ X m’) (assume m’ < m and n’ < n) What to do? What is the correct order of seam carving? Remove vertical seams first? Horizontal seams first? Or alternate between the two?

Optimal Seam Ordering We optimize where k = r+c, r = (m−m’), c = (n−n’) and  i is used as a parameter that determine if at step i we remove a horizontal or vertical seam:  = {0,1}

Optimal Seam Ordering Transport map Matrix of size n X m Each element T(r,c) holds the minimal cost needed to obtain an image of size n−r X m−c

Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal

Image Enlarging compute the optimal vertical (horizontal) seam ‘s’ on I and duplicate the pixels of s by averaging them with their left and right neighbors (top and bottom in the horizontal case)

Image Enlarging To reduce the width by ‘k’, compute first ‘k’ vertical seams and duplicate them All seams duplicated, 50% enlargement. For more do it in steps.

Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal

Content Amplification Scale the image, this will scale everything, content as well as non-content. Shrink the scaled-image by seam carving, this will carve out the non-content part.

Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal

User marks the target object to be removed. Then seams are removed from the image until all marked pixels are gone. To regain the original size, seam insertion is employed.

Object Removal Find the missing shoe…

Object Removal Object marking to remove unwanted results

Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy Functions? Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal Multi-size Images Limitations Links

Multi Size Images Methods mentioned so far are not real-time. We calculate each seam remove it and calculate the next seam. For real-time, we calculate- index map V of size n×m that encodes, for each pixel, the index of the seam that removed it, i.e., V(i, j) = t means that pixel (i, j) was removed by the t-th seam removal.

Multi Size Images Horizontal Index map (H) Vertical Index map (V) To get an image of width m’, we need to remove m-m’ pixels from each row we gather, in each row, all pixels with seam index greater than or equal to m−m’. Same for changing the height. Problem comes when we want to change both!

Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy Functions? Discrete Image Resizing Aspect Ratio Change Optimal Seam Order Image Enlarging Content Amplification Object Removal Multi-size Images Limitations Links

Limitations Too much content No space for seam to avoid content

Paper ftp://ftp1.idc.ac.il/Arik_shamir/SCweb/imret/vidret.pdf Youtube demo Search google for Seam Carving…. Many many blogs… Links

THANK YOU