1. Automatic Image Rescaling Preserving Design Intention Research Update Prasad Gabbur

2. Goal Simple scaling Original: Scaling that preserves the design intention of the objects:

3. Approach Split input image into background (bg) and foreground (fg) layers Scale the bg and fg layers separately Background Foreground

4. Background layer scaling Scale and shift elements to fit new page Ignore aspect ratio Classification of background elements AreaHorizontal (Top, Bottom) Vertical (Left, right)

5. Background layer scaling Scale and shift of background elements Scale Scale + Shift Scale +Shift Sx > SySy > Sx Area Horizontal Vertical To fit new page To fit new page width To fit new page height

6. Foreground layer scaling Scale and shift elements to fit new page Preserve aspect ratio Classification of foreground elements Corner (TL, TR, BL, BR) Horizontal (Top, Bottom) Vertical (Left, Right)

7. Foreground layer scaling Scale + Shift Scale + Shift Scale + Shift Corner Horizontal Vertical Sx > Sy Sy > Sx

8. Foreground layer scaling Scaling preserves aspect ratio (Scale factor = min (Sx, Sy)) Shifting preserves distance ratio (dL/dR = const, dT/dB = const) dL_olddR_old dB_old dT_old dT_new dB_new dL_newdR_new OriginalSx > Sy (dL_new / dR_new) = (dL_old / dR_old) Sy > Sx (dT_new / dB_new) = (dT_old / dB_old) Horizontal Vertical Shift of horizontal elementsShift of vertical elements

9. Element extraction Elements are connected regions in the foreground or background layer Multilayer image Background layer(s) Foreground layer(s) Extract alpha channel Label connected components

10. Connected component labeling Group together spatially connected pixels as a single component Each component is assigned a unique integer label 4-connectivity8-connectivity

11. Connected component labeling Region coloring algorithm (4-connected) [Ballard & Brown, 1982] –Each pixel (X c ) in the image is scanned with the following mask: XcXc XlXl XuXu new_label = 1 If (X u Є background & X l Є foreground), then label (X c ) = label (X l ) Else if (X u Є foreground & X l Є background), then label (X c ) = label (X u ) Else if (X u Є foreground & X l Є foreground), then label (X c ) = min ( label (X l ), label (X u ) ) Else label (X c ) = new_label new_label = new_label + 1

12. Connected component labeling Basic region coloring algorithm is slow Requires multiple passes through the image A faster version is realized with the help of a custom data structure An array of the above data type can store information about all connected components Only one pass through the image is necessary

13. Connected component labeling A single raster scan of the image gives rise to following structure … … … … … … … … Image with two connected components Data structure at the end of a single image scan

14. Connected component labeling Links in the array can be visualized as a tree structure Nodes in the tree are equivalent labels of a connected component 112 34 1112510 6789 A fictitious connected component

15. Connected component labeling Trees with different configurations are possible depending on region complexity All branches merge at the bottom One branchTwo branchesFour branches

16. Connected component labeling Resolving label equivalences All the equivalent labels are assigned the least value among them by stepping through the tree 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 10 11 12 Resolve

17. Connected component labeling 112 34 1112510 6789 Resolving label equivalences 111 11 1111 1111 Resolve

18. Elements Each connected component in the background or foreground layer is an element Geometric properties (bounding box center and limits) are computed as part of the labeling process Elements are classified based on the geometric properties Background layerForeground layer

19. Scaling issues* Scale up –Sparse distribution of pixels in the output image –Bilinear interpolation to fill in pixel values Scale down –Aliasing due to sub-sampling –Low pass filtering before sub-sampling * Thanks to Jian Fan, HP Labs.

20. Results Background layer Original Labeled Scaled (Sx >S y) Scaled (Sy >S x)

21. Results Foreground layer Original Labeled Scaled (Sx >S y) Scaled (Sy >S x)

22. Next Stitch together the foreground and background layers Work on an XML design for input

23. Thank you!