An Interactive Background Blurring Mechanism and Its Applications NTU CSIE 1 互動式背景模糊

Introduction Privacy Preserving 2 Our system Original image

Depth Map Generation Pop-up Card Perspective Box Image Defocus Defocus Blur Camera Settings Alpha Matting Lazy Snapping Object Segmentation Face Detection Segmentations Depth Map Input image Shallow focus result

Experimental Result Defocus blur method proposed in the system comparing with other blur filter results 4 Deep focus Shallow focus

Experimental Result Defocus blur method proposed in the system comparing with other blur filter results Gaussian blur Defocus blur Defocus blur + Bokeh 5

Experimental Result 6

Observation with / without depth variation in the background Real photograph taken by DSLR 7

Experimental Result with / without depth variation in the background Original photograph Result after post-processing 8 Depth variation Depth fixed

Observation ◦ Near-by object case 9 Focus on the flower

Applications - Privacy Preserving 10

Applications - Partial Viewing 11

Applications - Partial Viewing 12

13

Applications - Image Transition at Photo Browser 14

15

16 Applications - Image Transition at Photo Browser

Movie with defocus blur effect 17 Applications - Image Transition at Photo Browser

18 Concatenate two unrelated images

Conclusion We proposed an interactive refocusing tool for background blurring ◦ Simple user hint ◦ Defocus blur kernel ◦ Can be applied to many applications Future work ◦ concatenate two or more unrelated pictures  Color based image retrieval technique 19

END THANK YOU 20

Method Segment ◦ Lazy snapping ◦ Mean shift ◦ Alpha matting ◦ Face detection 21 Lazy Snapping. ACM Trans. On Graphics Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI A Bayesian Approach to Digital Matting. CVPR 2001.

Depth Map Generation Pop-up Card Perspective Box Image Defocus Defocus Blur Camera Settings Alpha Matting Lazy Snapping Object Segmentation Face Detection Segmentations Depth Map Input image Shallow focus result

Method Depth map ◦ Perspective Box  Vanish point  Rear wall 23 Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH 1997

Depth Map Generation Pop-up Card Perspective Box Image Defocus Defocus Blur Camera Settings Alpha Matting Lazy Snapping Object Segmentation Face Detection Segmentations Depth Map Input image Shallow focus result

Method Image defocus ◦ Blur circle diameter Blur circle diameter Segment Map Depth Map Camera setting Aperture size Focal length of the lens Distance of focus Aperture size Focal length of the lens Distance of focus uv b d vbvb ubub uv b d vbvb ubub (a)(b) 25

Method Defocus blur ◦ bokeh 26

Method 27 Defocus blur

Method ◦ Near-by object case 28 Focus on the flower

◦ Near-by object case Without inpainting Interpolation result Experimental Result 29

◦ Near-by object case Method Blur circle diameter Objects in front of the focus Objects behind the focus Objects behind the focus Defocus Blur Text synthesis inpainting Blur shape as alpha map Defocus Blur Interpolation by alpha map Shallow focus image 30

Q&A 31

END THANK YOU 32

Introduction Depth of field 33 Real light Circle of Confusion - Readability range Focus plane Photo plane Out of focus – blur range Depth of field

Introduction Real world Photo plane Shallow focus 34

Introduction Shallow focus ◦ Highlight the subject by softening background diffusion Deep focus by DC NIKON E4300 (2003) Shallow focus by DSLR NIKON D90(2008) 35

Outline Introduction Method ◦ Object Segmentation ◦ Depth Map Generation ◦ Image Defocus Experimental Result Applications Conclusion 36