Technological Uncanny K. S'hell, C Kurtz, N. Vincent et E. André et M. Beugnet 1

Context  The “ Research of the Technological Strangeness in the digital era", is organized by the UDPN network.  A group of artists from the university of Paris-Diderot are interested in new forms of strangeness caused by the digital, seen in the same time as a mean of archiving, and as a manipulation tool for image processing.  The project is divided in 2 sub-projects 2

Issues  Increasingly, artistic videos use the advantages of digital techniques to create predetermined feelings among the public by removing objects or replacing them by another.  Creating special effects with traditional methods: Either manually on film tapes Manually with computer help treating the video frame by frame using image ''photoshopping'' applications Automatic object manipulation in existing videos 3

How to remove an object from a video  Object Selection  Moving the object and filling the gap left behind  Generation of the procedure on video using tracking (a) (c) (b) 4

Outlines  Context  Issues  Object Extraction  Inpainting  Proposed Approach  Experimental results  Conclusion 5

Object Extraction  It’s about extracting an object from the first frame of the image  Interactive Segmentation of the first video frame using the Grabcut algorithm (among other methods) 6

Grabcut Segmentation How it works from user point of view ?  Initially the user draws a rectangle around the foreground region (foreground region should be completely inside the rectangle). Then the algorithm segments it iteratively to get the best result done. 7

8

What’s Inpainting?  It refers to the field of computer vision which aims at filling-in holes in an image sequence using spatial and temporal information from neighboring regions. The holes may correspond to missing parts or removed objects from the scenes (logos, text, etc.).  The main objective of video inpainting approaches is to complete the hole in a way which would be as visually plausible as possible both in space and time. 9

Technics of Image inpainting(1) PDE based inpainting A Partial Differential Equation (PDE) based iterative algorithm proposed by Bertalmio et.al paved the way for modern digital image inpainting. Borrowing heavily from the idea of manual inpainting, this iterative process propagates linear structures (edges) of the surrounding area also called Isophotes, into the hole region. 10

Technics of Inpainting(1): PDE based inpainting  Almost all the diffusion-based inpainting approaches are focused on prolonging the structure at the border to inside the inpainted region.  These approaches perform well for filling small gaps. However, they cannot deal with complex textures and large holes where they introduce blurring artifacts. 11

Technics of inpainting(2): Patch Match based inpainting  Another category of image inpainting methods solves the completion problem by using texture synthesis techniques which are sometimes referred to as region-growing methods. 12

Technics of inpainting(2): Patch Match based inpainting Texture synthesis based inpainting approaches are not suitable for images with a lot of structures corresponding to the object contours as most of the natural images. 13

Patch based Inpainting 14

Proposed Approach : Tracking 15

Proposed Approach : Color Filtering  To add accurency to the object tracking by the grabcut segmentation algorithm, especially when the object is too small to be dectected by the interactive segmentation we accumulate hue filtering to the segmentation result. 16

Proposed Approach : Morphological Operations  They are a set of operations that process images based on shapes.  Morphological operations apply a structuring element to an input image and generate an output image.  They have a wide array of uses. 17

Proposed Approach : The main procedure 18

Extra-steps for particular types of movies Case of Black and White Videos Inability of the grabcut algorithm to extract object from the frame Modification of Contrast Thresholding Recoloring 19

Extra-steps for particular types of movies 20

Extra-steps for particular types of movies Case of Cartoons  Deterioration of the quality of the image after inpainting Modification of Contrast Contour Extraction Blinding the inpainted image with its contour 21

Extra-steps for particular types of movies 22

Conclusion  We achieved digital modification on video using image processing techniques.  We are looking foward to implementing more optimazed techniques which can give us better looking results 23