Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 The Development of Image Completion and Tutorial Plug-ins for the GIMP By: Cathy Irwin Supervisors: Shaun Bangay and Adele Lobb

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Project Aims Automatic image completion plug-in for GIMP (GNU Image Manipulation Package) Tutorial for general plug-in development

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Image Completion Remove scratches, defects, writing & objects, reconstruct damaged images - Manual techniques are painstaking Automate process – user only selects region to be removed Plug-in: realistically fill in background regions by analysing known regions

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Related Work Texture synthesis Image inpainting Efros & Freeman (2001) Bertalmio, Sapiro, Caselles & Ballester (2000)

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Image Completion Plug-in ‘Fragment-Based Image Completion’ – Drori, Cohen-Or & Yeshurun (2003)

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 My Approach Re-implementing proven technique BUT Incorporating it into the GIMP Inevitable differences – due to language, environment, coding style Verify authors’ results Scenarios Textured regions Smooth regions Geometric shapes

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Fragment-Based Image Completion Algorithm Fast Approximation Confidence Map Candidate Position Map Search Composite Fragments

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Implementation in the GIMP GIMP 2.0 Use ‘empty’ plug-in templates as basis for implementation The implementation language: C Gimp and GTK libraries manipulate the GIMP-specific elements: images, drawables, channels and layers

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Fast Approximation X

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Fast Approximation Each step is added as a new layer to the original image Manipulate and view independently of one another

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Confidence Map Certainty with which the colour value of each pixel is known Gaussian falloff value used to decreases the confidence level towards the center of the unknown region

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Candidate Position Map Takes confidence map as input Pixel with the maximum value in the candidate map is concluded to be the next most appropriate target pixel to be used in the search and composite phases

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Search Phase Iterates through each pixel in a 20 x 20 pixel block around the target to find an appropriate source pixel Average value of the pixels around each potential source it is taken to determine the best possible match with the neighbourhood surrounding the target pixel Size of this neighbourhood region determined by the user at run time but is usually about 8 pixels wide The search formula finds the pixels which have a higher confidence in the source than in the target regions and correspond well in terms of colour

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Compositing Fragments Superimpose that source fragment over the target fragment so that the detail from the source merges seamlessly with the region around the target. Transparent edges - blend in convincingly Use existing GIMP functions: – Selections – Copy – Paste

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

Results Suited to: Stochastic (random) texture that occurs often in the surrounding region Effective at making the reconstructed area blend into the known regions

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Results cont… 3D structure not taken into account Complete image according to the direction the line was taking at the edge of the gap and not according to mathematical principles

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Results cont… Parameter values depend on the size of the gap and the type of texture in the picture 160 x 120 pixel image: between 12 minutes and an hour to process Smaller gaps need a smaller blur radius and smaller neighbourhood search region. Larger, smooth or stochastic regions can be accurately completed using a larger neighbourhood region that also decreases the processing time

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Results cont…

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 On-line Plug-in Tutorial General tutorial for writing plug-ins for the GIMP in C Tutorial PageTutorial Page Provide a starting point for beginners, not a definitive guide on every aspect of plug-in development Areas covered: – areas that I found problematic during the development of the plug-in – features that were discovered to be particularly useful – references to where more specific information can be found

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004 Conclusion Plug-in development aided by availability of templates and documentation regarding GIMP specific functions Some common image manipulation operations complex to implement Not possible to re-implement the original paper exactly within the GIMP environment due to the tool and function constraints and the extra processing required for certain operations A plausibly similar implementation has been achieved - takes advantage of GIMP functions to approximate the intentions of the original paper

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004