1. Author: Vera Kukić Supervisors: Shaun Bangay Adele Lobb George Wells
Investigating the Development of Colour Interpolation Techniques for GIMP
Author: Vera Kukić
Supervisors: Shaun Bangay
Adele Lobb
George Wells

2. Overview Presentation Overview Introduction Colour Transfer Design
Implementation
Results
Conclusion

3. Introduction Motivation for the study Hence our Objective:
Recent colourisation and colour interpolation techniques
Bring the power of colourisation to common graphics packages
Hence our Objective:
Explore and evaluate colour interpolation plug-ins in the GIMP framework
Integrate colourization into GIMP
Allows GIMP users the ability to use a colourisation tool
The best way to describe a project is to state its WHAT WHY HOW
WHY- The motivation
There have been recent advances in automated image colourisation (2001, 2002 papers) – which warrant investigation
Moreover, the integration of colourisation techniques with in a common graphics and image manipulation package is of great benefit to ordinary users.
They are able to easily colourise images in a fairly automated fashion (as opposed to traditional colour paletting methods) Applications for colourising old black and white family photographs, or movies ect abound
WHAT – the research objective
Hence, it was decided to explore and evaluate these emerging colourisation techniques in a wide used open source graphics package – GIMP
The benefits of this are thus – not only can we use the GIMP framework as an implementation and evaluation platform but also provide a useful colourisation plug-in for the wider GIMP community
Integrated it into a prolific graphics package so that the images can be alter in other ways as well – a stand alone application is not as useful

4. Colour Transfer Colour Transfer Techniques Colouring Book Method
Photoshop: manually paint colour with low opacity
Movie Industry: track polygons
Pseudocolouring
Global Transformation/Colour Map
Colour Transfer between Images
Colour Transfer to Greyscale Images

5. Colour Transfer Colour Transfer between Images [Reinhard et al] + =
Source
Target
Final

6. Colour Transfer Colour Transfer to Greyscale Images [Welsh et al] + =
Select colour source image
Match each target pixel with a few source pixels
choose best match using local neighbourhood statistics
Transfer Colour
Repeat for all pixels
Source
Target
Final

7. Colour Transfer Algorithm
Select Source and Target Images
Convert Images to the lαβ colour space
Image Matching
Colour Transfer

8. Colour Transfer Algorithm
Convert Images to the lαβ colour space
lαβ space was developed to reduce the correlation between the 3 coordinate axis of the colour space
l –> chromatic luminance channel
α –> yellow-blue channel
β –> red –green channel
Ruderman developed the colour space and the

9. lαβ colour space
the α and β values approximate to zero for neutral colours white, grey and black.
α as a measurement of red for α positive or green for α negative
β as a measurement of yellow for β positive or blue for β negative.

10. Colour Transfer Algorithm
Convert Images to the lαβ colour space
Images are converted using matrix manipulation
RGB -> XYZ tristimulus values
XYZ -> LMS space
LMS -> lαβ (using log conversion)

11. Colour Transfer Algorithm
Image Matching
Create luminance histograms of the two images
The image:
has a histogram:
Example of luminance histogram
This is the representation for twelve pixels only, histograms for complete images are denser
cover the complete range of all the luminance values {0,..,255}.

12. Colour Transfer Algorithm
Image Matching
Remap luminance histograms between source and target image
Scale and shift distributions globally
Luminance remapping, proposed by Hertzman et al, describes the process of matching the luminance histograms of the two images.
The luminance of the target image remains unchanged and the luminance of the source image is shifted and scaled to fit the histogram of the target image.
This transformation brings the histograms of the colour image and the greyscale image into correspondence
Source
-> Scaled
Target
and Shifted

13. Colour Transfer Algorithm
Image Matching
Pre-compute neighbourhood statistics for images
Use sampling techniques to get colour samples from the source image
Find best match from samples
Weighted metric of luminance, mean and standard deviation
Different sampling techniques can be used but the best one is jittered sampling as it reduces the number of samples that need to be made.
For each pixel in the greyscale image find a best match in the colour image – compare the luminance values of the greyscale image and the samples from the colour image
Use weighted average to base decision 50% luminance value 50% std deviation

14. Colour Transfer Algorithm
Transfer the α and β channels only (colour)
After the best match is found only transfer the alpha and beta values while retaining the original luminance value
Extracting luminance information + luminance adjustment + sampling and colour transfer are generic techniques different techniques can be implemented for each one.
Target Image
Colourized Result

15. Swatches User-assisted approach
User selects a small number of swatches
Transfer colour only to swatches
Colour entire target image
Allow more user interaction
User can selectively transfer colours between corresponding swatches
The user selectively chooses the regions to place swatches.
The same colour transfer procedure is used as in the global matching procedure except only between the source and target swatches.

16. Swatches Select swatches = + Source Target Final
Advantage is the selective colour transfer between the swatches which prevents the pixels with similar neighbourhood statistics but from the wrong part of the image from corrupting the target swatch colours.
Swatches allow for interactive colouring and give the user the power to colour areas that do not correspond in composition
The use of swatches reduces the number of comparisons needed to be made by the greyscale image to find the perfect match.
The comparisons are thus only made between the relating swatches.
The rest of the corresponding region is coloured in using texture synthesis
Expect better results between swatches
Expect better matching within an image
Source
Target
Final

17. Design
User Interface
The plug-in is experimental designed in an extensible way – release the source code so that it can be perfected
– extensible because of the way it has been designed
GIMP 1.2 works under Linux and Windows (cross platform)

18. Design
Extensible – can try deifferent matching and sampling techniques….

19. Implementation
Environment
Plug-in Architecture
Development Issues

20. Implementation Environment
GIMP – is the GNU Image Manipulation Package
Open source – free 
Suitable for image retouching, altering and image composition.
Uses external modules known as plug-ins
GIMP caters for C and Pearl
Our choice was based on prior knowledge, available resources and the functionality that the language offers.
Pearl is a scripting language and is not widely used. In terms of learning there is not much written in Pearl.
C is a low level, powerful language with a large and diverse user base.
GIMP is an open source image manipulation package.
It has most of the functionality other image manipulation packages do

21. Implementation Plug-in definition:
“A file containing data used to alter, enhance or extend the operation of a parent application program”. Free On-line Dictionary of Computing

22. Implementation Plug-in Architecture Implement a specified interface
Plug-in must
Register itself in the Procedure Database PDB
Have a main() function
Signal handling
Communication
PLUG_IN_INFO – global variable
query() – call to gimp_install_procedure
run() – sets the status code return value
For GIMP to invoke a developed plug-in the plug-in needs to implement a specified interface
GIMP interfaces with plug-ins by searching for files of the appropriate format and incorporating them into its menu structure.
The search is only done across the plug-in sub-directories.
The position where the plug-in is incorporated into the GIMP structure is defined within the plug-in
Gimp plug-ins are queried on start up and this when they register themselves in the PDB
Main() – is the first function that is called when the program is executed
sets up signal handling and communication between gimp and the plug-in
PLUG_IN_INFO – global variable used by the query procedure
consists of pointers to four functions
run() procedure makes sure that the plug-in is called correctly and that the return values are set correctly
query() – call to gimp_install_procedure

23. Implementation Development Issues GIMP API poorly documented
Wade through gimplib source
Trial and Error implementation
Negatively affected development timescales
Experiences with C
GUI development using GTK tedious
Lack of Documentation is often synonymous with open source development 
Frequently had to adopt a trial and error development style in order to discover the exact behaviour of functions in GIMP lib
GIMP handling of pixels is byte level – wrapper data types that deal with pixels and pixel areas

24. Results = +
Using grid sampling – divide image into grid and sample from middle of each grid

25. Results + =

26. Conclusion Application Enhance Scientific Data
Medical Imaging
Satellite Images
Scanning Electron Microscopy
Colourize black/white photographs and movies
Artistic Effects

27. Conclusion Future Work Robustness: more sophisticated matching
Colour correction
Automated choice of source image
Different metric for texture matches and similarity – allow for small surface colouring

28. Conclusion
Implemented a new general, fast and user-friendly approach to colouring greyscale images

29. Questions? ?