Interactive Optimization by Genetic Algorithms Cases: Lighting Patterns and Image Enhancement Janne Koljonen Electrical Engineering and Automation, University of Vaasa

Outline Interactive Evolutionary Computation (IEC) in general. Image enhancement. LED adaptive luminence lighting system (LEDall). Project work (LEDall).

Interactive optimization by GA In Interactive Evolutionary Computation (IEC), the computational fitness function is replaced by a human evaluator. –In other aspects the genetic algorithm may be as usual.

Application domains of IEC In cases, where the favorable output can be usually evaluated only subjectively, IEC is applied. –Such domains are e.g. music, graphics and image enhancement.

Constraints The human intervention is a bottleneck of IEC system: it is time consuming, difficult and even boring to evaluate outputs of a system. The time constraint and patience of the user can be overcome by limiting the number of fitness evaluations. –The attention has to be paid to the problem complexity and the algorithm so that search strategy can be guided gradually from a global phase into a fine-tune search by the user.

IEC strategies A few strategies for the subjective fitness evaluation have been reported: evaluation score points with n levels, selection of elite, which actually corresponds given score points from 2 levels, and pair-wise tournament. Others? How to compare n outputs in parallel/in sequence?

IEC strategy suggestions In addition to a fitness function, the user can be used to select the genetic operators that should be applied –Requires more expertise Alternatively, GA could have a pool of different operators and a mechanism to learn, which operators are efficient in different cases.

Image enhancement People use image processing tools increasingly as the costs of digital cameras have decreased. However, image processing tools contain nowadays dozens of filters with a few parameters each. –An inexperienced user is barely capable of deciding, which filters and parameters to use. Presumable the method of trial and error is applied in such cases, which is time consuming. –Moreover, rarely one single filter is enough for the desired output but a sequence of filters and integrations of filtered images are required.

Objective Image enhancement and image restoration are usually applied to improve the quality of the pictures or to emphasize certain features and details. The result is another image that meets better the requirements set for the image in a specific application. The difference, by definition, of image enhancement and image restoration is in the output evaluation. –While image enhancement is evaluated subjectively, the objective of restoration is to recover the original image subjected to e.g. noise or other degradation.

Objective Image enhancement and image restoration are usually applied to improve the quality of the pictures or to emphasize certain features and details. The result is another image that meets better the requirements set for the image in a specific application. The difference, by definition, of image enhancement and image restoration is in the output evaluation. –While image enhancement is evaluated subjectively, the objective of restoration is to recover the original image subjected to e.g. noise or other degradation.

Objective The objective of image pre-processing may be e.g. to remove noise from the image, to sharpen the image, to adjust color/gray scale intensities, or to highlight e.g. edges or other features that can be used in segmentation and pattern recognition stages of image analysis. Complex image enhancement and analysis tasks are difficult even for experts. Hence, a method to boost the search for an image processing sequence would be advantageous both for uninitiated and experts.

Applications Evolutionary computing or algorithms (EC/EA) have be applied to partially automate image enhancement, whose output may be subjected to visual inspection or act as the input for further image analysis and pattern recognition stages. Usually, the principle is to combine basic image processing operations drawn from a finite set and to optimize the relations between the operations and the internal parameters of to the operations.

Applications Interactive image enhancement optimization methods have been applied e.g. to magnetic resonance (MR) image pseudo-colorization using genetic programming. It has also been suggested that the user can be modeled to decrease the need for human intervention. Visual image enhancement with a desired output image has been studied by Nagao et al. –the objective was to search for, with a GA, an approximation of the transformation sequence leading to the given output. The desired output can also be defined by objective criteria by the user. Pre-processing optimization as a part of pattern recognition optimization has also been reported in the literature –experiments were done with radar signals.

LEDall Koljonen et al. (2004) have developed an interactive LED lighting system to optimize illumination pattern in close range optical imaging. An I/O board with digital voltage outputs controls 90 LEDs that are set around the object to be imaged. –Different lighting patterns can be searched for to enhance different features of the image. Shadows, illumination levels, etc.

PWM control Since cameras (and human eye) have a relative long expose time (time resolution), pulse-width-modulation (PWM) can be used to increase the number of luminance levels of the LEDs. –In LEDall 4 levels are used.  Totally 4 90 lighting combinations allowed! –Q: How to optimize? A: With GA!

Interactive GA Initial population of 9 random lighting patters. Resulting images shown as a 3x3 grid of images. User selects 0-8 images that contain favorable features (parents). Illuminations of the parents are operated by crossover and mutation to create offspring of potentially more favorable illumination. Occationally, new random offspring are created to retain diversity of the population.

Example Random lightingsAfter 3 GA genetations

Applications LEDall or a similar device can be utilized in many places and applications:

Improvements/project work More images, score points? New (user controlled?) genetic operators? More LEDs (LEDall2, PIC, Toni Harju)? Better camera? New applications? Semi-automatic fitness funcition? –Deterministic criteria. I/O and frame grapper routines exist –Native Java functions.

Questions?