The Art of Digital Image processing C. S. Tong Department of Mathematics Hong Kong Baptist University

Is the left center circle bigger? No, they're both the same size

It's a spiral, right? No, these are a bunch of independent circles

Keep staring at the black dot Keep staring at the black dot. After a while the gray haze around it will appear to shrink.

Can you find the dog?

How many colors do you see? There are only 3 colors: White, green, and pink. There seem to be two different shades of pink, but there is only one pink.

Count the black dots! :o)

Are the horizontal lines parallel or do they slope?

Do you see a musician or a girl's face?

Do you see the face? Or an Eskimo?

Do you see a cube missing a corner Do you see a cube missing a corner? Or do you see a small cube in a big one?

Is the blue on the inner left back or the outer left front?

What is a digital image? A digital image is just a 2D array of picture elements (pixels)

What is a digital image? Each pixel is associated with a number which represents its intensity or brightness Usually allow up to 256 levels of brightness (so called 8-bit images) how many levels do you think you can distinguish?

Effects of Quantization Effects of changing intensity resolution 8-Bit image 2-Bit image 3-Bit image 1-Bit image 6-Bit image 7-Bit image 5-Bit image 4-Bit image

Effects of Quantization The demo showed that the human eye can only resolve about 20-30 grey levels

What is a digital image? The density of pixels significantly affect the quality of the image A typical scanner or digital camera has a resolution of about 600 dpi (or about 1 million pixels per picture) By comparison, the human eye has a resolution of about 10,000 dpi (or 100 million cone cells)

Effects of Quantization Effects of changing spatial resolution

Effects of Quantization Can be used for concealing identify

What is a digital image? Colour can be represented by three primary colour components: Red, Green and Blue  24-bit RGB images For special editing effects such as transparency, some image formats support 32-bit RGB- , the additional 8-bit describes the  channel Video is just a sequence of images. Frame rate of over 24 pictures per second is often sufficient

What is a digital image? A more efficient image format for representing colours is the Index Image Format All the distinct colours that appear in an image are stored in a file called the colormap The colour image is now an array of indices, each of which specify the color of that pixel as the corresponding colour in the colormap

Editing Colormap

Editing Colormap

Chroma-keying The idea of editing the colormap can be used for many movie effects Take pictures of an actor in front of a blue screen Edit the colormap and make the blue color transparent Overlay the pictures to a desired background

Chroma-keying Map the black background to the Tsing Ma Bridge

Editing Colormap Increase intensity in the Red component Convert image to black and white image Increase intensity in the Blue component

Digital Negative

Contrast Stretching Contrast adjusted plus cropping Contrast adjusted Histogram Original image

Histogram Equalization Histogram Equalized Contrast Adjusted Original image

Median Filtering Original Image MF (3-by-3) 5% Binary Noise

Median Filtering 20% Binary Noise 50% Binary Noise MF (3-by-3)

Independent Component Analysis Denoising using ICA Original Image Noisy Image

Edge Detection Original Sobel Noise (0.01) Sobel Laplacian

High-boost Filter High-passed High-boost Original Low-passed

Fourier Transform Frequency Domain Spatial Domain F logF

Ghost-buster Ghost appears Ghost removed

Image Degradation Perfect Photo Blurred Photo

Blur removed using Wiener Filter (nsr=0.05) Image Restoration Blur removed using Wiener Filter (nsr=0.05) Motion Blurred Image Original Image

Blur removed using nsr=0.001 Image Restoration WF restorations CST restorations Original Blur removed using nsr=0.001 Blur removed using nsr=0.005 Blur removed using nsr=0.01 Blur removed using nsr=0.05 Blur removed using nsr=0.1

Result Restored Image Blurred Image

Other Blurring Function Restored Image Horizontal Blurred Vertical Blurred

How to Recognize Shapes? After appropriate translation, rotation , and scaling, we can now see the two shapes are the same!

How to Recognize Shapes? After all possible translation, rotation , and scaling, we can now see the two shapes are not the same!

Pattern Recognition: Overview Each pattern to be related to a set of features (feature vector) Distinguish a set of patterns by some measure of distance between feature vectors

Feature Extraction This is the most crucial part of a recognition system Usually prefer features which are invariant to translation, rotation and scaling Standard approach include: statistical moments and PCA Very much context-dependent

A small scale illustration Patterns Features Apples, Lemons Colour Size + Melons Shape + Bananas + Oranges, Grape Fruits... Texture...

Complexity Clearly, as the set of patterns grows, the number and complexity of the features grow There may not be any suitable distinguishing features Sometimes I can’t even read my own hand writing!

Chinese Character Recognition There are over 20,000 Chinese Characters Although not all are in common usage, at least 5,000 are needed in most applications Chinese Characters come in many font types

Chinese Character Recognition For fixed font character recognition, each character is represented by a N-by-M binary matrix (typically 24-by-24) Or equivalently, a character is a 576 dimensional vector Noise in scanning is modelled by bit-reversal (so called binary noise)

Chinese Character Recognition Handwriting is much more difficult: no natural representation available A character involves combining a number of elementary strokes in two spatial dimension Large variation in writing styles

Effects of Noise Binary noise at 0, 2, 4, 6, 8, 10% level

Regional decomposition Partition the character into 9 sub-regions and extract the mean intensity of each sub-region This yields a 9-component feature vector describing the local distribution of “ink” or weight of the character

Projection Codes Stroke information is obtained by projecting the character onto the horizontal and vertical axes Each projection profile is divided into 3 parts; and the maximal projection value in each part is extracted to give a 6-component feature vector describing the (global) distribution of strokes

Projection Codes

Local & Global Features The local and global features are combined into a 15-component code vector Further transformed to enhance the entropy of the code so as to improve the discriminating power of the codes

Why Max-Entropy? A code value that occurs frequently is useless as it does not discriminate A code value that occurs infrequently is highly discriminative but only when it occurs (which is rare!) Thus the most discriminative code is one with a uniform distribution

Results (Local & Global Features) Order-N recognition means the character is correctly identified as one of the top N matches

Results (Local & Global Features) Not good for outright recognition (low order-1 recognition rate) Quite good for classifying the character as belonging to a small group of characters (because the higher order recognition rates rapidly converge to 1 as order increases)

ICA: Character Recognition Use ICA to remove noise from noisy input Compare processed character from characters in the dictionary Identify the character as the one with the best match (1-norm)

ICA: Character Recognition Good recognition even for very high noise level The Bell & Sejnowski implementation is too slow as it involves the inverses of large matrices, especially when the dictionary is large

Two-Stage Approach Use local & global features in stage 1 to reduce the effective dictionary to a much smaller set Use ICA in stage 2 to complete the identification of the character Identify noisy input with the character in the dictionary with the best match (1-norm)

Results (2-Stage Approach)

Results (benchmark)

Results

Middle: interpolation MORPHING EFFECTS Start Middle: interpolation End

MORPHING EFFECTS Start: Human Head Middle: morph End: Orangutan

MORPHING EFFECTS Start: Lion Middle: morph End: Horse See website http://graphics.stanford.edu/cgi-bin/alumni/tolis/personal/getpage.cgi?morph.html

MORPHING EFFECTS View morphing See website http://www.cs.wisc.edu/computer-vision/projects/interp/interp.html

End of Presentation