1. The Art of Digital Image processing
C. S. Tong
Department of Mathematics
Hong Kong Baptist University

2. Is the left center circle bigger?
No, they're both the same size

3. It's a spiral, right?
No, these are a bunch of independent circles

4. Keep staring at the black dot
Keep staring at the black dot. After a while the gray haze around it will appear to shrink.

5. Can you find the dog?

6. How many colors do you see?
There are only 3 colors: White, green, and pink. There seem to be two different shades of pink, but there is only one pink.

7. Count the black dots! :o)

8. Are the horizontal lines parallel or do they slope?

9. Do you see a musician or a girl's face?

10. Do you see the face? Or an Eskimo?

11. Do you see a cube missing a corner
Do you see a cube missing a corner? Or do you see a small cube in a big one?

12. Is the blue on the inner left back or the outer left front?

13. What is a digital image?
A digital image is just a 2D array of picture elements (pixels)

14. What is a digital image?
Each pixel is associated with a number which represents its intensity or brightness
Usually allow up to 256 levels of brightness (so called 8-bit images)
how many levels do you think you can distinguish?

15. Effects of Quantization
Effects of changing intensity resolution
8-Bit image
2-Bit image
3-Bit image
1-Bit image
6-Bit image
7-Bit image
5-Bit image
4-Bit image

16. Effects of Quantization
The demo showed that the human eye can only resolve about grey levels

17. What is a digital image?
The density of pixels significantly affect the quality of the image
A typical scanner or digital camera has a resolution of about 600 dpi (or about 1 million pixels per picture)
By comparison, the human eye has a resolution of about 10,000 dpi (or 100 million cone cells)

18. Effects of Quantization
Effects of changing spatial resolution

19. Effects of Quantization
Can be used for concealing identify

20. What is a digital image?
Colour can be represented by three primary colour components: Red, Green and Blue  24-bit RGB images
For special editing effects such as transparency, some image formats support 32-bit RGB- , the additional 8-bit describes the  channel
Video is just a sequence of images. Frame rate of over 24 pictures per second is often sufficient

21. What is a digital image?
A more efficient image format for representing colours is the Index Image Format
All the distinct colours that appear in an image are stored in a file called the colormap
The colour image is now an array of indices, each of which specify the color of that pixel as the corresponding colour in the colormap

22. Editing Colormap

23. Editing Colormap

24. Chroma-keying
The idea of editing the colormap can be used for many movie effects
Take pictures of an actor in front of a blue screen
Edit the colormap and make the blue color transparent
Overlay the pictures to a desired background

25. Chroma-keying
Map the black background to the Tsing Ma Bridge

26. Editing Colormap Increase intensity in the Red component
Convert image to
black and white image
Increase intensity in the Blue component

27. Digital Negative

28. Contrast Stretching Contrast adjusted plus cropping Contrast adjusted
Histogram
Original image

29. Histogram Equalization
Histogram Equalized
Contrast Adjusted
Original image

30. Median Filtering
Original Image
MF (3-by-3)
5% Binary Noise

31. Median Filtering 20% Binary Noise 50% Binary Noise MF (3-by-3)

32. Independent Component Analysis
Denoising using ICA
Original Image
Noisy Image

33. Edge Detection Original Sobel Noise (0.01) Sobel Laplacian

34. High-boost Filter
High-passed
High-boost
Original
Low-passed

35. Fourier Transform
Frequency Domain
Spatial Domain
F
logF

36. Ghost-buster
Ghost appears
Ghost removed

37. Image Degradation
Perfect Photo
Blurred Photo

38. Blur removed using Wiener Filter (nsr=0.05)
Image Restoration
Blur removed using Wiener Filter (nsr=0.05)
Motion Blurred Image
Original Image

39. Blur removed using nsr=0.001
Image Restoration
WF restorations
CST restorations
Original
Blur removed using nsr=0.001
Blur removed using nsr=0.005
Blur removed using nsr=0.01
Blur removed using nsr=0.05
Blur removed using nsr=0.1

40. Result
Restored Image
Blurred Image

41. Other Blurring Function
Restored Image
Horizontal Blurred
Vertical Blurred

42. How to Recognize Shapes?
After appropriate translation, rotation , and scaling, we can now see the two shapes are the same!

43. How to Recognize Shapes?
After all possible translation, rotation , and scaling, we can now see the two shapes are not the same!

44. Pattern Recognition: Overview
Each pattern to be related to a set of features (feature vector)
Distinguish a set of patterns by some measure of distance between feature vectors

45. Feature Extraction
This is the most crucial part of a recognition system
Usually prefer features which are invariant to translation, rotation and scaling
Standard approach include: statistical moments and PCA
Very much context-dependent

46. A small scale illustration
Patterns
Features
Apples, Lemons
Colour
Size
+ Melons
Shape
+ Bananas
+ Oranges, Grape Fruits...
Texture...

47. Complexity
Clearly, as the set of patterns grows, the number and complexity of the features grow
There may not be any suitable distinguishing features
Sometimes I can’t even read my own hand writing!

48. Chinese Character Recognition
There are over 20,000 Chinese Characters
Although not all are in common usage, at least 5,000 are needed in most applications
Chinese Characters come in many font types

49. Chinese Character Recognition
For fixed font character recognition, each character is represented by a N-by-M binary matrix (typically 24-by-24)
Or equivalently, a character is a 576 dimensional vector
Noise in scanning is modelled by bit-reversal (so called binary noise)

50. Chinese Character Recognition
Handwriting is much more difficult: no natural representation available
A character involves combining a number of elementary strokes in two spatial dimension
Large variation in writing styles

51. Effects of Noise
Binary noise at 0, 2, 4, 6, 8, 10% level

52. Regional decomposition
Partition the character into 9 sub-regions and extract the mean intensity of each sub-region
This yields a 9-component feature vector describing the local distribution of “ink” or weight of the character

53. Projection Codes
Stroke information is obtained by projecting the character onto the horizontal and vertical axes
Each projection profile is divided into 3 parts; and the maximal projection value in each part is extracted to give a 6-component feature vector describing the (global) distribution of strokes

54. Projection Codes

55. Local & Global Features
The local and global features are combined into a 15-component code vector
Further transformed to enhance the entropy of the code so as to improve the discriminating power of the codes

56. Why Max-Entropy?
A code value that occurs frequently is useless as it does not discriminate
A code value that occurs infrequently is highly discriminative but only when it occurs (which is rare!)
Thus the most discriminative code is one with a uniform distribution

57. Results (Local & Global Features)
Order-N recognition means the character is correctly identified as one of the top N matches

58. Results (Local & Global Features)
Not good for outright recognition (low order-1 recognition rate)
Quite good for classifying the character as belonging to a small group of characters (because the higher order recognition rates rapidly converge to 1 as order increases)

59. ICA: Character Recognition
Use ICA to remove noise from noisy input
Compare processed character from characters in the dictionary
Identify the character as the one with the best match (1-norm)

60. ICA: Character Recognition
Good recognition even for very high noise level
The Bell & Sejnowski implementation is too slow as it involves the inverses of large matrices, especially when the dictionary is large

61. Two-Stage Approach
Use local & global features in stage 1 to reduce the effective dictionary to a much smaller set
Use ICA in stage 2 to complete the identification of the character
Identify noisy input with the character in the dictionary with the best match (1-norm)

62. Results (2-Stage Approach)

63. Results (benchmark)

64. Results

65. Middle: interpolation
MORPHING EFFECTS
Start
Middle: interpolation
End

66. MORPHING EFFECTS
Start: Human Head
Middle: morph
End: Orangutan

67. MORPHING EFFECTS Start: Lion Middle: morph End: Horse
See website

68. MORPHING EFFECTS View morphing
See website

69. End of Presentation