1. Image Pattern Recognition and Its Applications Chaur-Chin Chen ( 陳朝欽 ) Institute of Information Systems & Applications (Department of Computer Science) National Tsing Hua University HsinChu ( 新竹 ), Taiwan ( 台灣 ) cchen@cs.nthu.edu.tw May 3, 2013

2. Outline Fundamental Image Processing Fingerprint and Face Verification Supervised vs. Unsupervised Learning Watermarking and Steganography Microarray Image Analysis Some Other Application

3. Outline (Continuation) Some Other Applications Supervised vs. Unsupervised Learning Data Description and Representation 8OX and iris Data Sets Dendrograms of Hierarchical Clustering PCA vs. LDA A Comparison of PCA and LDA

4. Fundamental Image Processing ♪ A Digital Image Processing System Image Representation and Formats 1. Sensing, Sampling, Quantization 2. Gray level and Color Images 3. Raw, RGB, Tiff, BMP, JPG, GIF, (JP2) Image Transform and Filtering Histogram, Enhancement Segmentation, Edge Detection, Thinning Image Data Compression Fingerprint and Face Recognition Image Pattern Recognition Watermarking and Steganography Microarray Image Data Analysis [1] R.C. Gonzalez, R.E. Woods, S.L. Eddins, Digital Image Processing Using MATLAB, Pearson Prentice Hall, 2004 [2] R.C. Gonzalez and R.E. Woods, Digital Image Processing, Prentice-Hall, 2002+

5. Image Processing System A 2D image is nothing but a mapping from a region to a matrix A Digital Image Processing System consists of 1. Acquisition – scanners, digital camera, ultrasound, X-ray, MRI, PMT 2. Storage – HD (500GB, TeraBytes, PeraBytes, …), CD (700 MB), DVD (4.7 GB), Flash memory (2~32 GB) 3. Processing Unit – PC, Workstation (Sun Microsystems), PC-cluster 4. Communication – telephone lines, cable, wireless, Wi-Fi, LTE 5. Display – LCD monitor, laser printer, smart phone, i-Pad

6. Illustration of Image Processing System

7. Gray Level and Color Images

8. Pixels in a Gray Level Image

9. A Gray Level Image is a Matrix f(0,0) f(0,1) f(0,2) …. …. f(0,n-1) f(1,0) f(1,1) f(1,2) …. …. f(1,n-1)... f(m-1,0) f(m-1,1) f(m-1,2) … …. f(m-1,n-1) An image of m rows, n columns, f(i,j) is in [0,255]

10. Image Representation (Gray/Color) A gray level image is usually represented by an M x N matrix whose elements are all integers in {0,1, …, 255} corresponding to brightness scales A color image is usually represented by 3 M x N matrices whose elements are all integers in {0,1, …, 255} corresponding to 3 primary primitives of colors such as Red, Green, Blue

11. Gray and Color Image Data 0, 64, 144, 196, 225, 169, 100, 36 (R, G, B) for a color pixel Red – (255, 0, 0) Green – ( 0, 255, 0) Blue – ( 0, 0, 255) Cyan – ( 0,255, 255) Magenta – (255, 0, 255) Yellow – (255, 255, 0) Gray – (128, 128, 128)

12. RGB Hex Triplet Color Chart Red = FF0000 Green = 00FF00 Blue = 0000FF Cyan = 00FFFF Magenta= FF00FF Yellow = FFFF00

13. Koala and Its RGB Components

14. (R,G,B) Histograms of Koala

15. Sensing, Sampling, Quantization A 2D digital image is formed by a sensor which maps a region to a matrix Digitization of the spatial coordinates (x,y) in an image function f(x,y) is called Sampling Digitization of the amplitude of an image function f(x,y) is called Quantization

16. Sampling and Quantization

17. Image File Formats (1/2) The American National Standards Institute (ANSI) sets standards for voluntary use in US. One of the most popular computer standards set by ANSI is the American Standard Code for Information Interchange (ASCII) which guarantees all computers can exchange text in ASCII format BMP – Bitmap format from Microsoft uses Raster-based 1~24-bit colors (RGB) without compression or allows a run-length compression for 1~8-bit color depths GIF – Graphics Interchange Format from CompuServe Inc. is Raster-based which uses 1~8-bit colors with resolutions up to 64,000*64,000 LZW (Lempel-Ziv-Welch, 1984) lossless compression with the compression ratio up to 2:1

18. Some Image File Formats (2/2) Raw – Raw image format uses a 8-bit unsigned character to store a pixel value of 0~255 for a Raster-scanned gray image without compression. An R by C raw image occupies R*C bytes or 8RC bits of storage space TIFF – Tagged Image File Format from Aldus and Microsoft was designed for importing image into desktop publishing programs and quickly became accepted by a variety of software developers as a standard. Its built-in flexibility is both a blessing and a curse, because it can be customized in a variety of ways to fit a programmer’s needs. However, the flexibility of the format resulted in many versions of TIFF, some of which are so different that they are incompatible with each other JPEG – Joint Photographic Experts Group format is the most popular lossy method of compression, and the current standard whose file name ends with “.jpg” which allows Raster-based 8-bit grayscale or 24-bit color images with the compression ratio more than 16:1 and preserves the fidelity of the reconstructed image EPS – Encapsulated PostScript language format from Adulus Systems uses Metafile of 1~24-bit colors with compression JP2 - JPEG 2000 based on 5/3 and 9/7 wavelet transforms

19. Image Transforms and Filtering Feature Extraction – find all ellipses in an image Bandwidth Reduction – eliminate the low contrast “coefficients” Data Reduction – eliminate insignificant coefficients of Discrete Cosine Transform (DCT), Wavelet Transform (WT) Smooth filtering can get rid of noisy signals

20. Discrete Cosine Transform Partition an image into nonoverlapping 8 by 8 blocks, and apply a 2d DCT on each block to get DC and AC coefficients. Most of the high frequency coefficients become insignificant, only the DC term and some low frequency AC coefficients are significant. Fundamental for JPEG Image Compression

21. Discrete Cosine Transform (DCT) X: a block of 8x8 pixels A=Q 8 : 8x8 DCT matrix as shown above Y=AXA t

22. Quantized DCT Coefficients on a 8x8 Block

23. Lenna Image vs. Compressed Lenna

24. Wavelet Transform Haar, Daubechies’ Four, 9/7, 5/3 transforms 9/7, 5/3 transforms was selected as the lossy and lossless coding standards for JPEG2000, respectively A Comparison of JPEG and JPEG2000 shows that the latter is slightly better than the former, however, to replace image.jpg by image.jp2 needs time

25. 3-Scale Wavelet Transforms

26. Mean and Median Filtering X1 X2 X3 X4 X0 X5 X6 X7 X8 Replace the X0 by the mean of X0~X8 is called “mean filtering” X1 X2 X3 X4 X0 X5 X6 X7 X8 Replace the X0 by the median of X0~X8 is called “median filtering”

27. Example of Median Filtering

28. Image and Its Histogram

29. Enhancement and Restoration The goal of enhancement is to accentuate certain features for subsequent analysis or image display. The enhancement process is usually done interactively The restoration is a process that attempts to reconstruct or recover an image that has been degraded by using some unknown phenomenon

30. Example of Image Enhancement Support that A(i, j) is image gray level at pixel (i, j), μ and s 2 are the mean and variance of gray levels of input image, and α=150, γ=95, γ must satisfy γ>s. The enhanced image B( i, j ) is obtained by a contrast stretching given below B( i, j )  α + γ ＊ ([A ( i, j ) – μ] ／ s)

31. Result of Image Enhancement

32. Segmentation and Edge Detection Segmentation is basically a process of pixel classification: the picture is segmented into subsets by assigning the individual pixels into classes Edge Detection is to find the pixels whose gray values or colors being abruptly changed

33. Image Lenna and Its Histogram

34. Image Segmentation Algorithms Otsu (1979) Fisher (1936) Kittler and Illingworth (1986) Vincent and Soille (1991) Besag, Chen and Dubes (1986, 1991)

35. A Simple Thresholding Algorithm (1)

36. Image, Histogram, Thresholding

37. Binarization by Thresholding

38. ICM Segmentation Algorithm 1. Given an image Y, initialize a labeling X 2. For t=1:mxn X(t)←g 0 if Pr(X(t)=g 0 |X N(t),Y) > Pr(X(t)=g|X N(t),Y) for g,g 0 3. Repeat step 2 until “convergence” (6 runs) 4. X is the required labeling Chaur-Chin Chen and Richard C. Dubes Environmental Studies and ICM Segmentation Algorithm, Journal of Information Science and Engineering, Vol. 6, 325-337, 1990.

39. Image Segmentation: ICM vs. Otsu

42. Edge Detection -1 -2 -1 0 0 0  X 1 2 1 -1 0 1 -2 0 2  Y -1 0 1 Large (|X|+|Y|)  Edge

43. Thinning and Contour Tracing Thinning is to find the skeleton of an image which is commonly used for Optical Character Recognition (OCR) and Fingerprint matching Contour tracing is usually used to locate the boundaries of an image which can be used in feature extraction for shape discrimination

44. Image  Edge, Skeleton, Contour

45. Image Data Compression The purpose is to save storage space and to reduce the transmission time of information. Note that it requires 6 mega bits to store a 24-bit color image of size 512 by 512. It takes 6 seconds to download such an image via an ADSL (Asymmetric Digital Subscriber Line) with the rate 1 mega bits per second and more than 12 seconds to upload the same image Note that 1 byte = 8 bits, 3 bytes = 24 bits

46. Training Images for VQ

47. LBG Algorithm for Codebook Generation

48. Codebook and Decoded Images

49. Some Applications Fingerprint and Face Recognition Watermarking and Steganography Image Pattern Recognition Microarray Image Data Analysis

50. 美國啟用出入境指紋及人臉影像辨 識系統 美國國土安全部基於安全考慮，自 (2004) 元 月五日起，啟用數位化出入境身分辨識系 統 (US-VISIT) ，大部分來美的 14 歲至 79 歲 旅客，包括來自台灣、大陸、香港的留學 生，於進入美國國際機場及港口時，都要 接受拍照及留下指紋掃描紀錄以便辨識查 核。 (27 個免簽證國公民之入境待遇略有不 同，短期來美者，將受豁免。 ) ，亦將需接 受指紋掃描查核。

51. US-VISIT US-VISIT currently applies to all visitors (with limited exemptions) holding non- immigrant visas, regardless of country of origin. 2004 – US$ 330 million 2005 – US$ 340 million 2006 – US$ 340 million 2007 – US$ 362 million 2009 – US$ ??? million

52. 入境按指紋 日本 2007/11/20 實施 日本入境排隊長 指紋掃瞄會更長！ (2007 年 9 月 27 日 ) 日本入境排隊長 指紋掃瞄會更長！ 入境日本將按指紋 日官員赴台宣導新措施 (2007 年 9 月 27 日 ) 入境日本將按指紋 日官員赴台宣導新措施 日 11 月 20 日實施外國人入境須按指紋臉部 照片 (2007 年 9 月 25 日 ) 日 11 月 20 日實施外國人入境須按指紋臉部 照片 入境按指紋 日本 11 月將實施 (2007 年 9 月 2 日 ) 入境按指紋 日本 11 月將實施

53. A Typical Fingerprint Image

55. Flowchart of An AFIS

56. (a) Original image (b) Enhanced image (c) Binarization image (d) Smoothed image

57. Thinning [9] The purpose of thinning stage is to gain the skeleton structure of a fingerprint image. It reduces a binary image consisting of ridges and valleys into a ridge map of unit width. (d) Smoothed image (e) Thinned image

58. Minutiae Definition ♫ From a thinned image, we can classify each ridge pixel into the following categories according to its 8-connected neighbors. ♫ A ridge pixel is called : an isolated point if it does not contain any 8-connected neighbor. an ending if it contains exactly one 8-connected neighbor. an edgepoint if it has two 8-connected neighbors. a bifurcation if it has three 8-connected neighbors. a crossing if it has four 8-connected neighbors.

59. Example of Minutiae Extraction

60. Minutiae Pattern Matching

61. Is this Lady in your database?

62. Part of 5*40 Training Face Images

63. Missed Face Images and Their Wrongly-Best Matched Images

64. Are They the Same Person?

65. Challenges and Opportunities A perfect biometric recognition system did not exist and will never exists An application based on biometrics usually requests a perfect verification/identification A collection of biometric data is usually time consuming and more or less intrudes personal privacy The mechanism of achieving the trade-off between privacy and security merits studies.

66. Supervised Learning Problems ☺ The problem of supervised learning can be defined as to design a function which takes the training data x i (k), i=1,2, …n i, k=1,2,…, C, as input vectors with the output as either a single category or a regression curve. ☺ The unsupervised learning (Cluster Analysis) is similar to that of the supervised learning problem (Pattern Recognition) except that the categories are unknown in the training data.

67. Distinguish Eggplants from Bananas 1. Features(characteristics) Colors Shapes Size Tree leaves Other quantitative measurements 2. Decision rules: Classifiers 3. Performance Evaluation 4. Classification

68. Possum, Dingo, Fox, Wombat

69. Watermarking and Steganography Watermarking is the practice of hiding a message about an image, audio clip, video clip, or other work of media within that work itself. Steganography is the art of writing in cipher, or in character, which are not intelligible except to persons who have the key. In computer terms, steganography has evolved into the practice of hiding a message within a larger one in such a way that others cannot discern the presence or contents of the hidden message.

70. Examples of Watermarking and Steganography

71. Difference between Watermarking and Steganography Watermarking Insert a logo, pattern, a message, and etc. into an image, audio, video to claim the ownership. Steganography Put a cover image, audio, video, and etc. on a secret message to protect the secrecy during the transmission.

72. An Example of Steganography The Precious Night by Tsui Ping The southern winds lightly kiss my face, with the heavy scent of blossms The southern winds lightly kiss my? face, but the stars are sparse and the moon veiled We lie against each other, exchanging endless words of love We lie against each other, meaning everything we say We don't care that tomorrow we may bid each other farewell But remember tonight, and treasure it On the eve of parting, we rue the sun's imminent rising Lingering before parting, we promise to meet in a dream

73. Microarray Image Data Analysis

74. Each gene expression is a feature which is measured as average spot brightness Top: Tumor Tissues Bottom: Normal Tissues

75. Bar Code and QR code

76. Face and Fingerprint Images

77. License Plate

78. Fort San Domingo ( 淡水紅毛城 ) Entrance GateDutch Clogs

79. iGoogle APP Facebook LinkedIn Twitter Android APP iPhone App Newsletter RSS Feeds

80. Thank You For Your Attention Questions and Comments