1. Why D.I.P.? Reasons for compression –Image data need to be accessed at a different time or location –Limited storage space and transmission bandwidth Reasons for manipulation –Image data might experience nonideal acquisition, transmission or display (e.g., restoration, enhancement and interpolation) –Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) –To produce images with artistic effect (e.g., pointellism) Reasons for analysis –Image data need to be analyzed automatically in order to reduce the burden of human operators –To teach a computer to “see” in A.I. tasks

2. Beyond Image Processing The way of thinking –From art (heuristics) to science (principles) –The key is mathematics (I will write a separate blog about the role of mathematics in DIP) The holistic view –Everything is connected (recall the six-degree phenomenon in social science) The “Google”-style re-search –Ability to search is a basic part of learning

3. Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics

4. The Art of Image Compression Why are images compressible? –Redundancy in images (NOT random) How data compression works? –Probability theory and statistics –Shannon’s information theory What about the future of image compression? – I will discuss this in my weblog and facebook (Google “ what happened to Iterated Systems Incorporated ?”)

5. Shannon’s Picture on Communication (1948) source encoder channel source decoder sourcedestination Examples of source: Human speeches, photos, text messages, computer programs … Examples of channel: storage media, telephone lines, wireless transmission … super-channel channel encoder channel decoder The goal of communication is to move information from here to there and from now to then

6. Lossless vs. Lossy Compression Lossless: zero error tolerance –No information loss –Shannon’s entropy formula –For photographic images, compression ratio is modest (about 2:1) Lossy: the goal is to preserve the visual quality of images –Information loss visually acceptable –Shannon’s rate-distortion function –For photographic images, compression ratio is typically around 10-100

7. Popular Lossless Image Compression Techniques  WinZip - Based on the celebrated Lempel-Ziv algorithm invented nearly 30 years ago -Based on an enhanced version of LZ algorithm by Welch in 1983 -Was introduced by CompuServe in 1987 and made popular until it was not royalty-free in 1994  GIF (Graphic Interchange Format)  PNG (Portable Network Graphics) GIF Liberation Day: June 20, 2003

8. Lossy Image Compression JPEG decoder original raw image (262,144 bytes) compressed JPEG file (20,407 bytes) decompressed image high compression ratio low compression ratio low quality high quality Q Q 100 0

9. From JPEG to JPEG2000 JPEG (CR=64)JPEG2000 (CR=64) discrete cosine transform basedwavelet transform based

10. Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform

11. salt and pepper (impulse) noise Image Manipulation (I): Noise Removal Noise contamination is often inevitable during the acquisition additive white Gaussian noise You will learn how to design image filter in a principled way

12. License plate is barely legible due to motion blurring Image Manipulation (II): Deblurring You will learn the use of FT and the necessity of regularization

13. overly-exposed image Image Manipulation (III): Contrast Enhancement under-exposed image You will learn how to modify the histogram of an image

14. Example: aliasing artifacts in MRI image acquisition Tradeoff between scanning time and image quality (image reconstruction is covered by EE425) Ideal quality, slow scanning nonideal quality, fast scanning Image Manipulation (IV): Aliasing Reduction

15. small large digital zooming 1M pixels 4M pixels Resolution enhancement can be obtained by common image processing software such as Photoshop or Paint Shop Pro Image Manipulation (V): Image Interpolation You will learn the difference between digital and optical zooming

16. F.Y.I.: search “Gigapixel images” by Google http://triton.tpd.tno.nl/gigazoom/Delft2.htm = + Merge multiple images of the same scene into one with larger FOV Image Manipulation (VI): Image Mosaicing There exist several mosaicing software for automatic stitching

17. blocks contaminated by channel errors (this problem is covered in EE565) Image Manipulation (VII): Error Concealment

18. Block artifacts Image Manipulation (VIII): Deblocking/Deringing Ringing artifacts You will learn how to suppress those artifacts by nonlinear diffusion

19. jittering noise (you will see it in either bonus assignment or final project) Image Manipulation (IX): Dejittering

20. Image Manipulation (X): Image Inpainting You will learn how to do image inpainting in EE565

21. Image Inpainting Application: Restore Old Photos

22. 25,680 colors (24 bits)256 colors (8 bits) Applications: video cell-phone, gameboy, portable DVD Image Manipulation (XI): Color Quantization

23. grayscale: 0-255halftoned: 0/255 Image Manipulation (XII): Image Halftoning You will learn the famous Floyd-Steinberg diffusion in CA

24. original scrambled Image Manipulation (XIII): Image Scrambling/Hashing*

25. Original imageModified image Image Manipulation (XIV): Image Watermarking*

26. Image Manipulation (XV): Image Stylization*

27. Abyss computer generated Image Manipulation (XVI): Image Rendering*

28. Image-based Rendering*

29. Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology

30. Image Analysis (I): Edge Detection You will learn basic edge detectors based on derivatives

31. Image Analysis (II): Face Detection Deceivingly simple for humans but notoriously difficult for machines

32. Image Analysis (III): Change Detection

33. Change Detection in Medical Application

34. Image Analysis (IV): Image Matching Antemortem dental X-ray recordPostmortem dental X-ray record

35. Image Matching in Biometrics Two deceivingly similar fingerprints of two different people

36. Image Analysis (V): Image Segmentation

37. License number can be automatically extracted from the image of license plate Image Analysis (VI): Object Recognition

38. Object Recognition in Military Applications

39. Image-based monitoring system prevents drowning Image Analysis (VII): Event Recognition

40. Only send out “important” motion pictures such as home-runs Image Analysis (VIII): Video Summarization

41. retrieved building images Image Analysis (IX): Content-based Image Retrieval

42. Summary In EE465, you will learn –Image compression: Lempel-Ziv, Huffman coding, run-length coding and JPEG –Image manipulation: linear/nonlinear filtering, histogram-based processing, linear interpolation –Image analysis: edge/corner detection, circle/ line/ellipse detection, chain codes/shape numbers In EE565, you will learn –Advanced algorithms/techniques with stronger mathematical emphasis Not covered by the courses I offered –CBIR (multimedia database), face/pedestrian detection (Advanced biometrics), PDE-based image processing (medical image analysis)

43. General Cooking Recipe Motivations Problem statement Heuristic observations Examples/Illustrations Principled approaches (via mathematics) MATLAB implementations Computer assignment reviews Current state-of-the-art and future directions