Why D.I.P.? Reasons for compression –Image data need to be accessed at a different time or location –Limited storage space and transmission bandwidth Reasons for manipulation –Image data might experience nonideal acquisition, transmission or display (e.g., restoration, enhancement and interpolation) –Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) –To produce images with artistic effect (e.g., pointellism) Reasons for analysis –Image data need to be analyzed automatically in order to reduce the burden of human operators –To teach a computer to “see” in A.I. tasks

Beyond Image Processing The way of thinking –From art (heuristics) to science (principles) –The key is mathematics (I will write a separate blog about the role of mathematics in DIP) The holistic view –Everything is connected (recall the six-degree phenomenon in social science) The “Google”-style re-search –Ability to search is a basic part of learning

Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics

The Art of Image Compression Why are images compressible? –Redundancy in images (NOT random) How data compression works? –Probability theory and statistics –Shannon’s information theory What about the future of image compression? – I will discuss this in my weblog and facebook (Google “ what happened to Iterated Systems Incorporated ?”)

Shannon’s Picture on Communication (1948) source encoder channel source decoder sourcedestination Examples of source: Human speeches, photos, text messages, computer programs … Examples of channel: storage media, telephone lines, wireless transmission … super-channel channel encoder channel decoder The goal of communication is to move information from here to there and from now to then

Lossless vs. Lossy Compression Lossless: zero error tolerance –No information loss –Shannon’s entropy formula –For photographic images, compression ratio is modest (about 2:1) Lossy: the goal is to preserve the visual quality of images –Information loss visually acceptable –Shannon’s rate-distortion function –For photographic images, compression ratio is typically around

Popular Lossless Image Compression Techniques  WinZip - Based on the celebrated Lempel-Ziv algorithm invented nearly 30 years ago -Based on an enhanced version of LZ algorithm by Welch in Was introduced by CompuServe in 1987 and made popular until it was not royalty-free in 1994  GIF (Graphic Interchange Format)  PNG (Portable Network Graphics) GIF Liberation Day: June 20, 2003

Lossy Image Compression JPEG decoder original raw image (262,144 bytes) compressed JPEG file (20,407 bytes) decompressed image high compression ratio low compression ratio low quality high quality Q Q 100 0

From JPEG to JPEG2000 JPEG (CR=64)JPEG2000 (CR=64) discrete cosine transform basedwavelet transform based

Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform

salt and pepper (impulse) noise Image Manipulation (I): Noise Removal Noise contamination is often inevitable during the acquisition additive white Gaussian noise You will learn how to design image filter in a principled way

License plate is barely legible due to motion blurring Image Manipulation (II): Deblurring You will learn the use of FT and the necessity of regularization

overly-exposed image Image Manipulation (III): Contrast Enhancement under-exposed image You will learn how to modify the histogram of an image

Example: aliasing artifacts in MRI image acquisition Tradeoff between scanning time and image quality (image reconstruction is covered by EE425) Ideal quality, slow scanning nonideal quality, fast scanning Image Manipulation (IV): Aliasing Reduction

small large digital zooming 1M pixels 4M pixels Resolution enhancement can be obtained by common image processing software such as Photoshop or Paint Shop Pro Image Manipulation (V): Image Interpolation You will learn the difference between digital and optical zooming

F.Y.I.: search “Gigapixel images” by Google = + Merge multiple images of the same scene into one with larger FOV Image Manipulation (VI): Image Mosaicing There exist several mosaicing software for automatic stitching

blocks contaminated by channel errors (this problem is covered in EE565) Image Manipulation (VII): Error Concealment

Block artifacts Image Manipulation (VIII): Deblocking/Deringing Ringing artifacts You will learn how to suppress those artifacts by nonlinear diffusion

jittering noise (you will see it in either bonus assignment or final project) Image Manipulation (IX): Dejittering

Image Manipulation (X): Image Inpainting You will learn how to do image inpainting in EE565

Image Inpainting Application: Restore Old Photos

25,680 colors (24 bits)256 colors (8 bits) Applications: video cell-phone, gameboy, portable DVD Image Manipulation (XI): Color Quantization

grayscale: 0-255halftoned: 0/255 Image Manipulation (XII): Image Halftoning You will learn the famous Floyd-Steinberg diffusion in CA

original scrambled Image Manipulation (XIII): Image Scrambling/Hashing*

Original imageModified image Image Manipulation (XIV): Image Watermarking*

Image Manipulation (XV): Image Stylization*

Abyss computer generated Image Manipulation (XVI): Image Rendering*

Image-based Rendering*

Image Compression Image Manipulation Image Analysis Image Acquisition Image Perception Image Display Image Generation D.I.P. Theme Park DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology

Image Analysis (I): Edge Detection You will learn basic edge detectors based on derivatives

Image Analysis (II): Face Detection Deceivingly simple for humans but notoriously difficult for machines

Image Analysis (III): Change Detection

Change Detection in Medical Application

Image Analysis (IV): Image Matching Antemortem dental X-ray recordPostmortem dental X-ray record

Image Matching in Biometrics Two deceivingly similar fingerprints of two different people

Image Analysis (V): Image Segmentation

License number can be automatically extracted from the image of license plate Image Analysis (VI): Object Recognition

Object Recognition in Military Applications

Image-based monitoring system prevents drowning Image Analysis (VII): Event Recognition

Only send out “important” motion pictures such as home-runs Image Analysis (VIII): Video Summarization

retrieved building images Image Analysis (IX): Content-based Image Retrieval

Summary In EE465, you will learn –Image compression: Lempel-Ziv, Huffman coding, run-length coding and JPEG –Image manipulation: linear/nonlinear filtering, histogram-based processing, linear interpolation –Image analysis: edge/corner detection, circle/ line/ellipse detection, chain codes/shape numbers In EE565, you will learn –Advanced algorithms/techniques with stronger mathematical emphasis Not covered by the courses I offered –CBIR (multimedia database), face/pedestrian detection (Advanced biometrics), PDE-based image processing (medical image analysis)

General Cooking Recipe Motivations Problem statement Heuristic observations Examples/Illustrations Principled approaches (via mathematics) MATLAB implementations Computer assignment reviews Current state-of-the-art and future directions