Edward J. Delp Lucent January 12, 2000 Slide 1 Video and Image Processing At Purdue Edward J. Delp Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA

Edward J. Delp Lucent January 12, 2000 Slide 2 Acknowledgements Students - –Eduardo Asbun –Dan Hintz –Paul Salama –Ke Shen –Martha Saenz –Eugene Lin –Ray Wolfgang –Greg Cook –Sheng Liu

Edward J. Delp Lucent January 12, 2000 Slide 3 Image and Video Processing at Purdue Purdue has a rich history 60 year history in image and video processing.

Edward J. Delp Lucent January 12, 2000 Slide 4 VIPER Research Projects Scalable Video and Color Image Compression –still image compression (CEZW) –high and low bit rate video compression (SAMCoW) –wireless video Error Concealment Content Addressable Video Databases (ViBE) –Scene Change Detection and Identification –Pseudo-Semantic Scene Labeling Multimedia Security: Digital Watermarking

Edward J. Delp Lucent January 12, 2000 Slide 5 VIPER Research Projects Multicast Video Analysis of Mammograms Embedded Image and Video Processing

Edward J. Delp Lucent January 12, 2000 Slide 6 Other Purdue Projects Electronic Imaging - Jan Allebach and Charles Bouman –half-tone printing –compound document compression –image databases Remote Sensing - David Langrebe Medical Imaging - Charles Bouman, Peter Doerschuk, Thomas Talavage, Edward Delp –computed imaging –functional MRI –x-ray crystallography –breast imaging

Edward J. Delp Lucent January 12, 2000 Slide 7 Breast Cancer Second major cause of cancer death among women in the United States (after lung cancer) Leading cause of nonpreventable cancer death 1 in 8 women will develop breast cancer in her lifetime 1 in 30 women will die from breast cancer Evidence seems to indicate that “curable” tumors must be less than 1 cm in diameter

Edward J. Delp Lucent January 12, 2000 Slide 8 Mammography Mammograms are X-ray images of the breast Screening mammography is currently the best technique for reliable detection of early, non-palpable, potentially curable breast cancer Studies show that mammogram can reduce the overall mortality from breast cancer by up to 30%

Edward J. Delp Lucent January 12, 2000 Slide 9 Mammography In the United States, it is recommended that women over 50 years old receive annual mammograms –this probably too late to start Usually 4 views are taken (2 of each breast) –most mammograms are taken using X-Ray film (analog) –digital mammogram systems are now being deployed

Edward J. Delp Lucent January 12, 2000 Slide 10 Screening Mammography

Edward J. Delp Lucent January 12, 2000 Slide 11 A Digital Mammogram (normal)

Edward J. Delp Lucent January 12, 2000 Slide 12 Analysis of Mammograms Density 1Density 2Density 3Density 4

Edward J. Delp Lucent January 12, 2000 Slide 13 Digital Mammography Resolution - 50  pixel size –3000 x 4000 pixels (12,000,000 pixels) –8-16 bits/pixels 8 bits/pixel (12 MB) 16 bits/pixel (24 MB) Each study consists of MB! 200 patients per day can results to 20GB/day Problems with storage and retrieval

Edward J. Delp Lucent January 12, 2000 Slide 14 Three Types of Breast Abnormalities Micro- calcification Circumscribed Lesion Spiculated Lesion

Edward J. Delp Lucent January 12, 2000 Slide 15 Problems in Screening Mammography Radiologists vary in their interpretation of the same mammogram False negative rate is 4 – 20% in current clinical mammography Only 15 – 34% of women who are sent for a biopsy actually have cancer

Edward J. Delp Lucent January 12, 2000 Slide 16 Current Research in Computer Aided Diagnosis (CAD) The goal is to increase diagnostic accuracy as well as the reproducibility of mammographic interpretation Most work aims at detecting one of the three abnormal structures Some have explored classifying breast lesions as benign or malignant The implementation of CAD systems in everyday clinical applications will change the practice of radiology

Edward J. Delp Lucent January 12, 2000 Slide 17 Multiresolution Detection of Spiculated Lesions in Digital Mammograms Spiculation or a stellate appearance in mammograms indicates with near certainty the presence of breast cancer Detection of spiculated lesions is very important in the characterization of breast cancer

Edward J. Delp Lucent January 12, 2000 Slide 18 Block Diagram of Multiresolution Detection of Spiculated Lesions

Edward J. Delp Lucent January 12, 2000 Slide 19 Detection Results Automatic DetectionGround Truth A 12.4mm lesion detected at the second coarsest resolution

Edward J. Delp Lucent January 12, 2000 Slide 20 Detection Results A 6.6mm lesion detected at the finest resolution Automatic DetectionGround Truth

Edward J. Delp Lucent January 12, 2000 Slide 21 Research Team Charles Babbs - Department of Basic Medical Sciences Zygmunt Pizlo - Department of Psychological Sciences Sheng Lui - School of Electrical and Computer Engineering Valerie Jackson - IU Department of Radiology Funding - NSF, NIH, and Purdue Cancer Center

Edward J. Delp Lucent January 12, 2000 Slide 22 ViBE: A New Paradigm for Video Database Browsing and Search ViBE has four components –scene change detection and identification –hierarchical shot representation –pseudo-semantic shot labeling –active browsing based on relevance feedback ViBE provides an extensible framework that will scale as the video data grows in size and applications increase in complexity

Edward J. Delp Lucent January 12, 2000 Slide 23 Video Analysis: Overview Audio data Image data (DC frames) MPEG-related data (MVs, AC coeffs, etc.) Compressed video sequence Proc. Closed-caption information Proc. Transition locations and types Shot trees Proc. Captions Shot attributes Data Extraction Shot Transition Detection and Identification Intrashot Clustering Shot Labeling

Edward J. Delp Lucent January 12, 2000 Slide 24 Navigation via the Similarity Pyramid Zoom in Zoom out Zoom in Zoom out

Edward J. Delp Lucent January 12, 2000 Slide 25 Browser Interface Relevance SetSimilarity Pyramid Control Panel

Edward J. Delp Lucent January 12, 2000 Slide 26 Video Over IP: Unicast

Edward J. Delp Lucent January 12, 2000 Slide 27 Video Over IP: Multicast

Edward J. Delp Lucent January 12, 2000 Slide 28 Video Over IP Currently multicasting 3 streams Multicast experiments with Europe Multicast HDTV over Internet2 Issues: –what is the backward information? –which video compression technique? –how is network control connected to the server/encoder?

Edward J. Delp Lucent January 12, 2000 Slide 29 Scenario –an owner places digital images on a network server and wants to detect the redistribution of altered versions Goals –verify the owner of a digital image –detect forgeries of an original image –identify illegal copies of the image –prevent unauthorized distribution Why is Digital Watermarking Important?

Edward J. Delp Lucent January 12, 2000 Slide 30 Why is Watermarking Important?

Edward J. Delp Lucent January 12, 2000 Slide 31 Why is Watermarking Important?

Edward J. Delp Lucent January 12, 2000 Slide 32 Why Watermarking is Important?

Edward J. Delp Lucent January 12, 2000 Slide 33 Why is Watermarking Important?

Edward J. Delp Lucent January 12, 2000 Slide 34 VW2D Watermarked Image

Edward J. Delp Lucent January 12, 2000 Slide 35 Image Adaptive Watermarks (DCT)

Edward J. Delp Lucent January 12, 2000 Slide 36 Scalable Image and Video Compression Problem: desire to have a compression technique that allows decompression to be linked to the application –databases, wireless transmission, Internet imaging –will support both high and low data rate modes Other desired properties: –error concealment –will support the protection of intellectual property rights (watermarking)

Edward J. Delp Lucent January 12, 2000 Slide 37 Rate Scalable Image and Video Coding Applications –Internet streaming –Image and video database search - browsing –Video servers –Teleconferencing and Telemedicine –Wireless Networks

Edward J. Delp Lucent January 12, 2000 Slide 38 Scalability Picture Coding Symposium(April 1999) - panel on “The Future of Video Compression,” importance of scalability: –rate scalability (Internet and wireless) –temporal scalability (Internet and wireless) –spatial scalability (databases - MPEG-7) –content scalability (MPEG-4) (Computational Scalability - implementation issues)

Edward J. Delp Lucent January 12, 2000 Slide 39 Scalability “Author and Compress once - decompress on any platform feed by any data pipe”

Edward J. Delp Lucent January 12, 2000 Slide 40 Scalability: Compression Standards Scalability in JPEG –progressive mode –JPEG 2000 Scalability in MPEG-2 –scalability is layered Scalability in MPEG-4 –layered –“content” –fine grain scalability (fgs)

Edward J. Delp Lucent January 12, 2000 Slide 41 Color Embedded Zero-Tree Wavelet (CEZW) Developed new technique known as Color Embedded Zero-Tree Wavelet (CEZW) Modified EZW with trees connecting all color components –can be extended to other color spaces

Edward J. Delp Lucent January 12, 2000 Slide 42 Spatial Orientation Trees EZWSPIHT

Edward J. Delp Lucent January 12, 2000 Slide 43 New Spatial Orientation Tree (CEZW)

Edward J. Delp Lucent January 12, 2000 Slide 44 Color Image Compression OriginalCEZW JPEGSPIHT

Edward J. Delp Lucent January 12, 2000 Slide 45 Coding Artifacts Original CEZW JPEG SPIHT

Edward J. Delp Lucent January 12, 2000 Slide 46 Comparison JPEG 0.25 bits/pixelCEZW 0.25 bits/pixel

Edward J. Delp Lucent January 12, 2000 Slide 47 Color Compression - Experiments Objectives: –Evaluate scalable color image compression techniques –Color Transformations –Spatial Orientation Trees and Coding Schemes –Embedded Coding Embedded Zerotree Wavelet: Shapiro (Dec’93) Set Partitioning in Hierarchical Trees: Said & Pearlman (Jun’96) Color Embedded Zerotree Wavelets: Shen & Delp (Oct ‘97) M. Saenz, P. Salama, K. Shen and E. J. Delp, "An Evaluation of Color Embedded Wavelet Image Compression Techniques," VCIP 1999

Edward J. Delp Lucent January 12, 2000 Slide 48 SAMCoW New scalable video compression technique - Scalable Adaptive Motion COompensated Wavelet compression Features of SAMCoW: –use wavelets on entire frame and for prediction error frames –uses adaptive motion compensation to reduce error propagation –CEZW is used for the wavelet coder on both the intra- coded frames and prediction error frames

Edward J. Delp Lucent January 12, 2000 Slide 49 Generalized Hybrid Codec

Edward J. Delp Lucent January 12, 2000 Slide 50 Adaptive Motion Compensation

Edward J. Delp Lucent January 12, 2000 Slide 51 SAMCoW Enhancements B frames (ICIP98) unrestricted motion vectors (ICIP98) half-pixel motion searches (ICIP98) detailed study of PEF (ICIP99 and VLBW99) –denoising techniques bit allocation and rate control (ICIP99)

Edward J. Delp Lucent January 12, 2000 Slide 52 Error Concealment In data networks, channel errors or congestion can cause cell or packet loss When compressed video is transmitted, cell loss causes macroblocks and motion vectors to be removed from compressed data streams Goal of error concealment: Exploit redundant information in a sequence to recover missing data

Edward J. Delp Lucent January 12, 2000 Slide 53 Error Concealment Original frameDamaged frame

Edward J. Delp Lucent January 12, 2000 Slide 54 Approaches for Error Concealment Two approaches for error concealment: –Active concealment: Use of error control coding techniques and retransmission unequal error protection –Passive concealment: The video stream is post- processed to reconstruct missing data Passive concealment is necessary: –where active concealment cannot be used due to compliance with video transmission standards –when active concealment fails

Edward J. Delp Lucent January 12, 2000 Slide 55 Error Concealment

Edward J. Delp Lucent January 12, 2000 Slide 56 Error Concealment

Edward J. Delp Lucent January 12, 2000 Slide 57 Future Work Video Streaming (wired and wireless) Color Compression experiments (JPEG2000) Video databases ViBE Video watermarking Internet2 and multicasting scalable video Error concealment in embedded codecs

Edward J. Delp Lucent January 12, 2000 Slide 58 How I Spent My Summer