1. Edward J. Delp Intel December 3, 1999 Slide 1 Video and Image Processing At Purdue Edward J. Delp Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA email: ace@ecn.purdue.edu http://www.ece.purdue.edu/~ace

2. Edward J. Delp Intel December 3, 1999 Slide 2 Acknowledgements Students - –Eduardo Asbun –Dan Hintz –Paul Salama –Ke Shen –Martha Saenz –Eugene Lin –Ray Wolfgang –Greg Cook –Sheng Liu

3. Edward J. Delp Intel December 3, 1999 Slide 3 Intel T4E Project Purdue awarded $6.2 million in August 1997 for equipment –this is one of many strong relationships between Intel and Purdue Has had a very significant impact on how we do research! THANKS! http://www.cs.purdue.edu/homes/jtk/intel/

4. Edward J. Delp Intel December 3, 1999 Slide 4 Image and Video Processing at Purdue Purdue has a rich history 60 year history in image and video processing.

5. Edward J. Delp Intel December 3, 1999 Slide 5 VIPER Research Projects Scalable Video and Color Image Compression –still image compression (CEZW) –high and low bit rate video compression (SAMCoW) –wireless video Error Concealment Content Addressable Video Databases (ViBE) –Scene Change Detection and Identification –Pseudo-Semantic Scene Labeling Multimedia Security: Digital Watermarking

6. Edward J. Delp Intel December 3, 1999 Slide 6 VIPER Research Projects Multicast Video Analysis of Mammograms Embedded Image and Video Processing

7. Edward J. Delp Intel December 3, 1999 Slide 7 Other Purdue Projects Electronic Imaging - Jan Allebach and Charles Bouman –half-tone printing –compound document compression –image databases Remote Sensing - David Langrebe Medical Imaging - Charles Bouman, Peter Doerschuk, Thomas Talavage, Edward Delp –computed imagng –functional MRI –x-ray crystallography –breast imaging

8. Edward J. Delp Intel December 3, 1999 Slide 8 Analysis of Mammograms Density 1Density 2Density 3Density 4

9. Edward J. Delp Intel December 3, 1999 Slide 9 Detection Results Automatic DetectionGround Truth A 12.4mm lesion detected at the second coarsest resolution

10. Edward J. Delp Intel December 3, 1999 Slide 10 Detection Results A 6.6mm lesion detected at the finest resolution Automatic DetectionGround Truth

11. Edward J. Delp Intel December 3, 1999 Slide 11 ViBE: A New Paradigm for Video Database Browsing and Search ViBE has four components –scene change detection and identification –hierarchical shot representation –pseudo-semantic shot labeling –active browsing based on relevance feedback ViBE provides an extensible framework that will scale as the video data grows in size and applications increase in complexity

12. Edward J. Delp Intel December 3, 1999 Slide 12 Video Analysis: Overview Audio data Image data (DC frames) MPEG-related data (MVs, AC coeffs, etc.) Compressed video sequence Proc. Closed-caption information Proc. Transition locations and types Shot trees Proc. Captions Shot attributes Data Extraction Shot Transition Detection and Identification Intrashot Clustering Shot Labeling

13. Edward J. Delp Intel December 3, 1999 Slide 13 Navigation via the Similarity Pyramid Zoom in Zoom out Zoom in Zoom out

14. Edward J. Delp Intel December 3, 1999 Slide 14 Browser Interface Relevance SetSimilarity Pyramid Control Panel

15. Edward J. Delp Intel December 3, 1999 Slide 15 Video Over IP: Unicast

16. Edward J. Delp Intel December 3, 1999 Slide 16 Video Over IP: Multicast

17. Edward J. Delp Intel December 3, 1999 Slide 17 Video Over IP Currently multicasting 3 streams Multicast experiments with Europe Multicast HDTV over Internet2 Issues: –what is the backward information? –which video compression technique? –how is network control connected to the server/encoder?

18. Edward J. Delp Intel December 3, 1999 Slide 18 Scenario –an owner places digital images on a network server and wants to detect the redistribution of altered versions Goals –verify the owner of a digital image –detect forgeries of an original image –identify illegal copies of the image –prevent unauthorized distribution Why is Digital Watermarking Important?

19. Edward J. Delp Intel December 3, 1999 Slide 19 Why is Watermarking Important?

20. Edward J. Delp Intel December 3, 1999 Slide 20 Why is Watermarking Important?

21. Edward J. Delp Intel December 3, 1999 Slide 21 Why Watermarking is Important?

22. Edward J. Delp Intel December 3, 1999 Slide 22 Why is Watermarking Important?

23. Edward J. Delp Intel December 3, 1999 Slide 23 VW2D Watermarked Image

24. Edward J. Delp Intel December 3, 1999 Slide 24 Image Adaptive Watermarks (DCT)

25. Edward J. Delp Intel December 3, 1999 Slide 25 Scalable Image and Video Compression Problem: desire to have a compression technique that allows decompression to be linked to the application –databases, wireless transmission, Internet imaging –will support both high and low data rate modes Other desired properties: –error concealment –will support the protection of intellectual property rights (watermarking)

26. Edward J. Delp Intel December 3, 1999 Slide 26 Rate Scalable Image and Video Coding Applications –Internet streaming –Image and video database search - browsing –Video servers –Teleconferencing and Telemedicine –Wireless Networks

27. Edward J. Delp Intel December 3, 1999 Slide 27 Scalability Picture Coding Symposium(April 1999) - panel on “The Future of Video Compression,” importance of scalability: –rate scalability (Internet and wireless) –temporal scalability (Internet and wireless) –spatial scalability (databases - MPEG-7) –content scalability (MPEG-4) (Computational Scalability - implementation issues)

28. Edward J. Delp Intel December 3, 1999 Slide 28 Scalability “Author and Compress once - decompress on any platform feed by any data pipe”

29. Edward J. Delp Intel December 3, 1999 Slide 29 Scalability: Compression Standards Scalability in JPEG –progressive mode –JPEG 2000 Scalability in MPEG-2 –scalability is layered Scalability in MPEG-4 –layered –“content” –fine grain scalability (fgs)

30. Edward J. Delp Intel December 3, 1999 Slide 30 Color Embedded Zero-Tree Wavelet (CEZW) Developed new technique known as Color Embedded Zero-Tree Wavelet (CEZW) Modified EZW with trees connecting all color components –can be extended to other color spaces

31. Edward J. Delp Intel December 3, 1999 Slide 31 Spatial Orientation Trees EZWSPIHT

32. Edward J. Delp Intel December 3, 1999 Slide 32 New Spatial Orientation Tree (CEZW)

33. Edward J. Delp Intel December 3, 1999 Slide 33 Color Image Compression OriginalCEZW JPEGSPIHT

34. Edward J. Delp Intel December 3, 1999 Slide 34 Coding Artifacts Original CEZW JPEG SPIHT

35. Edward J. Delp Intel December 3, 1999 Slide 35 Comparison JPEG 0.25 bits/pixelCEZW 0.25 bits/pixel

36. Edward J. Delp Intel December 3, 1999 Slide 36 Color Compression - Experiments Objectives: –Evaluate scalable color image compression techniques –Color Transformations –Spatial Orientation Trees and Coding Schemes –Embedded Coding Embedded Zerotree Wavelet: Shapiro (Dec’93) Set Partitioning in Hierarchical Trees: Said & Pearlman (Jun’96) Color Embedded Zerotree Wavelets: Shen & Delp (Oct ‘97) M. Saenz, P. Salama, K. Shen and E. J. Delp, "An Evaluation of Color Embedded Wavelet Image Compression Techniques," VCIP 1999

37. Edward J. Delp Intel December 3, 1999 Slide 37 Metrics

38. Edward J. Delp Intel December 3, 1999 Slide 38 SAMCoW New scalable video compression technique - Scalable Adaptive Motion COompensated Wavelet compression Features of SAMCoW: –use wavelets on entire frame and for prediction error frames –uses adaptive motion compensation to reduce error propagation –CEZW is used for the wavelet coder on both the intra- coded frames and prediction error frames

39. Edward J. Delp Intel December 3, 1999 Slide 39 Generalized Hybrid Codec

40. Edward J. Delp Intel December 3, 1999 Slide 40 Adaptive Motion Compensation

41. Edward J. Delp Intel December 3, 1999 Slide 41 SAMCoW Enhancements B frames (ICIP98) unrestricted motion vectors (ICIP98) half-pixel motion searches (ICIP98) detailed study of PEF (ICIP99 and VLBW99) –denoising techniques bit allocation and rate control (ICIP99)

42. Edward J. Delp Intel December 3, 1999 Slide 42 Error Concealment In data networks, channel errors or congestion can cause cell or packet loss When compressed video is transmitted, cell loss causes macroblocks and motion vectors to be removed from compressed data streams Goal of error concealment: Exploit redundant information in a sequence to recover missing data

43. Edward J. Delp Intel December 3, 1999 Slide 43 Error Concealment Original frameDamaged frame

44. Edward J. Delp Intel December 3, 1999 Slide 44 Approaches for Error Concealment Two approaches for error concealment: –Active concealment: Use of error control coding techniques and retransmission unequal error protection –Passive concealment: The video stream is post- processed to reconstruct missing data Passive concealment is necessary: –where active concealment cannot be used due to compliance with video transmission standards –when active concealment fails

45. Edward J. Delp Intel December 3, 1999 Slide 45 Error Concealment

46. Edward J. Delp Intel December 3, 1999 Slide 46 Error Concealment

47. Edward J. Delp Intel December 3, 1999 Slide 47 Future Work Video Streaming (wired and wireless) Color Compression experiments (JPEG2000) Video databases ViBE Video watermarking Internet2 and multicasting scalable video Error concealment in embedded codecs

48. Edward J. Delp Intel December 3, 1999 Slide 48 How I Spent My Summer