MS I Image and Video Compression Edward J. Delp Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering Purdue University

MS I 2 Overview n Contributors Wojciech SzpankowskiWojciech Szpankowski Ananth GramaAnanth Grama Edward DelpEdward Delp n What are the demands on compression New approaches: scalable techniques and pattern matching approachesNew approaches: scalable techniques and pattern matching approaches Error robustness: concealmentError robustness: concealment SecuritySecurity

MS I 3 Purdue University n Purdue has a rich 65 year history in video and imaging n Why do compression?

MS I 4 The “Digital Image” Problem n A 1024x1024 image has 1,048,576 pixels at 24 bits/pixel = 25,165,824 bits24 bits/pixel = 25,165,824 bits n A video (NTSC/CCIR 601) 760x480 = 345,600 pixels760x480 = 345,600 pixels 30 frames/sec = 10,368,000 pixels/sec30 frames/sec = 10,368,000 pixels/sec 16 bits/pixel(4:2:2) = 165,888,000 bits/sec16 bits/pixel(4:2:2) = 165,888,000 bits/sec

MS I 5 Digital Video Rates n CIF (4:1:1) with 12 bits/pixel 31,104,000 bits/sec 31,104,000 bits/sec n CCIR 601 (4:2:2) with 16 bits/pixel 165,888,000 bits/sec 165,888,000 bits/sec n HDTV (GA 1920x1080, 4:2:2, 60 frames/sec, Proscan) with 20 bits/pixel 2,488,320,00 bits/sec 2,488,320,00 bits/sec

MS I 6 Scalable Scalable - “Author and compress ONCE  decompress on ANY platform feed by ANY data pipe”

MS I 7 Scalability n Date rate scalability n SNR or quality scalability n Spatial scalability n Temporal scalability n Computational scalability n “Content” scalability

MS I 8 Scalable Compression n Applications Internet delivery (aid in QoS)Internet delivery (aid in QoS) Image and video database search - browsingImage and video database search - browsing Video serversVideo servers Teleconferencing and telemedicineTeleconferencing and telemedicine Wireless networksWireless networks Kodak’s Photo-CDKodak’s Photo-CD Distributed multimedia documentsDistributed multimedia documents

MS I 9 n Scalability in JPEG Progressive modeProgressive mode JPEG 2000JPEG 2000 n Scalability in MPEG-2 Scalability is layeredScalability is layered n Scalability in MPEG-4 LayeredLayered “Content”“Content” Scalability: Standards

MS I 10 Embedded Coding n Continuously scalable n All compressed data embedded in a single bit stream n Embed the important information at the beginning of the bit stream n Can truncate at any data rate or decoded quality

MS I 11 Scalable Compression n Two new approaches Color Embedded Zero Tree Compression (CEZW)Color Embedded Zero Tree Compression (CEZW) Scalable Adaptive Motion Compensation Wavelet Compression (SAMCoW)Scalable Adaptive Motion Compensation Wavelet Compression (SAMCoW)

MS I 12 Original CEZW JPEG SPIHT Scalable Color Compression

MS I 13 Coding Artifacts Original CEZW JPEG SPIHT

MS I 14 Comparison JPEG 0.25 bits/pixelCEZW 0.25 bits/pixel

MS I 15 2D-Pattern Matching Compression n Where does this pattern match in image or video frame? Central Theme is lossy extension to Lempel-Ziv algorithmCentral Theme is lossy extension to Lempel-Ziv algorithm Strong theoretical underpinningsStrong theoretical underpinnings Use for both images and videoUse for both images and video Use for synthetic images and text - fits into MPEG-4Use for synthetic images and text - fits into MPEG-4

MS I 16 Pattern Matching Compression Pattern Matching JPEG

MS I 17 Error Concealment (1)

MS I 18 Error Concealment (2)

MS I 19 Security:Watermarking

MS I 20 ViBE n ViBE has four components Scene change detection and identificationScene change detection and identification Hierarchical shot representationHierarchical shot representation Pseudo-semantic shot labelingPseudo-semantic shot labeling Active browsing based on relevance feedbackActive browsing based on relevance feedback n ViBE provides an extensible framework

MS I 21 Zoom in Zoom out Zoom in Zoom out

MS I 22 Browser Interface Relevance Set Similarity Pyramid Control Panel

MS I 23 Proposed Equipment n Encoders/Decoders Used for populating databases with video and images using current standardsUsed for populating databases with video and images using current standards n Networking Systems Used to test new ideas in scalable compression and pattern matching techniquesUsed to test new ideas in scalable compression and pattern matching techniques