ECE 4331, Fall, 2009 Zhu Han Department of Electrical and Computer Engineering Class 22 Nov. 6 th, 2009.

Slides:



Advertisements
Similar presentations
Multimedia System Video
Advertisements

Introduction to H.264 / AVC Video Coding Standard Multimedia Systems Sharif University of Technology November 2008.
JPEG Compresses real images Standard set by the Joint Photographic Experts Group in 1991.
MPEG Moving Picture Experts Group. What defines good video quality? Size of pictures Bitrate of channel medium (especially in real-time applications)
Chapter 18 Discrete Cosine Transform. Dr. Naim Dahnoun, Bristol University, (c) Texas Instruments 2004 Chapter 18, Slide 2 Learning Objectives  Introduction.
Basics of MPEG Picture sizes: up to 4095 x 4095 Most algorithms are for the CCIR 601 format for video frames Y-Cb-Cr color space NTSC: 525 lines per frame.
Computer Vision – Coding Standards
Source Coding for Video Application
SWE 423: Multimedia Systems
SWE 423: Multimedia Systems
Computer Graphics Recitation 7. 2 Motivation – Image compression What linear combination of 8x8 basis signals produces an 8x8 block in the image?
MPEG: A Video Compression Standard for Multimedia Applications Didier Le Gall Communications of the ACM Volume 34, Number 4 Pages 46-58, 1991.
Department of Computer Engineering University of California at Santa Cruz Video Compression Hai Tao.
MPEG: A Video Compression Standard for Multimedia Applications Didier Le Gall Communications of the ACM Volume 34, Number 4 Pages 46-58, 1991.
CS :: Fall 2003 MPEG-1 Video (Part 1) Ketan Mayer-Patel.
JPEG Still Image Data Compression Standard
T.Sharon-A.Frank 1 Multimedia Image Compression 2 T.Sharon-A.Frank Coding Techniques – Hybrid.
5. 1 JPEG “ JPEG ” is Joint Photographic Experts Group. compresses pictures which don't have sharp changes e.g. landscape pictures. May lose some of the.
1 Image and Video Compression: An Overview Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur,
Why Compress? To reduce the volume of data to be transmitted (text, fax, images) To reduce the bandwidth required for transmission and to reduce storage.
MPEG-2 Digital Video Coding Standard
CSE679: MPEG r MPEG-1 r MPEG-2. MPEG r MPEG: Motion Pictures Experts Group r Standard for encoding videos/movies/motion pictures r Evolving set of standards.
Image and Video Compression
ECE 4371, Fall, 2014 Introduction to Telecommunication Engineering/Telecommunication Laboratory Zhu Han Department of Electrical and Computer Engineering.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 8 – JPEG Compression (Part 3) Klara Nahrstedt Spring 2012.
ECE472/572 - Lecture 12 Image Compression – Lossy Compression Techniques 11/10/11.
MPEG: A Video Compression Standard for Multimedia Applications Didier Le Gall Communications of the ACM Volume 34, Number 4 Pages 46-58, 1991.
1 Image Compression. 2 GIF: Graphics Interchange Format Basic mode Dynamic mode A LZW method.
MPEG MPEG-VideoThis deals with the compression of video signals to about 1.5 Mbits/s; MPEG-AudioThis deals with the compression of digital audio signals.
Video Coding. Introduction Video Coding The objective of video coding is to compress moving images. The MPEG (Moving Picture Experts Group) and H.26X.
M P E G A Presentation by Sebastian Graf and Thomas Winterscheid.
Introduction to JPEG and MPEG Ingemar J. Cox University College London.
MPEG-1 and MPEG-2 Digital Video Coding Standards Author: Thomas Sikora Presenter: Chaojun Liang.
MPEG: (Moving Pictures Expert Group) A Video Compression Standard for Multimedia Applications Seo Yeong Geon Dept. of Computer Science in GNU.
Video Compression Techniques By David Ridgway.
Multimedia Data Video Compression The MPEG-1 Standard Dr Mike Spann Electronic, Electrical and Computer.
Klara Nahrstedt Spring 2011
Concepts of Multimedia Processing and Transmission IT 481, Lecture 5 Dennis McCaughey, Ph.D. 19 February, 2007.
Videos Mei-Chen Yeh. Outline Video representation Basic video compression concepts – Motion estimation and compensation Some slides are modified from.
Image Processing and Computer Vision: 91. Image and Video Coding Compressing data to a smaller volume without losing (too much) information.
Codec structuretMyn1 Codec structure In an MPEG system, the DCT and motion- compensated interframe prediction are combined. The coder subtracts the motion-compensated.
MPEG MPEG : Motion Pictures Experts Group MPEG : ISO Committee Widely Used Video Compression Standard.
June, 1999 An Introduction to MPEG School of Computer Science, University of Central Florida, VLSI and M-5 Research Group Tao.
Image Compression Supervised By: Mr.Nael Alian Student: Anwaar Ahmed Abu-AlQomboz ID: IT College “Multimedia”
Compression video overview 演講者:林崇元. Outline Introduction Fundamentals of video compression Picture type Signal quality measure Video encoder and decoder.
ECE 6332, Fall, 2014 Wireless Communication Zhu Han Department of Electrical and Computer Engineering Class 21 Apr. 7 th, 2014.
Advances in digital image compression techniques Guojun Lu, Computer Communications, Vol. 16, No. 4, Apr, 1993, pp
Image/Video Coding Techniques for IPTV Applications Wen-Jyi Hwang ( 黃文吉 ) Department of Computer Science and Information Engineering, National Taiwan Normal.
MPEG.
Block-based coding Multimedia Systems and Standards S2 IF Telkom University.
STATISTIC & INFORMATION THEORY (CSNB134) MODULE 11 COMPRESSION.
Video Compression and Standards
6/9/20161 Video Compression Techniques Image, Video and Audio Compression standards have been specified and released by two main groups since 1985: International.
Principles of Video Compression Dr. S. M. N. Arosha Senanayake, Senior Member/IEEE Associate Professor in Artificial Intelligence Room No: M2.06
Image Processing Architecture, © Oleh TretiakPage 1Lecture 5 ECEC 453 Image Processing Architecture Lecture 5, 1/22/2004 Rate-Distortion Theory,
MPEG Video Coding I: MPEG-1 1. Overview  MPEG: Moving Pictures Experts Group, established in 1988 for the development of digital video.  It is appropriately.
H. 261 Video Compression Techniques 1. H.261  H.261: An earlier digital video compression standard, its principle of MC-based compression is retained.
Introduction to H.264 / AVC Video Coding Standard Multimedia Systems Sharif University of Technology November 2008.
Present by 楊信弘 Advisor: 鄭芳炫
JPEG Compression What is JPEG? Motivation
CSI-447: Multimedia Systems
JPEG Image Coding Standard
"Digital Media Primer" Yue-Ling Wong, Copyright (c)2013 by Pearson Education, Inc. All rights reserved.
JPEG.
Standards Presentation ECE 8873 – Data Compression and Modeling
MPEG4 Natural Video Coding
Judith Molka-Danielsen, Oct. 02, 2000
JPEG Still Image Data Compression Standard
Image Coding and Compression
Fundamentals of Video Compression
Presentation transcript:

ECE 4331, Fall, 2009 Zhu Han Department of Electrical and Computer Engineering Class 22 Nov. 6 th, 2009

Motivation – Image compression What linear combination of 8x8 basis signals produces an 8x8 block in the image?

Learning Objectives Introduction to the DCT and IDCT. Decomposition of a 2-D DCT to two 1-D DCTs. Implementation of a 2-D DCT using a 1-D DCT.

EE 554/454 Fall 2007 Introduction To perform the JPEG coding, an image (in colour or grey scales) is first subdivided into blocks of 8x8 pixels. The Discrete Cosine Transform (DCT) is the performed on each block. This generates 64 coefficients which are then quantised to reduce their magnitude.

Introduction The coefficients are then reordered into a one-dimensional array in a zigzag manner before further entropy encoding. The compression is achieved in two stages; the first is during quantisation and the second during the entropy coding process. JPEG decoding is the reverse process of coding.

Implementation of the DCT DCT-based codecs use a two-dimensional version of the transform. The 2-D DCT and its inverse (IDCT) of an N x N block are shown below: Note: The DCT is similar to the DFT since it decomposes a signal into a series of harmonic cosine functions. –2-D DCT: –2-D IDCT:

2-D DCT using a 1-D DCT Pair One of the properties of the 2-D DCT is that it is separable meaning that it can be separated into a pair of 1-D DCTs. To obtain the 2-D DCT of a block a 1-D DCT is first performed on the rows of the block then a 1-D DCT is performed on the columns of the resulting block. The same applies to the IDCT. This process is illustrated on the following slide.

2-D DCT using a 1-D DCT Pair

2-D DCT using a 1-D DCT Pair –1-D DCT: –1-D IDCT: ] 2 )12( cos[)()( 2 )( 1 0      N k N ki kXkC N ix  k = 0, 1, 2, …, N-1. and i = 0, 1, 2, …, N-1.

Implementation Issues Precalculate the DCT coefficients and scale them

Block-based DCT and IDCT in C The most straightforward way of implementing a DCT and an IDCT is to use the 1-D DCT and IDCT. The following C code shows how to translate the 1-D equations for the DCT and IDCT into C code. The program also takes into account the numerical issues associated with fixed-point processors.

Using DCT in JPEG  DCT on 8x8 blocks

Using DCT in JPEG  Block size : small block - faster - correlation exists between neighboring pixels large block - better compression in “flat” regions  Power of 2 – for fast implementation

Using DCT in JPEG  DCT – basis

Basis vectors

Comparison of DF and DCT

 For almost flat surface most G ij =0  For surface that oscillates much many G ij non zero  G 00 = DC coefficient  Numbers at top left of G ij contribution of low freq. sinusoidal to the surface, bottom right – high freq. Using DCT in JPEG

 Numbers at top left of G ij contribution of low freq. sinusoidal to the surface, bottom right – high freq.  Scan each block in zig-zag order Using DCT in JPEG

DCT enables image compression by concentrating most image information in the low frequencies Loose unimportant image info buy cut G ij at right bottom Decoder computes the inverse IDCT Image compression using DCT

Quantization and Coding Zonal Coding: Coefficients outside the zone mask are zeroed. The coefficients outside the zone may contain significant energy Local variations are not reconstructed properly

Poor Lena ( 30:1 compression)

JPEG : Lena survived after 12:1 compression

Motion JPEG JPEG system for compressing static images could be applied to a sequence of images, compressing each individually, this is called motion JPEG Motion JPEG takes no advantage of any correlation between successive images In a typical scene there will be a great deal of similarity between nearby images of the same sequence.

In a typical scene there will be a great deal of similarity between nearby images of the same sequence.

Motion Compensation Approach Basic idea of Motion Compensation: –Many “moving” images or image sequences consist of a static background with one or more moving foreground objects. We can get coding advantage from this. –we code the first frame by baseline JPEG and use this frame as reference image. – Treat the second image block by block and compare each block with the same block in the reference image. –For blocks that have identical block in reference image, we only send a special code instead of whole code. –For other blocks, we just encode them as usual.

Motion Compensation Approach(cont.) Motion Vectors –static background is a very special case, we should consider the displacement of the block. –Motion vector is used to inform decoder exactly where in the previous image to get the data. –Motion vector would be zero for a static background.

Motion Compensation Approach(cont.) Block Matching--how to find the matching block? –Matching criteria: u In practice we couldn’t expect to find the exactly identical matching block, instead we look for close match. u Most motion estimation schemes look for minimum mean square error(MMSE) between block. –Matching block size: u How large the matching block will affect coding efficiency u block size MPEG used: 16×16

Motion Compensation Approach(cont.) Search range: –It’s reasonable to consider an displacement of 360 pixles/s or about 60pixels/image in standard- definition television. –In real-world scenes there is usually more or faster motion horizontally than vertically, generally the width of search area should be twice the height. –Suggested search range: ±60 pixles × 30 pixles

Motion Compensation Approach(cont.) Residuals –The differences between the block being coded and it’s best match are known as residuals. –The residuals maybe encoded and transmitted along with the motion vector, so the decoder will be able to reconstruct the block. –We should compare the bits of transmitting the motion vector plus the residuals with the bits of transmitting the block itself and use the most efficient mechanism.

MPEG-1 Introduction –MPEG: Moving Pictures Experts Group. –MPEG-video is addressing the compression of video signals at about 1.5Mbits/s –MPEG-1 is asymmetric system, the complexity of the encoder is much higher than that of the decoder. Table 1: MPEG-1 Constraints

MPEG Hierarchy The six layers of MPEG video bit stream Sequence Layer: video clip, complete program item. Group of Pictures Layer(GOP): include three different coding ways. Frame Layer Slice Layer: in case the data is lost or corrupted. Macroblock Layer: 16×16 luminance block. Block Layer(DCT unit)

Frame Types in MPEG Intra frames (I-frames) –A I-frame is encoded using only information from within that frame(intra coded) -- no temporal compression(inter coded). Non-intra frames (P-frames and B-frames) –motion compensated information will be used for coding. –P frame (predicted frame) use preceding frame as reference image –B frame (bidirectional frame) use both preceding frame and following frame as reference images

Motion estimation for different frames XZ Y Available from earlier frame (X) Available from later frame (Z)

Reconstructing a reference frame that will be the same as at the decoder Frame being coded DCTQuantize Variable length code Reference frame Inverse DCT De- quantize To transmit buffer

A typical group of pictures in display order A typical group of pictures in coding order I P B B B P B B B P B B B I B B B P B B B P B B B P

Coding of Macroblock Y CBCB CRCR    Spatial sampling relationship for MPEG-1  -- Luminance sample  -- Color difference sample

Coding of Macroblock (cont.) Intra coding of macroblocks – just as what JPEG does – MPEG has two default quantization tables, one for intra coding, another one for non-intra coding of residuals MPEG quantization table(for intra coding) JPEG quantization table(luminance)

Coding of Macroblock (cont.) Non-intra coding of macroblocks The first step is to intra code the macroblock--just in case if we fail to find a reasonable match in motion estimation. Then we use motion estimation to find the nearest match and get the motion vector. Only luminance samples are used in motion estimation. Then each DCT block in macroblock will be treated separately. The residuals will be encode by DCT and quantization (use flat table) as in intra coding. DC along with AC This process is applied to all six blocks in the macroblock Motion vectors are coded predictively

Coding of Macroblock (cont.) P-frames –If the block can be skipped, we just send a “skip” code –otherwise, we compare the number of total bits of inter and intra coding, choose the more efficient one. Mark this block accordingly. B-frames –comparison among three methods of encoding

A Simplified MPEG encoder Frame recorder DCT Quantize Variable- length coder Transmit buffer Prediction encoder De- quantize Inverse DCT Motion predictor Reference frame Rate controller IN OUT Scale factor Buffer fullness Prediction Motion vectors DC

MPEG Standards MPEG stands for the Moving Picture Experts Group. MPEG is an ISO/IEC working group, established in 1988 to develop standards for digital audio and video formats. There are five MPEG standards being used or in development. Each compression standard was designed with a specific application and bit rate in mind, although MPEG compression scales well with increased bit rates. They include:Moving Picture Experts Group –MPEG1 –MPEG2 –MPEG4 –MPEG7 –MPEG21 –MP3

MPEG Standards MPEG-1 Designed for up to 1.5 Mbit/sec Standard for the compression of moving pictures and audio. This was based on CD-ROM video applications, and is a popular standard for video on the Internet, transmitted as.mpg files. In addition, level 3 of MPEG-1 is the most popular standard for digital compression of audio--known as MP3. MPEG-1 is the standard of compression for VideoCD, the most popular video distribution format thoughout much of Asia. MPEG-2 Designed for between 1.5 and 15 Mbit/sec Standard on which Digital Television set top boxes and DVD compression is based. It is based on MPEG-1, but designed for the compression and transmission of digital broadcast television. The most significant enhancement from MPEG-1 is its ability to efficiently compress interlaced video. MPEG-2 scales well to HDTV resolution and bit rates, obviating the need for an MPEG-3. MPEG-4 Standard for multimedia and Web compression. MPEG-4 is based on object-based compression, similar in nature to the Virtual Reality Modeling Language. Individual objects within a scene are tracked separately and compressed together to create an MPEG4 file. This results in very efficient compression that is very scalable, from low bit rates to very high. It also allows developers to control objects independently in a scene, and therefore introduce interactivity. MPEG-7 - this standard, currently under development, is also called the Multimedia Content Description Interface. When released, the group hopes the standard will provide a framework for multimedia content that will include information on content manipulation, filtering and personalization, as well as the integrity and security of the content. Contrary to the previous MPEG standards, which described actual content, MPEG-7 will represent information about the content. MPEG-7 MPEG-21 - work on this standard, also called the Multimedia Framework, has just begun. MPEG-21 will attempt to describe the elements needed to build an infrastructure for the delivery and consumption of multimedia content, and how they will relate to each other. MPEG-21

JPEG JPEG stands for Joint Photographic Experts Group. It is also an ISO/IEC working group, but works to build standards for continuous tone image coding. JPEG is a lossy compression technique used for full-color or gray-scale images, by exploiting the fact that the human eye will not notice small color changes.Joint Photographic Experts Group JPEG 2000 is an initiative that will provide an image coding system using compression techniques based on the use of wavelet technology.

DV DV is a high-resolution digital video format used with video cameras and camcorders. The standard uses DCT to compress the pixel data and is a form of lossy compression. The resulting video stream is transferred from the recording device via FireWire (IEEE 1394), a high-speed serial bus capable of transferring data up to 50 MB/sec. –H.261 is an ITU standard designed for two-way communication over ISDN lines (video conferencing) and supports data rates which are multiples of 64Kbit/s. The algorithm is based on DCT and can be implemented in hardware or software and uses intraframe and interframe compression. H.261 supports CIF and QCIF resolutions. –H.263 is based on H.261 with enhancements that improve video quality over modems. It supports CIF, QCIF, SQCIF, 4CIF and 16CIF resolutions. –H.264

DivX Compression DivX is a software application that uses the MPEG-4 standard to compress digital video, so it can be downloaded over a DSL/cable modem connection in a relatively short time with no reduced visual quality. The latest version of the codec, DivX 4.0, is being developed jointly by DivXNetworks and the open source community. DivX works on Windows 98, ME, 2000, CE, Mac and Linux.DivXNetworks