Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 1 ECE-C490 Image Processing Architecture MP-3 Compression Course Review Oleh Tretiak March 13, 2003

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 2  Reference: MPEG Handbook: MPEG-1, MPEG-2, MPEG-4 by John Watkinson, available through Drexel Library, Books 24/7

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 3 Acoustics and Audition  Acoustic fields are mechanical vibration in air  Common measure: sound pressure of sinusoidal temporal variation  Sound decibels are relative to 2x10 -5 Pascals rms, db(SPL)  Weighted response attempts to mimic human hearing

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 4 Subjective Weighting

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 5 Why Compress?  “With the rising density and falling cost of digital recording media, the use of compression in professional audio storage applications is hard to justify. However, where there is a practical or economic restriction on channel bandwidth compression becomes essential. Now that the technology exists to broadcast television pictures using compressed digital coding, it is obvious that the associated audio should be conveyed in the same way.”

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 6 How Big is Big  Telephone type audio  8 Ksamples per second, 8 bit/sample  64 k bit/sec  CD audio  40 k samples, 2 channels, 16 bits per sample  1.28 M bit/sec

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 7 Why 16 bits?  In short term, 8 bits gives good audio quality  Music intensity can vary over a large range  Range from threshold to very loud is 120 dB = 10 6 !

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 8 Compression Technologies  Companding  DPCM  Subband Coding

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 9 Companding Quantization Noise Q

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 10 DPCM (Delta Modulation)

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 11 Transform Coding  Split waveform into segment  Take transform  Code transform coefficients  To avoid artifacts, transforms of overlapping segments are used

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 12 Transform and Subbands  Fourier transform coefficients correspond to frequency bands  Only limited dynamic range is needed for each band

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 13 Sound Band Phenomena  Two tones at adjacent frequencies can combine to make a beat pattern  When the tones are separated in frequency, they may interfere or may be independent.  If one of the tones has a small amplitude, it is masked

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 14 Examples of Tone Pairs db -12 db -18 db -24 db -30 db -36 db Two second of 300 Hz sinewave plus second sinewave at adjacent frequency and at lower amplitude. Total power is constant.

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 15 MP3: MPEG Layer III  32 subband filters working over 1152 sample windows (~ 26 msec)  Subband outputs processed with MDCT (modified DCT), 18 frequencies  A total of 576 frequencies  Four window types, more than one window size

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 16 Window Types The window functions of Layer III coding. At (a) is the normal long window, whereas (b) shows the short window used to handle transients. Switching between window sizes requires transition windows (c) and (d). An example of switching using transition windows is shown in (e).

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 17 MPEG Layer III coder  The Layer III coder. Note the connection between the buffer and the quantizer which allows different frames to contain different amounts of data.

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 18 Examples of MP 3 Compression  Audio (13 seconds)  Music (13 seconds)

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 19 MP3 Compression Samples SpeechMusic wav 8 kbit/sec 16 kbit/sec 32 kbit/sec 64 kbit

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 20 Course Overview  Lossless compression  Huffman coding  Lossy compression  Image Predictive (DPCM) Block transform (DCT)  Video Motion estimation  Standards  JPEG  MPEG/ITU  Performance evaluation  Special Hardware

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 21 JPEG  Subsampling component images  MCU  Zig-zag coding  Quantization  JPEG run-length coding  Color conversion  JPEG bit stream  Not just still images!

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 22 MPEG/ITU  Family of Standards  I, P, B pictures  Different amounts of compression  Types of coding in pictures I, P, B, skipped blocks  Bit stream structure GOP ≠ giga operations  Standards and subsampling  Display and coding resolutions are different  Color conversion  MPEG-2 profiles

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 23 Performance Evaluation  Basic evaluation: instruction counts  Where’s the beef?  JPEG  MPEG  Implementation  Computer  RISC Cores  DSP  Special Hardware

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 24 Special Hardware  Hardware for DCT  RAC  Hardware for Motion Estimation  Linear arrays  Issues  Best algorithm for hardware ≠ best algorithm for software  Hardware removes program overhead parallel ALU and address calculation  Multiple processing units

Image Processing Architecture, © 2001, 2002, 2003 Oleh TretiakPage 25 Big Picture  Imaging/Video is key technology  Computer  Communications  Entertainment television, movies Games  Photography  Instrumentation  Web  Database  System perspective  hardware/software  system