ELE 488 F06 ELE 488 Fall 2006 Image Processing and Transmission ( ) Lossy wavelet encoding Subband decomposition & coding Wavelet transform Embedded zero tree Successive approximation quantization Digital Video 11/20

ELE 488 F06 Subband Decomposition + Coding Encoding / Decoding G 0 (z) = H 0 (z) = H 1 (-z) = - G 1 (-z) aliasing free H 0 – lowpass, H 1 – highpass Specified by a single filter H 0 (z)

ELE 488 F06 Filterbank and Discrete Wavelet Transform (Mitra – section 14.5, 14.6) More subbands Specified by a single filter H 1 (z)

ELE 488 F06 Total # of u samples same as # of x samples

ELE 488 F06 From x[n] to {u k [n]} Discrete Wavelet Transform (forward DWT) Mother wavelet Scaling function (level k) Discrete Wavelet Transform (forward DWT)

ELE 488 F06 Inverse Discrete Wavelet Transform (IDWT)

ELE 488 F06 Example – Haar Wavelet H 0 (z) = H 1 (-z) Other wavelets

ELE 488 F06

Examples of 1-D Wavelet Transform Note: low frequency components similar From Matlab Wavelet Toolbox Documentation UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 UMCP ENEE631 Slides (created by M.Wu © 2004) 2-D Wavelet Transform via Separable Filters From Usevitch (IEEE Sig.Proc. Mag. 9/01) Note numbering of freq bands

ELE 488 F06 Embedded Zero-Tree Wavelet Coding (EZW) Exploits multi-resolution and self-similar nature of wavelet decomposition –Energy is compacted into a small number of coeff. –Significant coeff w.r.t. a threshold tend to cluster at the same position in each frequency subband Two sets of info. to code: –Where are the significant coefficients? (significance map) –What values are the significant coefficients? Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 Key Concepts in EZW Parent-children relation among coeff. –A coeff at level k spatially correlates with 4 child coeff at level (k-1) of same orientation –A lowest band coeff correlates with 3 coeff. Coding significance map via zero-tree –Encode only high energy coefficients Need to send location info.  large overhead –Encode “insignificance map” with zero- trees Successive approximation quantization –Send most-significant-bits first and gradually refine coeff. value –“Embedded” nature of coded bit-stream Improve image quality by adding extra refining bits UMCP ENEE631 Slides (created by M.Wu © 2001) Usevitch (IEEE Sig.Proc. Mag. 9/01)

ELE 488 F06 EZW Algorithm and Example Initial threshold ~ 2 ^ floor(log 2 x max ) –Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands) –Assign symbol to each coeff. and entropy encode symbols ps – positive significance ns – negative significance iz – isolated zero ztr – zero-tree root –Significant coeff. move to subordinate list put zero in dominant list Subordinate Pass –Output one bit for subordinate list According to position in up/low half of quantization interval Repeat with half threshold –Until bit budget achieved From Usevitch (IEEE Sig.Proc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 1 st Pass UMCP ENEE631 Slides (created by M.Wu © 2001) x max = 53, 32< 53 < 64, T o = 32 Only 2 coef in [32,64) (32+64)/2 = 48 48<53<64  1, 32<34<48  0

ELE 488 F06 After 1 st Pass UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 2 nd Pass From Usevitch (IEEE Sig.Poc. Mag. 9/01) UMCP ENEE631 Slides (created by M.Wu © 2001) T o =32, T 1 =16, 16< 22 < 32, (16+32)/2 = 24

ELE 488 F06 EZW Algorithm and Example Initial threshold ~ 2 ^ floor(log 2 x max ) –Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands) –Assign symbol to each coeff. and entropy encode symbols ps – positive significance ns – negative significance iz – isolated zero ztr – zero-tree root –Significant coeff. move to subordinate list put zero in dominant list Subordinate Pass –Output one bit for subordinate list According to position in up/low half of quantization interval Repeat with half threshold –Until bit budget reached UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 Beyond EZW Cons of EZW –Poor error resilience –Difficult for selective spatial decoding SPIHT (Set Partitioning in Hierarchal Trees) –Further improvement over EZW to remove redundancy EBCOT (Embedded Block Coding with Optimal Truncation) –Used in JPEG 2000 –Address the shortcomings of EZW (random access, error resilience, …) –Embedded wavelet coding in each block + bit-allocations among blocks UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

ELE 488 F06 JPEG 2000: A Wavelet-Based New Standard Targets and features –Excellent low bit rate performance without sacrifice performance at higher bit rate –Progressive decoding to allow from lossy to lossless –Region-of-interest (ROI) coding –Error resilience For details –David Taubman: “High Performance Scalable Image Compression with EBCOT”, IEEE Trans. On Image Proc, July –JPEG2000 Tutorial by Skrodras et al IEEE Sig. Proc Magazine Sept 01 –Links and tutorials: UMCP ENEE631 Slides (created by M.Wu © 2001/2004)

ELE 488 F06

Examples JPEG2K vs. JPEG From Christopoulos (IEEE Trans. on CE 11/00) UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06

DCT vs. Wavelet 3dB improvement? –Wavelet compression claimed to have 3dB improvement over DCT-based compression –Comparison done on JPEG Baseline Improvement not all due to transforms –Improvement comes mainly from better rate allocation, advanced entropy coding, & smarter redundancy reduction via zero-tree –DCT coder can be improved to decrease the gap "A comparative study of DCT- and wavelet-based image coding" Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang, IEEE Trans. on Circuits and Systems for Video Tech., Aug 1999, pp UMCP ENEE631 Slides (created by M.Wu © 2001)

ELE 488 F06 Motion Picture Motion pictures started with an argument: –“Do They Or Don’t They?” –Muybridge (1877) - 12 cameras to take 12 pictures of running horse Limitation of human vision system –persistence and fusion

ELE 488 F06 Motion Picture Perception of motion - persistence and fusion 1877: Muybridge - 12 cameras to take 12 pictures of running horse 1882: Marey - one camera to take 12 pictures per second 1888: Dickson - motion picture camera –Kinetograph (patent 1893, 50ft perforated film, 40 fr/sec, battery, 1000lb) –Kenetoscope, single viewer projector (47 ft film, 25c for 5 shows) 1895: Lumiere - projector/camera (hand-crank, 16 fr/sec, 20lb) 1919: De Forest - optical sound on film 1926 Warner B: “Don Juan” (John Barrymore, NY Philharmonic) : Film industry 1940’s: Television 1980’s: Digital Video Recording event Re-live past moments Editing Creation of fictitious events / objects Manipulate perceived REALITY

ELE 488 F06 Motion Picture  Television  Digital Video Broadcast Television (analog) –movie at home - why invent new technology? –mass market –influence of movie on development Key Steps –convert pictures to electric signal –send electric signal –convert electric signal to picture Comparison with motion picture High Definition Television - analog  digital, compression Video telephone - analog predecessor Video conference - travel cost, people cost Cable (narrowcast), satellite, interactive,...

ELE 488 F06 Key Steps in Broadcast Television Convert picture to electric signal – video camera –initially only at TV studios, cost not as important –recording media (tape), editing, … –special equipment to convert movie Send electric signal –follow radio broadcast, needs spectrum allocation from FCC –VHF (Ch 2 – 13), UHF (Ch 14 & up) Convert electrical signal to picture –cathode ray tube offers economic solution –flat panel: LCD, LED, plasma panel –future

ELE 488 F06 Forming Picture on TV Tube (black-white) How many lines?

ELE 488 F06 How Many TV Lines? dot Cannot resolve if distance > 2000 x separation N = 500

ELE 488 F06 NTSC ( National Television Systems Committee ) 525 lines –2 dots less than 1/2000 of distance from eye are not separated (merge into one) –Assume view at distance 4 times the screen height. Dots on screen less than (1/2000) x 4 x H = H/500 apart are not separated by eye. No need to have more than 500 lines –NTSC set 525 lines (475 active) Movies in 1940 has 4:3 aspect ratio (width to height) 25 or more pictures per second to see continuous motion 50 or more pictures per second to avoid flicker –movies use 24 frames/sec, each shown twice 30 frames/sec with 2:1 interlace (60 even-odd fields/sec)

ELE 488 F06 Television Tube

ELE 488 F06 2:1 Interlaced Scanning even field odd field

ELE 488 F06 Bandwidth of Broadcast Television Without interlace (progressive scan), 60 frames/sec –500 lines alternating black and white gives 250 full cycles –each horizontal line has 250 x 4/3 ~ 350 full cycles –60 (frames/sec) x 500 (line) x 350 = 10,000,000 cycles/sec = 10 MHz –video ONLY With 2:1 interlace, 5 MHz for video FCC assigns 6 MHz per broadcast channel –real usable bandwidth is less, MUCH less –actual resolvable lines per vertical height ~250 Color insertion - must compatible with B/W receiver –Change R-G-B to Y-Cb-Cr –Y is luminance (brightness), Cb and Cr are chrominances –B/W sets converts Y to picture, color sets converts Y-Cb-Cr to R-G-B

ELE 488 F06 Digital Video What drives digital video? –Information technology: electronics, communication infrastructure, storage, functionality, … –HDTV R-G-B component video –640 x 480 (pixel) x 3 (color) x 8 (bits/color) x 30 = 221 Mb/sec Y-Cb-Cr with subsampled Cb and Cr –640 x 480 (pixel) x 1.5 (color) x 8 (bits/color) x 30 = 110 Mb/sec Compression - MPEG (motion picture expert group) –MPEG-1: CD-ROM, 1.5Mb/sec, 1.2Mb/sec for video, 352x240 (CIF), progressive scan, motion compensation –MPEG-2: extension of MPEG-1, interlace, HD –MPEG-4: object/region based –H.2xx

ELE 488 F06 Some Video Formats

ELE 488 F06 Video Coding Video consists of frames, each frame is a still picture –Motion JPEG Each frame is close to the previous frame –Code the difference - Differential Coding Most part of frame is unchanged, except for moving objects –Motion Compensated Coding –MPEG (motion picture expert group)

ELE 488 F06 Video Coding Differential encoding – prediction

ELE 488 F06 Video Coding Each frame is close to the previous frame –Code the frame difference - Differential Coding

ELE 488 F06 Video Coding Most of picture remains unchanged, some objects have moved. Code Displaced Frame Difference - Motion Compensated Coding MPEG (motion picture expert group) previous framecurrent frame

ELE 488 F06

Motion Compensated Encoding current frame previous frame Y

ELE 488 F06 Need for Bi-directional Encoding

ELE 488 F06

Motion Compensated Coding delay

ELE 488 F06 Coding of I-frame

ELE 488 F06 Coding of P frame

ELE 488 F06 More Detail