Roadmap to Lossy Image Compression

Name: Roadmap to Lossy Image Compression
Uploaded: 2017-07-06T05:37:02+00:00
Duration: PTM33S13
Channel: Eleanore Harris
Description: Roadmap to Lossy Image Compression

Roadmap to Lossy Image Compression
Lifting scheme: unifying prediction and transform First-generation schemes FBI WSQ standard Second-generation schemes Embedded Zerotree Wavelet (EZW) A unified where-and-what perspective A classification-based interpretation Scalable and ROI coding in JPEG2000 EE591b Advanced Image Processing Copyright Xin Li 2003

EE591b Advanced Image Processing Copyright Xin Li 2003
Lifting Scheme scale parameter (Wim Sweldens’1995) EE591b Advanced Image Processing Copyright Xin Li 2003

Step 1: Split … … n -3 -2 -1 1 2 3 4 sj(n) … … oddj(n) evenj(n) EE591b Advanced Image Processing Copyright Xin Li 2003

Step 2: Prediction n-1 n n+1 oddj evenj n-1 n n+1 High-band (difference of sj) EE591b Advanced Image Processing Copyright Xin Li 2003

Step 3: Updating n-1 n n+1 dj-1 evenj n-1 n n+1 Low-band (approximation of sj) EE591b Advanced Image Processing Copyright Xin Li 2003

Algorithmic Advantages
In-place operation: good for memory savings Computational efficiency: fewer floating operations than subband filtering implementations Parallelism: Inherent SIMO parallelism at all scales odd-length filter EE591b Advanced Image Processing Copyright Xin Li 2003

Structural Advantages
Inverse transform: simply run the split-prediction-updating backward, you will get the implementation of inverse transform (i.e., updating, prediction and merge) Generality: easy to be generalized into unconventional geometric settings such as curve, surface and volume EE591b Advanced Image Processing Copyright Xin Li 2003

Inverse Transform Reconstruct sj from (sj-1,dj-1) EE591b Advanced Image Processing Copyright Xin Li 2003

Forward vs. Inverse Forward transform Inverse transform obtain (sj-1,dj-1) from sj obtain sj from (sj-1,dj-1) EE591b Advanced Image Processing Copyright Xin Li 2003

Example (I) S-transform (a variant of Haar transform) EE591b Advanced Image Processing Copyright Xin Li 2003

Example (II) 5/3 transform (also called (2,2) interpolating transform) EE591b Advanced Image Processing Copyright Xin Li 2003

Generalization (I) Forward Transform Inverse Transform EE591b Advanced Image Processing Copyright Xin Li 2003

Factoring Wavelet Transform into Lifting Steps
Example: Daubechies’ 9-7 filter splitting P U P U scaling EE591b Advanced Image Processing Copyright Xin Li 2003

Generalization (II) Conventional subband-filtering based WT is not suitable for lossless coding (it is simply impossible to preserve real numbers with finite precision) Lifting scheme elegantly solves this problem because inverse transform is always guaranteed by lifting structure (so just round off those real numbers) EE591b Advanced Image Processing Copyright Xin Li 2003

Example Note: outputs (sj-1,dj-1) are both integers, just like the input sj Integer-to-integer (Reversible) 5/3 transform (Adopted by JPEG2000 for lossless image compression) EE591b Advanced Image Processing Copyright Xin Li 2003

Lifting scheme: unifying prediction and transform First-generation schemes FBI WSQ standard Second-generation schemes Probabilistic modeling of wavelet coefficients Embedded Zerotree Wavelet (EZW) SPIHT coder A unified where-and-what perspective JPEG2000 EE591b Advanced Image Processing Copyright Xin Li 2003

Early Attempts Each band is modeled by a Guassian random variable with zero mean and unknown variance (e.g., WSQ) Only modest gain over JPEG (DCT-based) is achieved Question: is this an accurate model? and how can we test it? EE591b Advanced Image Processing Copyright Xin Li 2003

FBI Wavelet Scalar Quantization (WSQ)
Each band is approximately modeled by a Gaussian r.v. k: band index image size Given R, minimize mk= subband size EE591b Advanced Image Processing Copyright Xin Li 2003

Rate Allocation Problem* LL HL Given a quota of bits R, how should we allocate them to each band to minimize the overall MSE distortion? LH HH Solution: Lagrangian Multiplier technique (we will study it in detail on the blackboard) EE591b Advanced Image Processing Copyright Xin Li 2003

Proof by Contradiction (I)
Assumption: our modeling target Ω is the collection of natural images Suppose each coefficient X in a high band does observe Gaussian distribution, i.e., X~N(0,σ2), then flip the sign of X (i.e., replace X with –X) should not matter and generates another element in Ω (i.e., a different but meaningful image) Let’s test it! EE591b Advanced Image Processing Copyright Xin Li 2003

Proof by Contradiction (II) DWT sign flip IWT EE591b Advanced Image Processing Copyright Xin Li 2003

What is wrong with that? Think of two coefficients: one in smooth region and the other around edge, do they observe the same probabilistic distribution? Think of all coefficients around the same edge, do they observe the same probabilistic distribution? Ignorance of topology and geometry EE591b Advanced Image Processing Copyright Xin Li 2003

The Importance of Modeling Singularity Location Uncertainty
Singularities carry critical visual information: edges, lines, corners … The location of singularities is important Recall locality of wavelets in spatial-frequency domain Singularities in spatial domain → significant coefficients in wavelet domain EE591b Advanced Image Processing Copyright Xin Li 2003

Where-and-What Coding Communication context Where The location of significant coefficients What The sign and magnitude of significant coefficients ? communication channel picture Alice Bob EE591b Advanced Image Processing Copyright Xin Li 2003

Embedded Zerotree Wavelet (EZW)’1993 Set Partition In Hierarchical Tree (SPIHT)’1995 Space-Frequency Quantization (SFQ)’ 1996 Estimation Quantization (EQ)’1997 Embedded Block Coding with Optimal Truncation (EBCOT)’2000 Least-Square Estimation Quantization (LSEQ)’2003 EE591b Advanced Image Processing Copyright Xin Li 2003

Embedded Zerotree Wavelet (EZW) Coding
T=T0 Dominant Pass Code the position information (where are the significant coefficients?) Core Engine Subordinate Pass Code the intensity information (what are the significant coefficients?) T=T/2 Reach the specified Bit rate? No Significance testing: |X|>T Yes EE591b Advanced Image Processing Copyright Xin Li 2003

Zerotree Data Structure
EE591b Advanced Image Processing Copyright Xin Li 2003

Ancestor-and-Descendent
Parent-and-Children Ancestor-and-Descendent EE591b Advanced Image Processing Copyright Xin Li 2003

Zerotree Terminology Zerotree root (ZRT): it and its all descendants are insignificant Isolated zero (IZ): it is insignificant but its descendant is not Positive significant (POS): it is significant and have a positive sign Negative significant (NEG): it is significant and have a negative sign EE591b Advanced Image Processing Copyright Xin Li 2003

Dominant Pass: Significance Testing (Where-coding)

Subordinate Pass: Magnitude Refinement (What-coding)
For Significant coefficients (POS/NEG), refine their magnitude by sending one bit indicating if it is larger than 1.5T, i.e., to resolve the ambiguity whether it is within [T,1.5T) or within [1.5T,2T) EE591b Advanced Image Processing Copyright Xin Li 2003

Toy Example EE591b Advanced Image Processing Copyright Xin Li 2003

Dominant Pass Note: T=32 LH1 contains POS LH1 contains POS EE591b Advanced Image Processing Copyright Xin Li 2003

Subordinate Pass 32 48 64 40 56 EE591b Advanced Image Processing Copyright Xin Li 2003

Where-and-What Interpretation
Zerotree data structure effectively resolves the location uncertainty (where) of insignificant coefficients The dominant and subordinate passes defined in EZW can be viewed as “where” and “what” coding respectively Dyadic choice of T values (i.e., T=128,64, 32,16,…) renders embedded coding EE591b Advanced Image Processing Copyright Xin Li 2003

A Simpler Two-Stage Coding
Position coding stage (where) Generate a binary map indicating the location of significant coefficients (|X|>T) Use context-based adaptive binary arithmetic coding (e.g., JBIG) to code the binary map Intensity coding stage (what) Code the sign and magnitude of significant coefficients EE591b Advanced Image Processing Copyright Xin Li 2003

A Different Interpretation Two-class modeling of high-band coefficients Significant class: |X|>T Insignificant class: |X|<T Why does classification help? Nonstationarity of image source A probabilistic modeling perspective EE591b Advanced Image Processing Copyright Xin Li 2003

Classification-based Modeling
Insignificant class Significant class Mixture EE591b Advanced Image Processing Copyright Xin Li 2003

Classification Gain Without classification With classification Classification gain EE591b Advanced Image Processing Copyright Xin Li 2003

Example EE591b Advanced Image Processing Copyright Xin Li 2003

Advanced Wavelet Coding SPIHT: a simpler yet more efficient implementation of EZW coder SFQ: Rate-Distortion optimized zerotree coder EQ: Rate-Distortion optimization via backward adaptive classification EBCOT (adopted by JPEG2000): a versatile embedded coder EE591b Advanced Image Processing Copyright Xin Li 2003

Another New Perspective
“What” and “Where” in human brain Ventral stream for object vision (what) Dorsal stream for spatial vision (where) If human vision system (HVS) understands the world in this where-and-what fashion and if we believe in the superiority of human intelligence, shouldn’t we represent images in a similar manner? Understanding HVS is as important as understanding image data EE591b Advanced Image Processing Copyright Xin Li 2003

From JPEG to JPEG2000 What is wrong with JPEG? • Poor low bit-rate performance • Separate lossy and lossless compression • Awkward progressive transmission • Do not support Region-Of-Interest (ROI) coding • Do not support random access and processing • Poor error resilience and security EE591b Advanced Image Processing Copyright Xin Li 2003

EBCOT System Overview Source image data WT Q C encoder channel Reconstructed image data IWT Q-1 C-1 decoder Embedded Block Coding with Optimized Truncation (EBCOT) EE591b Advanced Image Processing Copyright Xin Li 2003

What is new with EBCOT? Block tiling How is it different from block DCT? What do we buy from it? To support rate and resolution scalability To support ROI and random access To enhance error resilience capability R-D optimized truncation Implement R-D optimized embedded coding EE591b Advanced Image Processing Copyright Xin Li 2003

Scalable vs. Multicast What is scalable coding? Lena.pgm Lena.pgm lena.cod Lena_0.125bpp.cod Lena_0.25bpp.cod Lena_0.5bpp.cod Lena_1.00bpp.cod 0.25pp 0.5bpp 1bpp Multicast Scalable coding EE591b Advanced Image Processing Copyright Xin Li 2003

Spatial scalability 1 … EE591b Advanced Image Processing Copyright Xin Li 2003

SNR (Rate) scalability 1 … PSNR=30dB PSNR=40dB PSNR=35dB EE591b Advanced Image Processing Copyright Xin Li 2003

Embedded Zerotree Wavelet (EZW) Coding
T=T0 Dominant Pass Code the position information (where are the significant coefficients?) Subordinate Pass Code the intensity information (what are the significant coefficients?) T=T/2 Reach the specified Bit rate? No Yes EE591b Advanced Image Processing Copyright Xin Li 2003

Bit-Plane Coding MSB 1 1 1 1 1 1 1 1 1 1 1 1 1st pass 2nd pass 3rd pass … dominant LSB subordinate …… Successive refinement of coefficient magnitude EE591b Advanced Image Processing Copyright Xin Li 2003

Rate-Distortion Optimization in Scalable Image Coding
An old problem Given a bit budget, how to allocate them in such a way that the total distortion is minimized? A new challenge (due to embedded coding constraint) D a b We need to make sure R-D is optimized not only for a and d but also for b and c c b’ d c’ R R1 R2 EE591b Advanced Image Processing Copyright Xin Li 2003

Fractional Bit-plane Coding
sub-pass 1 sub-pass 2 sub-pass 3 MSB 1 1 1 1 1 1 1 1 1 1 1 1 … LSB …… EE591b Advanced Image Processing Copyright Xin Li 2003

Comparison between JPEG and JPEG2000 (I)
JPEG (0.25bpp) JPEG2000 (0.25bpp) EE591b Advanced Image Processing Copyright Xin Li 2003

Comparison between JPEG and JPEG2000 (II)
JPEG (0.5bpp) JPEG2000 (0.5bpp) EE591b Advanced Image Processing Copyright Xin Li 2003

JPEG2000 vs. WSQ Decoded fingerprint image by WSQ at compression ratio of 27 EE591b Advanced Image Processing Copyright Xin Li 2003

JPEG2000 vs. WSQ Decoded fingerprint image by JPEG2000 at compression ratio of 27 EE591b Advanced Image Processing Copyright Xin Li 2003

Region-Of-Interest (ROI) Coding

Block Tiling Tiling DWT on each tile DC level shifting EE591b Advanced Image Processing Copyright Xin Li 2003

Tile, Subband, Precinct and Block
code-block precinct Tile partitions into subbands, precincts and code-blocks EE591b Advanced Image Processing Copyright Xin Li 2003

Bit-plane Lifting Strategy
MSB MSB ROI BG ROI BG BG BG LSB LSB Scale up the coefficients in the region of interest EE591b Advanced Image Processing Copyright Xin Li 2003

Open Problems Related to Image Coding
Coding of specific class of images (e.g., Satellite, microarray, fingerprint) Coding of color-filter-array (CFA) images Error resilient coding of images Perceptual image coding Image coding for pattern recognition EE591b Advanced Image Processing Copyright Xin Li 2003

Coding of Specific Class of Images
How to design specific coding algorithms for each class? EE591b Advanced Image Processing Copyright Xin Li 2003

CFA Image Coding Approach I CFA Interpolation (demosaicing) Color image compression Approach II CFA data compression CFA Interpolation (demosaicing) Bayer Pattern Which one is better and why? EE591b Advanced Image Processing Copyright Xin Li 2003

Error Resilient Image Coding
channel encoder channel decoder source channel destination super-channel source encoder source decoder How can we optimize the end-to-end performance in the presence of channel errors? EE591b Advanced Image Processing Copyright Xin Li 2003

Perceptual Image Coding
Characterizing image distortion is difficult! How do we objectively define mage quality which has to be subject to individual opinions? EE591b Advanced Image Processing Copyright Xin Li 2003

Image Coding for PR image sensor Communication channel Pattern recognition How does coding distortion affect the recognition performance? We need to develop a new image representation which Can simultaneously support low-level (e.g., compression, denoising) and high-level (e.g., recognition and retrieval) vision tasks EE591b Advanced Image Processing Copyright Xin Li 2003

Roadmap to Lossy Image Compression

Similar presentations

Presentation on theme: "Roadmap to Lossy Image Compression"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Roadmap to Lossy Image Compression

Similar presentations

Presentation on theme: "Roadmap to Lossy Image Compression"— Presentation transcript:

Similar presentations

About project

Feedback