The Johns Hopkins University Quantization Trac D. Tran ECE Department The Johns Hopkins University Baltimore, MD 21218

Outline Review Quantization Nonlinear mapping Forward and inverse quantization Quantization errors Clipping error Approximation error Error model Optimal scalar quantization Examples

Reminder original signal compressed bit-stream reconstructed signal Information theory VLC Huffman Arithmetic Run-length Quantization

Quantization Entropy coding techniques Quantization Perform lossless coding No flexibility or trade-off in bit-rate versus distortion Quantization Lossy non-linear mapping operation: a range of amplitude is mapped to a unique level or codeword Approximation of a signal source using a finite collection of discrete amplitudes Controls the rate-distortion trade-off Applications A/D conversion Compression

Typical Quantizer Q Forward Quantizer x y input output y 111 110 101 000 001 010 011 100 quantization level or codeword clipping/overflow quantization cell/bin/interval x decision boundaries clipping/overflow

Typical Inverse Quantizer Typical reconstruction Quantization error y x Q Inverse Quantizer ^ -1 y 111 110 101 100 011 010 001 000 x clipping, overflow decision boundaries

Mid-rise versus Mid-tread y y x x Uniform Midrise Quantizer Uniform Midtread Quantizer Popular in ADC For a b-bit midrise Popular in compression For a b-bit midtread

Quantization Errors Approximation error Clipping error Lack of quantization resolution, too few quantization levels, too large quantization step-size Causes staircase effect Solution: increases the number of quantization levels, and hence, increase the bit-rate Clipping error Inadequate quantizer range limits, also known as overflow Solution Requires knowledge of the input signal Typical practical range for a zero-mean signal

Quantization: Error Model Assumptions: x Q ^ -1 + x ^ Quantization error: Mean-squared distortion measure:

Quantization Error Variance + x ^ would like to minimize

Uniform Quantization – Bounded Input y x b-bit Quantizer x q q high bit-rate assumption

Uniform Quantization – Bounded Input x q q b-bit quantizer high bit-rate assumption

Signal-to-Noise Ratio Definition of SNR in decibel (dB) power of the signal power of the noise For quantization noise Suppose that we now add 1 more bit to our Q resolution:

Example Design a 3-bit uniform quantizer for a signal with range [0,128] Maximum possible number of levels: Quantization stepsize: Quantization levels: Reconstruction levels: Maximum quantization error:

Example of Popular Quantization Round Floor Ceiling y x Uniform midtread quantizer from Round and Floor

Quantization from Rounding y –14 –10 –6 –2 2 6 10 14 x Uniform Quantizer, step-size=4

Dead-zone Scalar Quantization The bin size around zero is doubled Other bins are still uniform Create more zeros Useful for image/video Output ∆ 2∆ 3∆ Input -3∆ -2∆ -∆ 3.5∆ 2.5∆ 1.5∆ -1.5∆ -2.5∆ -3.5∆ Quantization mapping: De-quantization mapping: -2∆ -∆ 2∆ ∆ x

Non-Uniform Quantization Uniform quantizer is not optimal if source is not uniformly distributed For given M, to reduce MSE, we want narrow bin when f(x) is high and wide bin when f(x) is low f(x) x

Optimal Scalar Quantization Problem Statement: Optimal Encoder for a Given Decoder: Notes: Non-uniform quantizer under consideration Reconstruction can be anywhere, not necessarily the center of the interval

Optimal Scalar Quantization Fundamental Theorem of Calculus Nearest Neighbor Rule

Optimal Scalar Quantization Optimal Decoder for a Given Encoder: Centroid Rule

Lloyd-Max Quantizer Main idea [Lloyd 1957] [Max 1960] Assumptions solving these 2 equation iteratively until D converges Assumptions Input PDF is known and stationary Entropy has not been taken into account Nearest Neighbor Partitioning Centroid Computation Input Codebook Updated Codebook index of m-th iteration

Example y –b –a x a b x

Example y –b –a x a b x

Embedded Quantization ^ x y x -1 Q Q x S MSB LSB F1 F2 y Discard N integer bit planes + all fractional bit planes Also called bit-plane quantization, progressive quantization Most significant information is transmitted first JPEG2000 quantization strategy

Embedded Quantization R = 1 R = 2 R = 3

Embedded Forward Quantization y 16 12 8 –16 –12 –8 4 –4 x Dead Zone Embedded Quantizer, N=2

Embedded Inverse Quantization 1 X 1 Truncate 4 bit planes X 1 Receive 1 refinement bit X 1 Receive 2 refinement bits Original symbol x = 22 Range=[16, 32) Range=[16, 24) Range=[20, 24) ^ ^ ^ x = 24 x = 20 = 24 – 4 x = 22 = 20 + 2 N-bit-plane truncation = scalar quantization with

Vector Quantization n-dimensional generalization of scalar quantizer Nearest neighbor and centroid rule still apply n-dimensional input vectors codebook, containing code-vectors or codewords Vector Q Separable Scalar Q