Multimedia Processing Chapter 2. Digital Data Acquisition

Introduction Introduction

Issues Issues Sampling Quantization Bit Rate

Sampling xs (n )= x (nT) , where T is the sampling period f=1/T is the sampling frequency In reality, more complex, as sampling comes with filtering Surprisingly, we may not lose information in sampling

Quantization Quantization quantization is the process of approximating a continuous range of values (or a very large set of possible discrete values) by a relatively-small set of discrete symbols or integer values. Irreversible and lossy.

24-bit RGB color 16 colors Quantization 6 bits 5 bits 4 bits 3bits

Bit Rate Bit Rate = Sampling rate × Quantization per sample Audio (MP3) 32 kbit/s — MW (AM) quality 96 kbit/s — FM quality 128–160 kbit/s — Standard Bitrate quality; difference can sometimes be obvious (e.g. bass quality192 kbit/s — DAB (Digital Audio Broadcasting) quality. Quickly becoming the new 'standard' bitrate for MP3 music; difference can be heard by few people 224–320 kbit/s — Near CD quality. Sound is nearly indistinguishable from most CDs.

8 kbit/s — telephone quality (using speech codecs) Bit Rate Other audio 800 bit/s — minimum necessary for recognizable speech (using special-purpose FS-1015 speech codecs) 8 kbit/s — telephone quality (using speech codecs) 500 kbit/s–1 Mbit/s — lossless audio 1411 kbit/s — PCM sound format of Compact Disc Digital Audio

128 – 384 kbit/s — business-oriented videoconferencing system quality Bit Rate Video (MPEG2) 16 kbit/s — videophone quality (minimum necessary for a consumer-acceptable "talking head" picture) 128 – 384 kbit/s — business-oriented videoconferencing system quality 1.25 Mbit/s — VCD quality 5 Mbit/s — DVD quality 15 Mbit/s — HDTV quality 36 Mbit/s — HD DVD quality 54 Mbit/s — Blu-ray Disc quality

Signals and Systems Continuous and smooth Continuous and not smooth Neither smooth nor continuous Odd/Even Periodic: f(x+T)=f(x) Compact support: 0 outside finite interval g(t) periodic version of f(t): g(t)=Σkf(t-kT)

Linear Time Invariant Systems Linearity: Signals and Systems Linear Time Invariant Systems Linearity: yk(t) is the output resulting from the sole input xk(t). Time invariance means that whether we apply an input to the system now or T seconds from now, the output will be identical, except for a time delay of the T seconds. If the output due to input x(t) is y(t), then the output due to input x(t − T) is y(t − T). More specifically, an input affected by a time delay should effect a corresponding time delay in the output, hence time-invariant.

Signals and Systems Time/frequency

The convolution theorem states that Signals and Systems Convolution The convolution of  f   and   g    is written  f*g  . It is defined as the integral of the product of the two functions after one is reversed and shifted. The convolution theorem states that

Signals and Systems Useful signals The Dirac delta or Dirac's delta is a mathematical construct introduced by the British theoretical physicist Paul Dirac. Informally, it is a function representing an infinitely sharp peak bounding unit area: a function δ(x) that has the value zero everywhere except at x = 0 where its value is infinitely large in such a way that its total integral is 1.

Signals and Systems Sifting property

Signals and Systems Comb A Dirac comb is an infinite series of Dirac delta functions spaced at intervals of T.

Signals and Systems Step function

Signals and Systems Box

Signals and Systems Sinc Function

The Fourier Transform The Fourier Transform Fourier series are named in honor of Joseph Fourier (1768-1830), who made important contributions to the study of trigonometric series, after preliminary investigations by Euler, d'Alembert, and Bernoulli. He applied this technique to find the solution of the heat equation, publishing his initial results in 1807, and publishing his Théorie analytique de la chaleur in 1822.

This could not be solved analytically before! The Fourier Transform . Fourier This could not be solved analytically before!

What he said The Fourier Transform Multiplying both sides by and then integrating from y = − 1 to y = + 1 yields: For periodic functions of period T=1 Harmonic decomposition

Lagrange, Laplace, Legendre, : The Fourier Transform What they said Lagrange, Laplace, Legendre, : ...the manner in which the author arrives at these equations is not exempt of difficulties and [...] his analysis to integrate them still leaves something to be desired on the score of generality and even rigour. Existence? Convergence? Why exponentials? Finite support functions?

Fourier thought any function would work (not true) The Fourier Transform Properties Fourier thought any function would work (not true) Discontinuities (Gibbs effect)

The Fourier Transform Examples

Finite support  Infinite support Delta  constant Sinc  box The Fourier Transform Properties: duality |F(ω)|2 =|f(t)| 2 Periodic  Periodic Finite support  Infinite support Delta  constant Sinc  box Gaussian  Gaussian Comb  Comb

The Uniform Sampling Theorem A bandlimited signal f(t) with max frequency ωF is fully determined from its samples f (nT) is 2π/T > 2 ωF The continuous signal can then be reconstructed from its samples f (nT) by convolution with the filter r(t)=sinc(ωF(t-nT)/2π)

The Uniform Sampling Theorem Illustration

The Uniform Sampling Theorem Aliasing Artifacts resulting from an improper sampling frequency. Sound Images (Moire) Video (wagon wheel)

The Uniform Sampling Theorem

The Uniform Sampling Theorem Anti Aliasing In practice: smooth the data with a low-pass filter before sampling Issues: ideal filters cannot be realized

Homework Homework Read Text Chapter 3. Prepare Presentation