Image Processing and Coding 1

Image  Rich info. from visual data  Examples of images around us natural photographic images; artistic and engineering drawings scientific images (satellite, medical, etc.)  “Motion pictures” => video movie, TV program; family video; surveillance and highway/ferry camera A picture is worth 1000 words. A video is worth 1000 sentences? 2

Image Processing  Enhancement and restoration  Remove artifacts and scratches from an old photo/movie  Improve contrast and correct blurred images 3

Image Processing  Composition (for magazines and movies), Display, Printing …  Transmission and storage  images from oversea via Internet, or from a remote planet  Information analysis and automated recognition  Providing “human vision” to machines Face detection in CMU CS, 4

An Example  A Navigation system study for Accident Scene Analysis and Obstacle Avoidance Based on Mobile Multimedia sensor Platform 5

Image Processing  Medical imaging for diagnosis and exploration  Security, forensics and rights protection  Encryption, hashing, digital watermarking, digital fingerprinting … Visible Watermark invisible Watermark 6

Digital Image  “Exactness”  Perfect reproduction without degradation  Perfect duplication of processing result  Convenient & powerful computer-aided processing  Can perform sophisticated processing through computer hardware or software  Even kindergartners can do some!  Easy storage and transmission  1 CD can store hundreds of family photos!  Paperless transmission of high quality photos through network within seconds 7

Examples of Digital Image Processing  Compression  Manipulation and Restoration  Restoration of blurred and damaged images  Noise removal and reduction 8

Examples of Digital Image Processing  Applications  Face detection  Visible and invisible watermarking  Error concealment and resilience in video transmission 25% blocks in a checkerboard pattern are corrupted corrupted blocks are concealed via edge- directed interpolation (a) original lenna image (c) concealed lenna image (b) corrupted lenna image 9

 International Conf. on Image Processing 10

What is An Image? 11  Grayscale image  A grayscale image is a function I(x,y) of the two spatial coordinates of the image plane.  I(x,y) is the intensity of the image at the point (x,y) on the image plane.  I(x,y) takes non-negative values  assume the image is bounded by a rectangle [0,a]  [0,b] Color image – Can be represented by three functions, R(x,y) for red, G(x,y) for green, and B(x,y) for blue. y x y x I(x, y)

Matlab Example x=imread('lena.bmp'); imshow(x); [n,m]=size(x); x(5:40,5:40)=0; imshow(x) x(90:150, 90:150)=255; Imshow(x) 12

Different Ways to View an Image (More generally, to view a 2-D real- valued function) In 3-D (x,y, z) plot with z= I (x,y); red color for high value and blue for low Equal value contour in (x,y) plane Intensity visualization over 2-D (x,y) plane 13

Sampling and Quantization  Computer handles “discrete” data.  Sampling  Sample the value of the image at the nodes of a regular grid on the image plane.  A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j).  Quantization  Is a process of transforming a real valued sampled image to one taking only a finite number of distinct values.  Each sampled value in a 256-level grayscale image is represented by 8 bits. 0 (black) 255 (white) 14

Image Compression  Savings in storage and transmission  Multimedia data (esp. image and video) have large data volume  Difficult to send real-time uncompressed video over current network  Accommodate relatively slow storage devices  -they do not allow playing back uncompressed multimedia data in real time  1x CD-ROM transfer rate ~ 150 kB/s  320 x 240 x 24 fps color video bit rate ~ 5.5MB/s => 36 seconds needed to transfer 1-sec uncompressed video from CD 15

PCM coding  How to encode a digital image into bits?  Sampling and perform uniform quantization  “Pulse Coded Modulation” (PCM)  8 bits per pixel ~ good for grayscale image/video  bpp ~ needed for medical images  Reduce # of bpp for reasonable quality via quantization  Quantization reduces # of possible levels to encode  Visual quantization: companding, contrast quantization, dithering, etc.  Halftone use 1bpp but usually upsampling ~ saving less than 2:1 16

DiscussiononImprovingPCMDiscussion on Improving PCM  Quantized PCM values may not be equally likely  – Can we do better than encode each value using same # bits?  Example  P(“0” ) = 0.5, P(“1”) = 0.25, P(“2”) = 0.125, P(“3”) =  If use same # bits for all values  Need 2 bits to represent the four possibilities if treat  If use less bits for likely values “0” ~ Variable Length Codes (VLC)  “0” => [0], “1” => [10], “2” => [110], “3” => [111]  Use 1.75 bits on average ~ saves 0.25 bpp!  Bring probability into the picture  Now use prob. distr. to reduce average # bits per quantized sample 17

Variable Length Code 18  If the probabilities of (a, b, c, d) were (1/2,1/4,1/8,1/8), the expected number of its used to represent a source symbol would be bits/per symbol:

EntropycodingEntropy coding 19

E.g.ofEntropyCoding:HuffmanCodingE.g. of Entropy Coding: Huffman Coding 20

HuffmanCoding(cont’d)Huffman Coding (cont’d) 21

HuffmanCoding:Pros&ConsHuffman Coding: Pros & Cons  Pro  Simplicity in implementation (table lookup)  For a given block size Huffman coding gives the best coding efficiency  Con  Need to obtain source statistics  Improvement  Code a group of symbols as a whole to allow fractional # bit per symbol  Arithmetic coding  Lempel-Ziv coding or LZW algorithm  “universal”, no need to estimate source statistics 22

Discussion: Coding a Binary Image  How to efficiently encode it?  – “ …”  Run-length coding (RLC)  Code length of runs of “0” between successive “1”  run-length of “0” ~ # of “0” between “1”  good if often getting frequent large runs of “0” and sparse “1”  – E.g., => (4) (0) (3) (1) (6) (0) (0) … …   – Assign fixed-length codeword to run-length  Or use variable-length code like Huffman to further improve  RLC Also applicable to general binary data sequence with sparse “1” (or “0”) 23

24 Any Questions?

25 Things To Do  Find your partners for the class project  Feb. 11, Thursday ( me the team information)  Each group has 3-4 members  Check out the class website about  lecture notes  reading assignments Next Topics Image Compression-JPEG