1. 1 Introduction to Digital Images Lecture on the image part (#1) Automatic Perception (AP1) Volker Krüger Aalborg Media Lab Aalborg University Copenhagen vok@media.aau.dk

2. 2 Who am I? Graduated 1996 as a Computer Scientist (Dipl. Inf.) from the University of Kiel, Germany, Specialized in computer vision Ph.D. (Dr. Ing.) in Kiel, 2000 Work at the University of Maryland, Washington DC Research interests: Computer vision, motion capture, human gesture recognition, Biometrics, AI Teaching since 1996 –Supervision –Signal (image) processing, pattern recognition I am with Medialogi since its start in Fall 2002

3. 3 Why are digital image interesting? Humans are visual creatures in a visual world Images are (often) the primary sense –Imagine you could only keep one sense… ”A picture is worth a 1000 words” Words are many times ambiguous So, if we want to build systems capable of human skills, then they should be capable of understanding images (many applications) Images in a computer are DIGITAL, as opposed to analog images in an (old) photo-camera

4. 4 Why are digital images interesting?

5. 5 The (rough) plan for the 15 lectures Understand the media: Images –#1 + all others Understand how images can be manipulated in order to create visual effects –#2-11 Generate control signals for an application (project) –Video processing –#12-14 Summary: #15 PE-course: Let me know your needs!

6. 6 General information AP lectures –Timing What is most suitable Lecture, break, lecture, exercises –Feel free to interrupt! –Exams: How to pass –Slides: for main key words, images, videos, graphs Necessary to make notes!!! –Questions and participation during the lectures –Leave your comments at the end of each lecture at the exit!!

7. 7 General Information Literature –”Digital image processing” Nick Efford –Will not use the Java part –Alternative: Gonzalez & Woods, Digital Image Processing Web –Literature, slides, PE-questions, key-words, Slides will be online latest 24h before the lecture. Exercises –In the end of the slides, –We will talk about the exercises in the beginning of each lecture. –Florian Pilz is the student helper. SW –EyesWeb

8. 8 Change in Schedule Thu, 9:00-10:30: Exercise 10:40-12:00 Lecture Tue: Please check the schedule after Thursday carefully for possible changes !!!

9. 9 Plan for today Digital images –Applications, definitions Color images Exercises

10. 10 Applications

11. 11 Digital Image Why digital ? Before 1920: Image transmission from USA to Europe: more than a week: by ship! Early 1920s: Bartline cable picture transmission system: Transmission in three hours! Transmission via telegraph/wire, radio signals for newspapers

12. 12 Small Progress Small progress in digital imaging until 1964 Jet Propulsion Lab (JPL) in Pasadena, CA –Transmission and correction of lunar images from Ranger 7. Not so good quality so the images had to be processed before they could be viewed Since then many applications…

13. 13 Examples: Image Correction Needed when image data is erroneous: –Bad transmission –Bits are missing: Salt & Pepper Noise

14. 14 Image Deblurring: Motion Blur Can be used when a camera or object is moved during exposure

15. 15 Deblurring Can be used when the camera was not focused properly!!

16. 16 Image manipulation Image improvement, e.g. too dark image Rotate + scale

17. 17 Medical Image Processing Image Processing is widely used E.g. Analysis of microscopic images

18. 18 Medical Image Processing MR/CT Imaging of a human body Use for Brain Surgery

19. 19 Conveyer belt applications Checking and sorting –For example: checking bottles in the supermarket Quality control –Does the object have the correct dimensions, color, shape, etc.? –Is the object broken? Robot control –Find precise location of the object to be picked

20. 20 Biometrics Recognizing/verifying the identity of a person by analyzing one or more characteristics of the human body Characteristics: –Fingerprint, eye (retina, iris), ear, face, heat profile, shape (3D face, hand), motion (gait, writing), … Applications: –Verifying: Access control (bio-passports) –Recognizing: Surveillance: 9/11

21. 21 Chroma keying

22. 22 Analysis of Sport Motions Here: Analysis of motion of Sarah Hughes 3D Tracking of body parts Motion interpretation Action recognition

23. 23 Motion Capture Andy Serkis Special effects –Advertising –Movies

24. 24 Motion Capture

25. 25 Why should MED-students learn about digital images? Because images are fun Understand the media: Images Understand the tools: e.g. Photoshop Understand how images can be manipulated in order to create visual effects –Rotation, scaling, blurring, etc. As done in standard image manipulation tools Remove parts of an image –Combine graphics with real images –Combine (part of) one image with another Generate control signals for an application (project) –Understand how to find and follow (well defined) objects in an image Recognize objects (many industrial applications)

26. 26 What you will be able to do at the end of this semester

27. 27 Why should MED-students learn about digital images? Discuss this during the break!

28. 28 Websites The course website is: www.media.aau.dk/med3  courses  ip Site contains: Plan for the lectures, links to slides, Software Exercises

29. 29 Image definitions

30. 30 Where does an image come from?

31. 31 Where does an image come from? Charged coupled device CCD-chip

32. 32 Where does an image come from? Integration over time –Exposure time –Maximum charge Saturation Blooming Under exposed Over exposed Correct exposed

33. 33 Where does an image come from? Image elements, picture elements, pels, pixels

34. 34 Imaging system Image acquisition Illumination –Passive: sun –Active: ordinary lamp, X-ray, radar, IR Camera lens –Focus the light on the CCD chip

35. 35 Digital Image Representation Image is seen as a discrete function f(x,y) as opposed to a continuous function (show) x and y cannot take on any value! y x f(x,y) Origin

36. 36 Discrete image coordinate system y x f(x,y) Origin y x f(0,0) f(?,?) f(2,6)

37. 37 Digital Image Representation An image f(x,y) is represented as an Array Width = number of pixels in x-direction Height = number of pixels in y-direction Size (width x height, width > height) ROI = region of interest –To reduce the amount of data Width Height ROI

38. 38 Spatial Image Resolution: Resolution –The size of an area in a scene that is represented by one pixel in the image Different Resolutions are possible (256x256….16x16) Lower resolution leads to data reduction!

39. 39 Digital Image Representation Pixel representation (bits) –A few words on bits and bytes: One bit: {0,1} One byte = eight bits –One pixel: one byte = eight bits = one number: [0,255] (show) –Grey-scale, intensity, black/white: 8 bits = [0,255] –Binary image: 1 bit {0,1}. Black and white: visualized as: 8 bit {0,255} –Colors: after the break Image representation (2D image versus 3D data) –(show: 2D-gel: crop lower left corner)

40. 40 Gray-level Resolution: Quantization Different gray-level resolutions: 256, 128, …, 2 Less gray-levels leads to data reduction. For 256, 128, 64 gray-levels: Difference hardly visible

41. 41 Working with images…. Image manipulation –Simple operations, e.g., scale image Image processing –Improve the image, e.g., remove noise Image analysis –Analyze the image, e.g., find the person in the image Machine vision –Industry, e.g., Quality control, Robot control Computer vision –Everything: multiple cameras, video-processing, etc.

42. 42 Fundamental Steps in Computer Vision Knowledge base Problem domain Image acquisition Preprocessing Segmentation Representation and description Recognition and Interpretation Result Point 1: 22,33 Point 2: 24, 39 ….. Actor sitting

43. 43 Image file types image.jpg, image.tif, image.gif, image.png, image.ppm, …. Raw: –No data is lost –Header + data (234 235 32 21…) –For example: image.pgm –The file can be viewed Lossless compression: –No data is lost, but the file cannot be viewed –For example: image.gif Lossy compression: –Better compression –Some data is lost (optimized from the HVS’ point of view) –The file cannot be viewed –For example: image.jpg

44. 44 Image file types Normally you don’t care about the file type –The application will take care of it for you: –For example: rotate Application –image.x => raw –Rotate the raw image –Rotated raw => rotated_image.x But to write your own programs from scratch the images need to be in the raw format (without a header). EyesWeb will do this for you

45. 45 What have you learned today?

46. 46 Exercises Questions to the lecture? What was good about the lecture and what could have been better? Discuss the P0-Projects in the light of what you have heard, today. What other image applications can you think of? Given a 512 x 512 x 8bit image. How is the memory size reduced when you: –Decrease the grayscale resolution repeatedly by 2 –Decrease the size of the image repeatedly by 2

47. 47 PE-Questions 1) Name a few application areas where computer vision can be applied 2) What is the difference between discrete images and continuous images 3) What is a CCD chip? 4) What does it mean that a pixel is saturated? 5) How do you calculate the amount of memory needed to store a raw image? 6) What is Quantization? 7) How do you convert from a binary string, for example [1 0 1 1 0 0 1 0], to a standard number (base 10)?