Lecture 01 Introduction to Computer Vision Course: T Computer Vision Year: 2013

Learning Outcomes After carefully listening this lecture, students will be able to do the following:  Show what computer vision is all about and basic principles of various computer vision applications (LO1)  Explain approaches in computer vision systems (three level processing) (LO1)  Explain how a digital image is formed and other related matters such as light and visual perception (LO1) T Computer Vision2

Learning Outline What is Computer Vision? Image Processing Computer Vision Face Detection Text Detection & Recognition High Level Capability Approach in Computer Vision The Three Processing Level Image Formation Image Digitization Digital Image T Computer Vision3

What is Computer Vision? Ballard and Brown  The construction of explicit, meaningful description of physical objects from images. Forsyth and Ponce  Extracting descriptions of the world from pictures or sequences of pictures. T Computer Vision4

5 What is Computer Vision? A field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions.

Application of Computer Vision Interaksi manusia dan robot (Human Robot Interaction) Pengontrolan proses industri Pendeteksi nomor plat kendaraan Surveillance (monitor penyusup, analisa trafik jalan tol dan lainnya) Robot Vision, Humanoid Robot and Soccer Robot. Modeling obyek atau lingkungan T Computer Vision6

Image Processing T Computer Vision7

Image Enhancement T Computer Vision8

Image Processing (cont’d) Image Restoration (e.g., correcting out-focus images) T Computer Vision9

Computer Graphics Geometric modeling T Computer Vision10

Computer Vision T Computer Vision11

An Industrial Computer Vision System T Computer Vision12

T Computer Vision13 Low-level image analysis: Identify edges, regions Mid-level: Distinguish “cap” from “no cap” Estimation: What are orientation of cap, height of liquid? Cap Inspection System

Face Detection How is this like the bottle problem on the previous slide? T Computer Vision14

Text Detection & Recognition Similar to face finding: Where is the text and what does it say? Viewing at an angle complicates things... T Computer Vision15 from J. Zhang et al.

Text Detection & Recognition (cont’d) Text Detection and Recognition Apps  T Computer Vision16 Google Goggles

High Level Capability Computer Vision System (CVS) is expected to have high level capabilities like Human Visual System (HVS) does such as: Object detection – is an object present in the scene? If so, where is its boundaries Recognition – putting a label on an object Description – assigning properties to objects 3D inference – interpreting a 3D object from 2D views Interpreting motion T Computer Vision17

Approach in Computer Vision T Computer Vision18 3-D World Objects Images Objects Detection & Recognitions Edges/Region/Depth Models and Assumptions Features/Surfaces

The Three Processing Level Low-level processing  Standard procedures are applied to improve image quality  Procedures are required to have no intelligent capabilities T Computer Vision19

The Three Processing Level (cont’d) Intermediate-level processing  Extract and characterize components in the image  Some intelligent capabilities are required T Computer Vision20

The Three Processing Level (cont’d) High-level processing  Recognition and interpretation  Procedures require high intelligent capabilities T Computer Vision21

How are images represented in the computer? T Computer Vision22

Image Formation There are two parts to the image formation process:  The geometry of image formation, which determines where in the image plane the projection of a point in the scene will be located.  The physics of light, which determines the brightness of a point in the image plane as a function of illumination and surface properties. T Computer Vision23

Image formation (cont’d) Geometric parameters  type of projections  position and orientation of camera in space  perspective distortions introduced by the imaging process Photometric parameters  type, intensity, and direction of illumination  reflectance properties of the viewed surfaces Optical parameters of the lens  lens type  focal length  field of view T Computer Vision24

A Simple Model of Image Formation The scene is illuminated by a single source The scene reflects radiation towards the camera The camera senses it via chemicals on film T Computer Vision25

What is Light? The visible portion of the electromagnetic (EM) spectrum It occurs between wavelengths of approximately 400 and 700 nanometers T Computer Vision26

CCD (Charged-Coupled Device) Cameras Tiny solid state cells convert light energy into electrical charge. The image plane acts as a digital memory that can be read row by row by a computer. T Computer Vision27

Frame Grabber Usually, a CCD camera plugs into a computer board (frame grabber). The frame grabber digitizes the signal and stores it in its memory (frame buffer). T Computer Vision28

Image Digitization Sampling means measuring the value of an image at a finite number of points. Quantization is the representation of the measured value at the sampled point by an integer. T Computer Vision29

Image Digitization (cont’d) T Computer Vision30

Image Quantization T Computer Vision gray levels (8 bpp) 32 gray levels (5 bpp) 16 gray levels (4 bpp) 8 gray levels (3 bpp) 4 gray levels (2 bpp) 2 gray levels (1 bpp)

Image Sampling T Computer Vision32 Original Image Sampled by a Factor of 2 Sampled by a Factor of 4 Sampled by a Factor of 8

T Computer Vision33 Digital Image An image is represented by a rectangular array of integers. An integer represents the brightness or darkness of the image at that point. N: # of rows, M: # of columns, Q: # of gray levels  N =, M =, Q = (q is the # of bits/pixel)  Storage requirements: NxMxQ (e.g., N=M=1024, q=8, 1MB)

Image File Formats Many image formats adhere to the simple model shown below (line by line, no breaks between lines). The header contains at least the width and height of the image. Most headers begin with a signature or “magic number” - a short sequence of bytes for identifying the file format. T Computer Vision34

Common Image File Formats GIF (Graphic Interchange Format) PNG (Portable Network Graphics) JPEG (Joint Photographic Experts Group) TIFF (Tagged Image File Format) PGM (Portable Gray Map) FITS (Flexible Image Transport System) T Computer Vision35

OpenCV Computer vision is the most important technology in the future in the development of intelligent robot. Computer vision is In the simplest terms, computer vision is the discipline of "teaching machines how to see." OpenCV (Open Source Computer Vision Library) and released under a BSD license and hence it’s free for both academic and commercial use. It has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. T Computer Vision36

Color and grayscale grayscale image = ( (0.3 * R) + (0.59 * G) + (0.11 * B) ) T Computer Vision37

Optical Flow Optical flow or optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene. T Computer Vision38

Homework Install Visual Studio 2012 Express Edition and OpenCV Create a program to display an image and play a video. Create a program to display video from a Webcam T Computer Vision39

DisplayImage.cpp // Menampilkan Image menggunakan cvLoadImage #include "stdafx.h" #include int _tmain(int argc, _TCHAR* argv[]) { IplImage *img = cvLoadImage("f:\handsome.jpg"); cvNamedWindow("OpenCV",1); cvShowImage("OpenCV",img); cvWaitKey(0); cvDestroyWindow("OpenCV "); cvReleaseImage(&img); return 0; } T Computer Vision40

C++ 2.x Style //DisplayImage2.cpp #include "stdafx.h" #include using namespace std; int _tmain(int argc, _TCHAR* argv[]) { cv::Mat img=cv::imread("f:/fruits.jpg"); cv::imshow("Belajar OpenCV",img);//tampilkan cv::waitKey(); return EXIT_SUCCESS; } T Computer Vision41

Camera CvCapture* capture; capture = cvCaptureFromCAM(0); if( capture ) { while( true ) { frame = cvQueryFrame( capture ); T Computer Vision42

Face Library String face_cascade_name = "lbpcascade_frontalface.xml"; String eyes_cascade_name = "haarcascade_eye_tree_eyeglasses.xml"; T Computer Vision43

Final Project ( 5-6 person in group) Create a program using OpenCV Presentation 10 minutes in session 13. Example projects : 1.Face Recognition using AAM 2.Car Plate Recognition 3.Stereo Vision 4.Object Detection and Tracking 5.Image Processing for specific purposes. T Computer Vision44

References Textbook Forsyth, D., Ponce, J. (2012). Computer Vision: A Modern Approach. 2nd ed. Prentice Hall. ISBN: Web Lecture 01 Introduction to Computer Vision T Computer Vision45