OpenCV Training course By Theerayod Wiangtong

Goals Develop a universal toolbox for research and development in the field of Computer Vision

Why use OpenCV? Fast development time, more than 500 algorithms in OpenCV libraries C/C++ based programming Both Windows and Linux supported Open and free, BSD license Loads of developers using OpenCV Loads of information and documents Etc

History of OpenCV Originally developed by Intel, currently maintained by Willow Garage

OpenCV - Features Cross-platform and extremely portable Free! for both research and commercial use Targeted for real-time applications Table Courtesy Learning OpenCV: Computer Vision with the OpenCV Library

OpenCV – Architecture & Modules CvAux Area for experimental algorithms: e.g. HMM, Stereo vision, 3D tracking, Bg/fg segmentation, camera calibration, Shape matching, Gesture recognition, ..

OpenCV Comparisons

Examples of Using OpenCV functions Click here

OpenCV: Algorithmic Content

(more than 500 algorithms!!) OpenCV Functionality Basic structures and operations Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction (more than 500 algorithms!!)

Image Thresholding Fixed threshold; Adaptive threshold;

Statistics min, max, mean value, standard deviation over the image Multidimensional histograms Norms C, L1, L2

Multidimensional Histograms Histogram operations : calculation, normalization, comparison, back project

Histogram Equalization

Histograms comparison

Image Pyramids Change the picture to something more clear!

Convolution in image The source pixel and its surrounding pixels are all mathematically merged to produce a single destination pixel. The matrix slides across the surface of the source image, producing pixels for the destination image http://beej.us/blog/data/convolution-image-processing/

Image Pyramids Gaussian and Laplacian

Morphological Operations Two basic morphology operations using structuring element: erosion dilation

Distance Transform Calculate the distance for all non-feature points to the closest feature point Two-pass algorithm, 3x3 and 5x5 masks, various metrics predefined

Flood Filling grayscale image, floating range grayscale image, fixed range

Feature Detection Fixed filters (Sobel operator, Canny operator, Laplacian, Scharr filter) Hough transform (find lines and circles) http://www.stevens-tech.edu/wireless/klin/EdgeDetection/EdgeDetectionInfo.htm

Edge detection operators This means: pixel(i,j) = 2*pixel(i,j) - pixel(i,j+1) - pixel(i+1,j). Simple Cross 2 -1 1 -1 1 -1 Template 1: Template 2:  pixel(i,j) = maximum(template 1, template 2)

Edge detection operators Prewitt Sobel 1 -1 1 -1 1 -1 2 -2 1 2 -1 -2 X-axis Template: Y-axis Template: pixel(i,j) = sqrt((x-axis template)^2 + (y-axis template)^2)

Canny Edge Detector http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html

Hough Transform

Another sample of using the Hough Transform Source picture Result

Contour Retrieving The contour representation: Chain code (Freeman code) Polygonal representation Initial Point Chain code for the curve: 34445670007654443 Contour representation

Hierarchical representation of contours Image Boundary (W1) (W2) (W3) (B2) (B3) (B4) Get the english picture! (W5) (W6)

Contours Examples Source Picture (300x600 = 180000 pts total) Retrieved Contours (<1800 pts total) After Approximation (<180 pts total) And it is rather fast: ~70 FPS for 640x480 on complex scenes

Contour Processing Approximation: RLE algorithm (chain code) Teh-Chin approximation (polygonal) Douglas-Peucker approximation (polygonal); Contour moments (central and normalized up to order 3) Matching of contours

Contours matching Matching based on hierarchical representation of contours

Object Recognition: Eigen Image

Object Recognition: HMM One person – one HMM Stage 1 – Train every HMM Stage 2 – Recognition Pi - probability Choose max(Pi) 1 … n Get the more clear pictures! i

Motion Analysis and Object Tracking Background subtraction Motion templates Optical flow Active contours Estimators

Background Subtraction Background: any static or periodically moving parts of a scene that remain static or periodic over the period of interest. How about waving trees, light on/off..?!?

Background statistics functions Average Standard deviation Connect component

Background Subtraction Example Reconsider this slide!

Motion Templates Object silhouette Motion history images Motion history gradients Motion segmentation algorithm MHG silhouette MHI

Motion Templates Example Motion templates allow to retrieve the dynamic characteristics of the moving object

Object tracking Mean-shift Cam-shift: Choose a search window (width and location) Compute the mean of the data in the search window Center the search window at the new mean location Repeat until convergence Cam-shift: Continuously Adaptive Mean SHIFT

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

Mean shift Region of interest Center of mass Slide by Y. Ukrainitz & B. Sarel

Object tracking Particle filter Optical flow, LK Optical flow is the relation of the motion field. It is a 2D projection of the physical movement of points relative to the observer Optical Flow Velocity vectors

OpenCV shape classification capabilities Contour approximation Moments (image&contour) Convexity analysis Pair-wise geometrical histogram Fitting functions (line, ellipse)

Using contours and geometry to classify shapes Given the contour classify the geometrical figure shape (triangle, circle, etc)

Moments Contour moments (faster) Hu invariants Here p is the x-order and q is the y-order, whereby order means the power to which the corresponding component is taken in the sum just displayed. E.g. m00 moment is actually just the length in pixels of the contour. Contour moments (faster) Not applicable for different sizes, orientation Hu invariants

Image segmentation Separate image into coherent “objects” human segmentation

Segmentation Methods Edge-based approach Color segmentation: histogram Apply edge detector (sobel, laplace, canny, gradient strokes). Find connected components in an inverted image Calculate the histogram. Find the objects of the selected histogram in the image.

OpenCV: Getting started

Getting Started Download OpenCV http://opencv.willowgarage.com/wiki/ There exists a short walkthrough video on YouTube at http://www.youtube.com/watch?v=9nPpa_WiArI Learning OpenCV: Computer Vision with the OpenCV Library by Gary Bradski and Adrian Kaehler http://proquest.safaribooksonline.com/9780596516130

OpenCV 2.1 with Visual Studio 2008 Download the OpenCV 2.1.0 Windows installer from SourceForge - "OpenCV-2.1.0-win32-vs2008.exe". Install it to a folder (without any spaces in it), say "C:\OpenCV2.1\". This article will refer to this path as $openCVDir During installation, enable the option "Add OpenCV to the system PATH for all users".

Configure Visual Studio 2008 Open VC++ Directories configuration: Tools > Options > Projects and Solutions > VC++ Directories Choose "Show directories for: Include files" Add "$openCVDir\include\opencv" Choose "Show directories for: Library files" Add "$openCVDir\lib" Choose "Show directories for: Source files" Add "$openCVDir\src\cv" Add "$openCVDir\src\cvaux" Add "$openCVDir\src\cxcore" Add "$openCVDir\src\highgui"

Configure your Project Open Project Properties: Project > %projectName% Properties... Open Linker Input properties: Configuration Properties > Linker > Input Open the "..." window to edit "Additional Dependencies" and on each line put: "cv210.lib" "cxcore210.lib" "highgui210.lib" And any other lib file, e.g, cvaux.lib, necessary for your project Your project should now build. If you get any errors try restarting Visual Studio and then doing a clean Rebuild.

More info http://opencv.willowgarage.com/documentation/c/index.html http://dasl.mem.drexel.edu/~noahKuntz/openCVTut1.html http://sapachan.blogspot.com/search/label/Learning%20OpenCV http://www.shervinemami.co.cc/introToOpenCV.html http://note.sonots.com/OpenCV/Install.html

Questions