EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 17 Wenbing Zhao wenbing@ieee.org

Outline Emgu CV Software installation Configure a new project with Emgu CV Working with Emgu CV images

Software Needed OpenCV does not support C#: cannot use it directly We will be using Emgu CV: a cross platform .Net wrapper for OpenCV http://www.emgu.com/wiki/index.php/Main_Page Due to unknown reason, later version of EmguCV does not work on x64 Windows platform Latest version that worked for me is: http://sourceforge.net/projects/emgucv/files/emgucv/2.2.1/libemgucv-windows-x64-2.2.1.1150.zip/download

Configuration with EmguCV It is tricky to use EmguCV in Visual Studio 2010 You must add the following EmguCV dll files as references Emgu.CV.dll Emgu.CV.UI.dll  Emgu.Util.dll   Furthermore, you must copy at least the following OpenCV dlls to your project’s bin/Debug or bin/Release directory opencv_core220.dll     opencv_imgproc220.dll Infrared data: 16 bits per pixel at 640x480 at 30 FPS

Configuration with EmguCV Alternatively, you can add opencv dlls as files into your project. For each dll, right click and select properties. Then choose “copy always”. The dll will be copied to the output directory for you. More one critical step. Right click your project name in the “Solution Explorer” pane, and select properties. Then, choose the “Build” tab and change the platform target to “x64”

Configuration with EmguCV

Working with EmguCV Images EmguCV Image Class Creating Image Getting/setting pixels Drawing objects on image 5/22/2018 EEC492/693/793 - iPhone Application Development

Generic Parameters for EmguCV Images Image<TColor, TDepth> Depth: number of bits used to represent the color of a single pixel Color: type of color representation. Emgu CV supports Supported TDepth types: Byte SByte Single (float) Double UInt16 Int16 Int32 (int) Supported TColor types: Gray Bgr (Blue Green Red) Bgra (Blue Green Red Alpha) Hsv (Hue Saturation Value) Hls (Hue Lightness Saturation) Lab (CIE L*a*b*) Luv (CIE L*u*v*) Xyz (CIE XYZ.Rec 709 with D65 white point) Ycc (YCrCb JPEG) Image<Gray, Byte> image = new Image<Gray, Byte>( width, height);

Creating EmguCV Images Creating a new image No background With blue background Creating an image from an existing file Creating an image from bitmap Image<Bgr, Byte> img1 = new Image<Bgr, Byte>(480, 320); Image<Bgr, Byte> img1 = new Image<Bgr, Byte>(480, 320, new Bgr(255, 0, 0)); Image<Bgr, Byte> img1 = new Image<Bgr, Byte>("MyImage.jpg"); Image<Bgr, Byte> img = new Image<Bgr, Byte>(bmp); 5/22/2018 EEC492/693/793 - iPhone Application Development

Accessing Image Data (Pixels) Direct access to Image //Color Image My_Image[0, 0] = new Bgr(Color.Red); //Gray Image gray_image[0, 0] = new Gray(200); //Color Image Bgr my_Bgr = My_Image[0, 0]; //Gray Image Gray my_Gray = gray_image[0, 0];

Accessing Image Data (Pixels) Access via Image.Data property Must respect the image depth when accessing the Data property A grayscale image has a depth of one (i.e., 1 bit) [x,y,0] RGB color image has a depth of 3 (i.e., 23 bits or one Byte) [x,y,0], [x,y,1], [x,y,2] represent the blue, green, and red spectrums //Color Image Color R = Color.Red; My_Image.Data[0,0,2] = R.R; //Write to the Red Spectrum My_Image.Data[0,0,1] = R.G; //Write to the Green Spectrum My_Image.Data[0,0,0] = R.B; //Write to the Blue Spectrum //Gray Image gray_image[0, 0] = new Gray(200);

Accessing Image Data: Examples Using column per row access: http://stackoverflow.com/questions/5101986/iterate-over-pixels-of-an-image-with-emgu-cv private void settingPixelsSlow(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; Bgr color = new Bgr(100, 40, 243); evaluator.Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img.Cols; j++) { for (int i = 0; i < img.Rows; i++){ img[i, j] = color; } evaluator.Stop(); Console.WriteLine("Average execution time for {0} iteration \n using column per row access: {1}ms\n", repetitions, evaluator.ElapsedMilliseconds / repetitions);

Accessing Image Data: Examples Using Data property private void settingPixelsFaster(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; evaluator.Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img.Cols; j++) { for (int i = 0; i < img.Rows; i++) { img.Data[i, j,0] = 100; img.Data[i, j, 1] = 40; img.Data[i, j, 2] = 243; } evaluator.Stop(); Console.WriteLine("Average execution time for {0} iteration \n using Data property: {1}ms\n", repetitions, evaluator.ElapsedMilliseconds / repetitions);

Accessing Image Data: Examples Using Data property outside the loop: private void settingPixelsFastest(Image<Bgr, Byte> img) { Stopwatch evaluator = new Stopwatch(); int repetitions = 100; byte[, ,] data = img.Data; evaluator.Start(); for (int run = 0; run < repetitions; run++) { for (int j = 0; j < img.Cols; j++) { for (int i = 0; i < img.Rows; i++) { data[i, j, 0] = 100; data[i, j, 1] = 40; data[i, j, 2] = 243; } evaluator.Stop(); Console.WriteLine("Average execution time for {0} iteration \n using Data property smartly: {1}ms\n", repetitions, evaluator.ElapsedMilliseconds / repetitions);

Automatic Garbage Collection The Image<TColor, TDepth> class automatically take care of the memory management and garbage collection Once the garbage collector decided that there is no more reference to the Image<TColor, TDepth> object, it will call the Disposed method, which release the unmanaged IplImage structure using (Image<Gray, Single> image = new Image<Gray, Single>(1000, 800)) { ... //do something here in the image } //The image will be disposed here and memory freed

First EmguCV WPF App Basic image type is: Image<Rgb,byte> It is a wrapper for IplImage class in OpenCV Image<> offers many constructors public Image(Bitmap bmp); public Image(string fileName); public Image(int width, int height); public Image(int width, int height, TColor value); public Image(int width, int height, int stride, IntPtr scan0); In the app, we will create a new Image object, draw “Hello world”, and display via WPF image control TColor value: image color

First EmguCV WPF App Create a new WPF project Add emgu cv dlls and opencv dlls Add System.Drawing reference Change platform target to x64 Add one image control in MainWindow Draw “Hello world” using EmguCV Image<> Convert the image to BitmapSource using the provided helper function Set the BitmapSource as the WPF image control source

First EmguCV WPF App Add the following namespaces: using Emgu.CV; using Emgu.CV.CvEnum; using Emgu.CV.Structure; // for Image to BitmapSource conversion using System.IO; using System.Runtime.InteropServices; using System.Drawing; using System.Drawing.Imaging; // for Stopwatch using System.Diagnostics;

First EmguCV WPF App public MainWindow() { InitializeComponent(); //Create an image of 400x200 of Blue color Image<Bgr, Byte> img = new Image<Bgr, byte>(400, 200, new Bgr(255, 0, 0)); MCvFont f = new MCvFont(FONT.CV_FONT_HERSHEY_COMPLEX, 1.0, 1.0); //Create the font //Draw "Hello, world." on the image using the specific font img.Draw("Hello, world", ref f, new System.Drawing.Point(10, 80), new Bgr(0, 255, 0)); BitmapSource imgsource=ToBitmapSource(img); image1.Source = imgsource; }

First EmguCV WPF App // taken from: http://blogs.claritycon.com/blog/2012/11/blob-tracking-kinect-opencv-wpf/ [DllImport("gdi32")] private static extern int DeleteObject(IntPtr o); public BitmapSource ToBitmapSource(IImage image) { using (System.Drawing.Bitmap source = image.Bitmap) { IntPtr ptr = source.GetHbitmap(); //obtain the Hbitmap BitmapSource bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap( ptr, IntPtr.Zero, Int32Rect.Empty, System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions()); DeleteObject(ptr); //release the HBitmap return bs; }}

Have a little more fun EmguCV Image also provides a number of drawing methods Draw a Circle of the specific color and thickness // thickness: If thickness is less than 1, the circle is filled up void Draw(CircleF circle, TColor color, int thickness); // others void Draw(Ellipse ellipse, TColor color, int thickness); void Draw(IConvexPolygonF polygon, TColor color, int thickness); void Draw(LineSegment2D line, TColor color, int thickness); void Draw(Rectangle rect, TColor color, int thickness); PointF pt = new PointF(50, 50); CircleF circ = new CircleF(pt, 50); img.Draw(circ, new Bgr(0, 255, 0), 2);

Accessing Pixels in Image Try out the three ways of accessing pixels in the image Draw more customized patterns in the image