5/19/2015 EEC484/584: Computer Networks 1 EEC-490 Senior Design (CE) Kinect Programming Tutorial 1 Wenbing Zhao

Slides:



Advertisements
Similar presentations
Page 1 | Microsoft Work With Depth Data Kinect for Windows Video Courses Jan 2013.
Advertisements

EEC-492/592 Kinect Application Development Lecture 15 Wenbing Zhao
Work With Skeleton Data
Kinect SDK Tutorial Skeleton and Camera (RGB) Anant Bhardwaj.
By Rishabh Maheshwari. Objective of today’s lecture Play Angry Birds in 3D.
Chapter 6 Graphical User Interface (GUI) and Object-Oriented Design (OOD)
Page 1 | Microsoft Work With Color Data Kinect for Windows Video Courses Jan 2013.
®® Microsoft Windows 7 for Power Users Tutorial 2p2 Customizing Microsoft Windows 7.
Page 1 | Microsoft Streams sync and coordinate mapping Kinect for Windows Video Courses.
Graphics and Multimedia In visual Studio. Net (C#)
Microsoft Visual Basic 2012 CHAPTER TWELVE (ENRICHMENT CHAPTER) Windows Store Apps.
ASP.Net, Web Forms and Web Controls 1 Outline Introduction Simple HTTP Transaction System Architecture Creating and Running a Simple Web Form Example Web.
EEC-492/592 Kinect Application Development
EEC-492/592 Kinect Application Development Lecture 10 Wenbing Zhao
Neal Stublen Class Objectives  Develop an understanding of the.NET Framework  Gain proficiency using Visual Studio  Begin learning.
Kinect SDK Crash Course (In 12 slides or less) Elliot Babchick.
Tutorial 11 Using and Writing Visual Basic for Applications Code
® Microsoft Access 2010 Tutorial 11 Using and Writing Visual Basic for Applications Code.
An Introduction to Silverlight Matt Harrington Developer Evangelist, Microsoft October 20, 2011.
Page 1 | Microsoft Work With Skeleton Data Kinect for Windows Video Courses Jan 2013.
Page 1 | Microsoft Work With Color Data Kinect for Windows Video Courses Jan 2013.
1 EEC-492/592 Kinect Application Development Lecture 2 Wenbing Zhao
1 Chapter Eleven Handling Events. 2 Objectives Learn about delegates How to create composed delegates How to handle events How to use the built-in EventHandler.
12/5/2015 EEC492/693/793 - iPhone Application Development 1 EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 4 Wenbing Zhao
12/20/2015 EEC484/584: Computer Networks 1 EEC-490 Senior Design (CE) Lecture 1 Wenbing Zhao
KINECT FOR WINDOWS Ken Casada Developer Evangelist, Microsoft Switzerland | blogblog.
Introduction to Kinect For Windows SDK
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 8 Wenbing Zhao
Simple Clicker App WPF App using C#. App Requirement Need a ‘counter’ display, which starts at 0 Need a ‘clicker’ button ! Pressing the clicker every.
2/16/2016 EEC492/693/793 - iPhone Application Development 1 EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 4 Wenbing Zhao
1NetBeans Tutorial Using the “Properties” menu, name the List “List1” and the button “Button1”. NetBeans Tutorial6.
COMPREHENSIVE Access Tutorial 11 Using and Writing Visual Basic for Applications Code.
3/3/2016 EEC492/693/793 - iPhone Application Development 1 EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 4 Wenbing Zhao
REC [ ] How to implement CAMERA RECORDING for USB WEBCAM or IP CAMERA in C#.NET SOURCE CODE: ! Welcome to this presentation that explains.
EEC-693/793 Applied Computer Vision with Depth Cameras Lecture 9 Wenbing Zhao
1/64. 2/64 3/64 4/64 5/64 6/64 7/64 8/64 9/64.
Introduction to Microsoft Kinect Sensor Programming
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
Chapter Eleven Handling Events.
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
Objectives Learn about Function procedures (functions), Sub procedures (subroutines), and modules Review and modify an existing subroutine in an event.
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-490 Senior Design (CE)
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-492/592 Kinect Application Development
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
EEC-693/793 Applied Computer Vision with Depth Cameras
Tutorial 11 Using and Writing Visual Basic for Applications Code
Presentation transcript:

5/19/2015 EEC484/584: Computer Networks 1 EEC-490 Senior Design (CE) Kinect Programming Tutorial 1 Wenbing Zhao

Kinect Application Development Requirement Hardware: a relatively powerful Windows 7 PC/Laptop (Windows XP not supported) Software  Visual Studio 2010 (or Express) for C# (install this first!)  Kinect for Windows SDK  For speech recognition Speech platform runtime (32-bit & 64-bit) Microsoft speech platform sdk MS Kinect English language pack

Kinect Getting Started Basic Create a new Windows Presentation Foundation app  File->New->Project…  Select “WPF Application”, name the project “KinectBasic”, click OK Add reference to Kinect library  Right click References =>Add Reference…  Select Microsoft.Research.Kinect =>OK

Kinect Getting Started Basic Define UI  TextBlock Text FontSize  Button Content  Images Leftmost image: change variable name to image Middle image: change variable name to imageCmyk32 Rightmost image: change variable name to depth

Kinect Getting Started Basic Add events  When “Start” / “Stop” button is pushed In Properties pane => Events tab In “Click” field: can specify event method name Otherwise, a default method name is used (in the form buttonName_Click())

Kinect Getting Started Basic MainWindow.xaml.cs  Add header include using Microsoft.Research.Kinect.Nui;  Add member variable In public partial class MainWindow:Window {  Runtime kinectNui;  int totalFrames = 0; // for frame rate calculation  int lastFrames = 0; // frame rate  DateTime lastTime = DateTime.MaxValue; // frame rate  const int RED_IDX = 2;  const int GREEN_IDX = 1;  const int BLUE_IDX = 0;  byte[] depthFrame32 = new byte[320 * 240 * 4]; // for frame manipulation

Kinect Getting Started Basic MainWindow.xaml.cs  Add Kinect init method: private void InitializeNui() {}  Declares kinectNui as a Runtime object, which represents the Kinect sensor instance nui = Runtime.Kinects[0];  Open the video and depth streams, and sets up the event handlers that the runtime calls when a video, depth, or skeleton frame is ready  An application must initialize the Kinect sensor by calling Runtime.Initialize before calling any other methods on the Runtime object kinectNui.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor);

Kinect Getting Started Basic MainWindow.xaml.cs, InitializeNui{}  To stream color images: The options must include UseColor Valid image resolutions are Resolution1280x1024 and Resolution640x480 Valid image types are Color, ColorYUV, and ColorYUVRaw kinectNui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.ColorYuv);

Kinect Getting Started Basic MainWindow.xaml.cs, InitializeNui{}  To stream depth and player index data: The options must include UseDepthAndPlayerIndex Valid resolutions for depth and player index data are Resolution320x240 and Resolution80x60 The only valid image type is DepthAndPlayerIndex kinectNui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);

Kinect Getting Started Basic MainWindow.xaml.cs, InitializeNui{}  Init timestamp for frame rate calculation: lastTime = DateTime.Now;  Add event handler to handle incoming frames kinectNui.VideoFrameReady += new EventHandler (NuiVideoFrameReady); kinectNui.DepthFrameReady += new EventHandler (NuiDepthFrameReady);

Kinect Getting Started Basic MainWindow.xaml.cs  Video frame ready event handler void NuiVideoFrameReady(object sender, ImageFrameReadyEventArgs e) { PlanarImage Image = e.ImageFrame.Image; image.Source = BitmapSource.Create( Image.Width, Image.Height, 96, 96, PixelFormats.Bgr32, null, Image.Bits, Image.Width * Image.BytesPerPixel); imageCmyk32.Source = BitmapSource.Create( Image.Width, Image.Height, 96, 96, PixelFormats.Cmyk32, null, Image.Bits, Image.Width * Image.BytesPerPixel); }

Kinect Getting Started Basic MainWindow.xaml.cs: Depth frame ready event handler  Depth is different because the image you are getting back is 16-bit and we need to convert it to 32-bit void NuiDepthFrameReady(object sender, ImageFrameReadyEventArgs e) { var Image = e.ImageFrame.Image; var convertedDepthFrame = convertDepthFrame(Image.Bits); depth.Source = BitmapSource.Create( Image.Width, Image.Height, 96, 96, PixelFormats.Bgr32, null, convertedDepthFrame, Image.Width * 4); CalculateFps(); }

Kinect Getting Started Basic MainWindow.xaml.cs: convert depth frame  Converts a 16-bit grayscale depth frame which includes player indexes into a 32-bit frame that displays different players in different colors byte[] convertDepthFrame(byte[] depthFrame16) { for (int i16 = 0, i32 = 0; i16 < depthFrame16.Length && i32 < depthFrame32.Length; i16 += 2, i32 += 4) { int player = depthFrame16[i16] & 0x07; int realDepth = (depthFrame16[i16 + 1] > 3); // transform 13-bit depth information into an 8-bit intensity appropriate // for display (we disregard information in most significant bit) byte intensity = (byte)(255 - (255 * realDepth / 0x0fff)); depthFrame32[i32 + RED_IDX] = intensity; depthFrame32[i32 + BLUE_IDX] = intensity; depthFrame32[i32 + GREEN_IDX] = intensity; } return depthFrame32;}

Kinect Getting Started Basic MainWindow.xaml.cs: calculate frame rate void CalculateFps() { ++totalFrames; var cur = DateTime.Now; if (cur.Subtract(lastTime) > TimeSpan.FromSeconds(1)) { int frameDiff = totalFrames - lastFrames; lastFrames = totalFrames; lastTime = cur; frameRate.Text = frameDiff.ToString() + " fps"; }

Kinect Getting Started Basic MainWindow.xaml.cs: windows loading/closing private void WindowLoaded(object sender, RoutedEventArgs e) { InitializeNui(); } private void WindowClosed(object sender, EventArgs e) { kinectNui.Uninitialize(); } Time to run the app!