KINECT Vinayak Thapliyal and Noah Balsmeyer 1

Overview  What is the Kinect?  Why was it made?  How does it work?  How does it compare to other sensors?  How can we use it? 2

What is the Kinect?  Input Device  Gesture Recognition  Facial Recognition  Voice Recognition  Programmable  C++  C#  Visual Basic.NET 3

What is the Kinect?  Gesture and Facial Recognition  RGB Camera  Depth Sensor  Infrared Laser Projector  Monochrome CMOS Sensor  Voice Recognition  Multi-Array Microphone Kinect2-deepmap.png /Kinect2-ir-image.png 4

Why was the Kinect made?  Make UI More Intuitive to Human Motion  Release a Competitor to Nintendo Wii  Led to More Affordable 3D Imaging and Motion Detection 5

How Does it Work?  Infrared Projection  Dot Pattern  Motion Detection  30 Hz Output from CMOS Sensor  40 Tracking Points  20 joints per player  Depth Mapping  Object Triangulation  Device Reaction 6 age_thumb_6F4828EC.png Il5DwjPVHn5NMQl*Ugoh8CpXN-W9b82XW/PrimeSensorDepth_Diagram.gif

Kinect Pin Layout 7  Modified USB Connector for More Power  Motorized Tilt Mechanism  RGB Camera, IR Projector, and CMOS Sensor  Internal Components

Depth Sensor  Field of View  57 Degree Horizontal, 43 Vertical  Movable by additional 27 Degrees Up or Down  IR Projector  Received by Monochrome CMOS Sensor  Video Output  9 – 30 Hz  640 x 480px  Additional Processing  IR Filter at Projector Wavelength 8

Depth Sensor  IR Projector emits predefined Dotted Pattern  Lateral Shift between projector and sensor  Shift in pattern dots  Shift in Dots determines Depth of Region 9

Kinect Accuracy  OpenKinect SDK  11 Bit Accuracy  2 11 = 2048 possible values  Measured Depth  Calculated 11 bit value  2047 = maximum distance  Approx ft.  0 = minimum distance  Approx ft.  Reasonable Range  4 – 10 feet  Provides Moderate Slope 10 Values from:

Kinect Accuracy  OpenKinect SDK  11 Bit Accuracy  2 11 = 2048 possible values  Measured Depth  Calculated 11 bit value  2047 = maximum distance  Approx ft.  0 = minimum distance  Approx ft.  Reasonable Range  4 – 10 feet  Provides Moderate Slope 11 Values from:

How Does the Kinect Compare?  Distance Sensing  Alternatives Cheaper than Kinect  ~$2 Single-Point Close-Range Proximity Sensor  Motion Sensing and 3D Mapping  High Performing Devices with Higher Cost 12 Taos%20USA%20Photos/TSL2771xFN _sml.jpg derDSDa_5070.jpg  Good Performance for Distance and Motion Sensing  Provides a bridge between low cost and high performance sensors

How Can We Use the Kinect?  Kinect Software Development Kit (SDK)  Free Download available for Windows 7 and 8  us/kinectforwindows/develop/overview.aspx us/kinectforwindows/develop/overview.aspx  Must have Windows OS Capable Processor  Very well documented and great online community support  Other SDKs, including OpenKinect, have been ported to Linux and Mac 13

How Can We Use the Kinect? 14

How Can We Use the Kinect?  3D Room Mapping 15

How Can We Use the Kinect?  Hand Signals and Motion Recognition 16

How Can We Use the Kinect?  Object Tracking object-tracking.jpg ct_tracking.jpg?w=470&h=313 17

How Can We Use the Kinect?  Wide Range of Applications  Possibly for your EECS 373 Project! 18

Questions? 19

References    stereo-triangulation/ stereo-triangulation/     filer/Tekniske_rapporter/Technical_Report_ECE-TR-6-samlet.pdf filer/Tekniske_rapporter/Technical_Report_ECE-TR-6-samlet.pdf  US/kinectsdknuiapi/thread/4da8c75e-9aad-4dc3-bd83-d77ab4cd2f82 US/kinectsdknuiapi/thread/4da8c75e-9aad-4dc3-bd83-d77ab4cd2f82 20