Kinect Development By: Richard Isely III

Outline What is the Kinect History ▫How it started ▫Microsoft Project The Components of the Kinect ▫What they are/How they work ▫Development Issues ▫Solution Hacking the Kinect ▫OpenKinect Microsoft SDK Demo

What is the Kinect? Image from:

History It all started with the Wii Moore’s Big Plans Project loses momentum

Project Natal Alex Kipman takes over project PrimeSense Chosen to Create Prototype Demo for Executives

Components of the Kinect Image From:

Components of the Kinect Major components ▫IR Projector ▫IR Camera ▫VGA Camera ▫Microphone Array ▫Motorized Tilt Image From:

VGA Camera Allows the Player to be shown on screen. Take still shots.

Microphone Array Four Microphones Position Detection Noise Cancelation

Microphone Array (Development Issue) Voice Recognition ▫Background Noise ▫Various Accents

Microphone Array (Solution) Background Noise ▫MSR Team develops Audio processing Solution  Audio Processing Tricks Various Accents ▫Acoustical Model built into the Array

Motorized Tilt Built into the base of the Kinect 54 Degree tilt range ▫27 Degrees up ▫27 Degrees down Records Actual angle of Kinect

IR Projector and Camera (Depth Sensor) What’s being projected Time of Flight How the Kinect does it Image From:

IR Projector and Camera (Depth Sensor) Continued ps1080 Chip What it produces ▫Depth Image Image From:

Using the Depth Image Pixel Data ▫16-bit number  Depth Data  Given in millimeters  Player Data  Range 0-6

Depth Image (Development Issue) Initializing Recognition Losing Player Mid-Game Body Type Issue

Depth Image (Solution) Breaking up the data ▫From 1 Blob to 31 body parts ▫Body parts to joints ▫Joints to Skeletal image Decision Tree

Depth Image (Solution)

Hacking the Kinect Race to create a driver ▫Bounty set ▫Cracked in 7 days OpenKinect ▫PrimeSense Releases Libraries

Microsoft SDK First Release (June 17 th, 2011) ▫Skeletal Recognition ▫Voice Recognition ▫Beta Version Latest Release and new Kinect ▫Near mode ▫Commercial Development ▫C++, C#, and VB

Demo

Conclusion What the future may hold ▫Possible Application ▫Cheap form of motion detection ▫Commercial Availability

Sources [1] Ashley, James and Jarrett Webb. Beginning Kinect Programming with the Microsoft Kinect SDK. Apress, [eBook]. [2] Hall, Jonathan, Sean Kean, and Phoenix Perry. Meet the Kinect: An Introduction to Programming Natural User Interfaces. Apress, [eBook] [3] Borenstein, Greg. Making Things See: 3D vision with Kinect, Processing, Arduino, and MakerBot. Make, [eBook] [4] “Kinect for Windows.” Microsoft Support. Sat. 10 Mar windows-info

Sources (continued) [5] “OpenNI.” PrimeSense. Sun. 11 Mar [6] “About PrimeSense.” PrimeSense. Sun. 11 Mar [7] “OpenKinect: About.” OpenKinect. Sat. 17 Mar [8] “Kinect for Windows Quickstart Series.” Channel9. By: Dan Fernandez. Sat. 17 Mar