1. Kinect Part II
Anna Loparev

2. OpenNI API Actually general purpose API
Need 3 things (
SensorKinect Driver
OpenNI Library
NITE Middleware (gesture support)
Requirements:
Kinect
Visual Studio 2008/2010
C++/C#
Requirements (From Tutorial):
Python
SWIG (Allow Python to interface with OpenNI)
OpenCV (Allow Python to work with image data)
NumPy

3. Features Recording for debugging Fully backwards compatible
Under the GNU Lesser General Public License (LGPL)

4. OpenNI vs Microsoft SDK
Easy to install
Don’t have to do calibration pose but gets more false positives
Predictive tracking of joints (Good for when sensor losses player)
Leads to false positives
Uses lots of processing power
Gets full 1024x768 resolution from camera
Full access to microphone
Support for tilt
OpenNI
Open source (Allows for commercial use)
Works across all platforms
Hand-only tracking
Gesture recognition
Rotation in joints
Tilt has to be done through motor drivers

5. Sample Applications (OpenNI)
General
NiCRead (Prints out depth at center of image)
NiSimpleCreate (C++ version of NiCRead without using SamplesConfig XML)
NiSimpleRead (C++ version of NiCRead using SamplesConfig XML)
SimpleRead (C# version of NiCRead)
NiSimpleViewer (Depth and video viewer)
SimpleViewer (C# version of NiSimpleViewer)
NiViewer (Shows a bunch of different views and allows recording)
NiUserTracker (Pose detection and skeletal tracker)
UserTracker (C# version of NiUserTracker)
NiUseExtension
NiAudioSample
Recording
NiBackRecorder (Allows user to save certain number of seconds)
NiRecordRaw
NiRecordSynthetic (Opens recording, does transformation, and re-records)
NiConvertXToONI (Copies recording)

6. Sample Applications (NITE)
General
Sample-Boxes (Highlight boxes based on hand movement and gestures)
Sample-Boxes.net (C# version of Sample-Boxes)
Sample-CircleControl (Perform a circle gesture and circle is drawn)
Sample-Players (Identifies multiple players and their skeletons)
Sample-PointServer
Sample-PointViewer (Tracks hand and leaves behind trail)
Sample-SceneAnalysis (Identifies people)
Sample-SingleControl (Identifies wave gesture)
Sample-StickFigure (Places skeleton and people in virtual room)
Sample-TrackPad (Clickable grid)

7. Overview

8. Overview

9. NITE Abilities Focus gestures Gesture Status Hand tracking
Click
Wave
Swipe left
Swipe right
Raise hand candidate
Hand candidate moved
Gesture Status
Gesture started
Gesture completed
Hand tracking
User segmentation (Identify and track people)
Skeletal tracking

10. Capabilities
Optional extensions (Providers choose if they want to implement them.)
Alternative View (Data appears to come from another sensor)
Cropping
Frame Sync (between multiple devices)
Mirror
Pose Detection
Skeleton
User Position (Optimize output depth map)
Error State
Lock Aware (Allows node to be locked outside of context)

11. Generators Map Generator Depth Generator Image Generator IR Generator
Scene Analyzer
Audio Generator
Gesture Generator
Hand Point Generator
User Generator

12. Generators (Cont)
xn::Generator::StartGenerating() - begin generating data
xn::Generator:StopGenerating() - stop generation without destroying node
xn::Generator::WaitAndUpdateData() - Update data when it becomes available
xn::Context::Wait[…]UpdateAll() - Update all generators ([…] determines when)

13. Simple Program // Set status to OK XnStatus nRetVal = XN_STATUS_OK;
// Create and initialize new context
xn::Context context;
nRetVal = context.Init();
// Create generators
xn::DepthGenerator depth;
nRetVal = depth.Create(context)
// Start generating data
nRetVal = context.StartGeneratingAll();

14. Simple Program (Cont) // Do main loop while (bShouldRun) {
// Wait for new data
nRetVal = context.WaitOneUpdateAll(depth);
// Make sure update worked
if (nRetVal != XN_STATUS_OK)
printf(“Failed updating data: %s\n”, xnGetStatusString(nRetVal));
continue; }
// DO STUFF

15. Simple Program (Cont) // Clean-up context.Shutdown(); OR
depth.Release();
context.Release();

16. DEMOS

17. References/More Info Setup:
Extensions:
Other: