1. 9/9/2015 EEC492/693/793 - iPhone Application Development 1 EEC-492/592 Kinect Application Development Lecture 1 Wenbing Zhao wenbing@ieee.org

2. 9/9/2015 EEC492/693/793 - iPhone Application Development 2 Introduction Instructors  Wenbing Zhao, wenbing@ieee.orgwenbing@ieee.org Office Hours  Mondays 6-8pm  Wednesdays noon-2pm Course Web site  http://academic.csuohio.edu/zhao_w/teaching/EEC492- S14/eec492.htm http://academic.csuohio.edu/zhao_w/teaching/EEC492- S14/eec492.htm

3. 9/9/2015 EEC492/693/793 - iPhone Application Development 3 Quick Survey Have you programmed in C#? Have you programmed in C++? Have you programmed in Java? Have you ever played Kinect games? Have you installed Visual Studio and done Windows application development?

4. 9/9/2015 EEC492/693/793 - iPhone Application Development 4 Requirements Prerequisite:  Moderate programming background  Basic knowledge of data structure and algorithms  Basic knowledge of operating systems

5. 9/9/2015 EEC492/693/793 - iPhone Application Development 5 Textbooks and References Main textbook  Kinect for Windows SDK Programming Guide, by Abhijit Jana, PACKT publishing, 2012. References:  Kinect Hacks, by Jared St. Jean, O’Reilly, 2013  Hacking the Kinect, by Jeff Kramer et al., Apress, 2012.Reference materials  C# in a Nutshell, By Peter Drayton, Ben Albahari, Ted Neward, O'Reilly, 1 st Edition March 2002

6. 9/9/2015 EEC492/693/793 - iPhone Application Development 6 Required Hardware & Software Hardware  Windows PC or laptop  Kinect sensor Software  Visual Studio 2010  Kinect SDK  Other related libraries

7. 9/9/2015 EEC492/693/793 - iPhone Application Development 7 Kinect’s Capability Capturing real-time video using the color sensor Tracking a human body and then responding to its movements and gestures as a natural user interface Generating depth map of the objects tracked Recognizing a human voice and developing hands- free applications that can be controlled by voice

8. 9/9/2015 EEC492/693/793 - iPhone Application Development 8 Real-World Applications of Kinect Healthcare: exercise measurement, monitoring patients, their body movements, and so on Robotics: as a navigation system for robots either by tracking human gestures, voice commands, or by human body movements Education: help them to learn subjects either by their gesture and voice commands Security system: track human body movement or face and send the notifications Virtual Reality Trainer: measuring the movements of human body joints, providing live feedback to users if the joints are moving in an appropriate manner by comparing the movements with previously stored data Military: build intelligent drones to spy on enemy lines

9. 9/9/2015 EEC492/693/793 - iPhone Application Development 9 Healthcare with Kinect Reflexion Health  Advancing physical therapy  http://blogs.msdn.com/b/kinectforwindows/archive/2013/05/ 09/reflexion-health-advancing-physical-therapy-with-kinect- for-windows.aspx http://blogs.msdn.com/b/kinectforwindows/archive/2013/05/ 09/reflexion-health-advancing-physical-therapy-with-kinect- for-windows.aspx

10. 9/9/2015 EEC492/693/793 - iPhone Application Development 10 Healthcare with Kinect ReMotion360: uses Kinect to measure and provide on screen feedback for range of motion exercises either in a clinic or at home  http://www.infostrat.com/solutions/Remotion360 http://www.infostrat.com/solutions/Remotion360  https://www.youtube.com/watch?feature=player_embedded &v=-jhpYslVygs https://www.youtube.com/watch?feature=player_embedded &v=-jhpYslVygs

11. 9/9/2015 EEC492/693/793 - iPhone Application Development 11 Improving Safety at Workplace Siemens uses Kinect for Windows to improve nuclear plant employee safety  virtual 3-D human avatars integrated to model motion- controlled actions input with a Kinect  http://blogs.msdn.com/b/kinectforwindows/archive/2013/04/ 19/siemens-uses-kinect-for-windows-to-improve-nuclear- plant-employee-safety.aspx http://blogs.msdn.com/b/kinectforwindows/archive/2013/04/ 19/siemens-uses-kinect-for-windows-to-improve-nuclear- plant-employee-safety.aspx

12. 9/9/2015 EEC492/693/793 - iPhone Application Development 12 Enabling Retailers to Drive Business Benefits:  Capture a customer's attention  Educate customers about products  Move a customer through the decision-making cycle to close a sale Ski Shop scenario:  https://www.youtube.com/watch?feature=player_embedded&v=O RWYr1gx5eU https://www.youtube.com/watch?feature=player_embedded&v=O RWYr1gx5eU Toy store scenario:  https://www.youtube.com/watch?feature=player_embedded&v=w ZfcDOsBs2E https://www.youtube.com/watch?feature=player_embedded&v=w ZfcDOsBs2E Retail clothing scenario  https://www.youtube.com/watch?feature=player_embedded&v=M r71jrkzWq8 https://www.youtube.com/watch?feature=player_embedded&v=M r71jrkzWq8

13. 9/9/2015 EEC492/693/793 - iPhone Application Development 13 Styku: Apparel Shopping Help find clothes that fit Virtually try on clothes before they buy To be offered by major retailers, such as Brooks Brothers, IM-Label, and others

14. 9/9/2015 EEC492/693/793 - iPhone Application Development 14 Shoppers who approach the Kinect-based showroom are guided through an array of natural movements to activate hotspots on the Pathfinder model, allowing them to inspect the car inside and out  https://www.youtube.com/watch?feature=player_embedded &v=rQVycUmeqB4 Nissan Pathfinder Virtual Showroom

15. 9/9/2015 EEC492/693/793 - iPhone Application Development 15 Education Tool nsquared sky spelling  makes it fun for children to learn to spell in an interactive and collaborative environment  capture letters that complete the spelling of various words

16. 9/9/2015 EEC492/693/793 - iPhone Application Development 16 Hospitality Tool Nsquared Hotel Kiosk  augments the concierge function in a hotel by providing guidance to hotel guests through an intuitive, interactive experience  use a “grip” gesture to drag content across the screen and a “push” gesture to select content

17. 9/9/2015 EEC492/693/793 - iPhone Application Development 17 Ubi Interactive: Turn any Surface into a Touch Screen Instruct classes in an interactive lecture hall Access product information on a store’s window front Recipes can be projected onto kitchen countertops without having to worry about getting anything dirty Use wall to play interactive games like Angry Birds Video: https://www.youtube.com/watch?feature=player_embedded&v =Iu2XH5p_hMM https://www.youtube.com/watch?feature=player_embedded&v =Iu2XH5p_hMM

18. 3D Scanning Kinect fusion: perform 3D scanning with a single Kinect sensor!  Nice match with 3D printers 9/9/2015 EEC492/693/793 - iPhone Application Development 18

19. 9/9/2015 EEC492/693/793 - iPhone Application Development 19 New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch A combined 2-D and 3-D camera from Intel will be built into laptops and tablets from a range of manufacturers, Intel announced at the 2014 International Consumer Electronics Show (CES)

20. 9/9/2015 EEC492/693/793 - iPhone Application Development 20 New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch Toshiba also demonstrated a dual-camera module for smartphones, tablets, and laptops at 2014 CES  Gesture recognition, refocus images, etc.

21. New Development on Depth Cameras SoftKinetic demonstrate a depth camera for 3D scanning  Need to move the camera around the subject Apple will also come to the depth camera field  Acquired Primesense, the company invented the technology behind Kinect  Filed a patent in 3D camera for iOS devices 9/9/2015 EEC492/693/793 - iPhone Application Development 21

22. 9/9/2015 EEC492/693/793 - iPhone Application Development 22 Main Topics Understand the Kinect sensor and how they work Learn how to capture data streams from the Kinect sensor Learn the uses of color, depth and IR data and how to leverage them in your applications Use skeleton-tracking for interactive applications Learn basic gesture recognition methods and develop gesture-enabled applications Build speech-enabled applications with Speech API, and text to speech as well Learn 3D game development with Unity3D for Kinect (virtual reality!) Applied computer vision with OpenCV and Kinect

23. 9/9/2015 EEC492/693/793 - iPhone Application Development 23 Assignments and Grading Assignments  In-class exercises  Term project Reading several research papers on the application of Kinect Develop a working prototype program using Kinect Project presentations and demonstration Grading policy  Class participation: 60% (any missed session will lead to credit reduction)  Project: 40%  No exams

24. 9/9/2015 EEC492/693/793 - iPhone Application Development 24 Demos, Software Installation and Q/A