1. EEC-693/793 Applied Computer Vision with Depth Cameras
Lecture 1
Wenbing Zhao
5/19/2018
EEC492/693/793 - iPhone Application Development

2. EEC492/693/793 - iPhone Application Development
Introduction
Instructors
Wenbing Zhao,
Office Hours
MW 2-3:50pm
Course Web site
5/19/2018
EEC492/693/793 - iPhone Application Development

3. EEC492/693/793 - iPhone Application Development
Quick Survey
Have you programmed in C#?
Have you programmed in C++?
Have you programmed in Java?
Have you ever played Kinect games?
Have you installed Visual Studio and done Windows application development?
5/19/2018
EEC492/693/793 - iPhone Application Development

4. EEC492/693/793 - iPhone Application Development
Requirements
Prerequisite:
Moderate programming background
Basic knowledge of data structure and algorithms
Basic knowledge of operating systems
5/19/2018
EEC492/693/793 - iPhone Application Development

5. Textbooks and References
Main textbook
Kinect for Windows SDK Programming Guide, by Abhijit Jana, PACKT publishing, 2012.
References:
Kinect Hacks, by Jared St. Jean, O’Reilly, 2013
Hacking the Kinect, by Jeff Kramer et al., Apress, 2012.Reference materials
C# in a Nutshell, By Peter Drayton, Ben Albahari, Ted Neward, O'Reilly, 1st Edition March 2002
5/19/2018
EEC492/693/793 - iPhone Application Development

6. Required Hardware & Software
Windows PC or laptop
Kinect sensor
Software
Visual Studio 2015
Kinect SDK v1.8
Unity 5
OpenCV
Other related libraries
5/19/2018
EEC492/693/793 - iPhone Application Development

7. EEC492/693/793 - iPhone Application Development
Kinect’s Capability
Capturing real-time video using the color sensor
Tracking a human body and then responding to its movements and gestures as a natural user interface
Generating depth map of the objects tracked
Recognizing a human voice and developing hands-free applications that can be controlled by voice
5/19/2018
EEC492/693/793 - iPhone Application Development

8. Real-World Applications of Kinect
Healthcare: exercise measurement, monitoring patients, their body movements, and so on
Robotics: as a navigation system for robots either by tracking human gestures, voice commands, or by human body movements
Education: help them to learn subjects either by their gesture and voice commands
Security system: track human body movement or face and send the notifications
Virtual Reality
Trainer: measuring the movements of human body joints, providing live feedback to users if the joints are moving in an appropriate manner by comparing the movements with previously stored data
Military: build intelligent drones to spy on enemy lines
5/19/2018
EEC492/693/793 - iPhone Application Development

9. Healthcare with Kinect
Reflexion Health
Advancing physical therapy
5/19/2018
EEC492/693/793 - iPhone Application Development

10. Healthcare with Kinect
ReMotion360: uses Kinect to measure and provide on screen feedback for range of motion exercises either in a clinic or at home
5/19/2018
EEC492/693/793 - iPhone Application Development

11. Improving Safety at Workplace
Siemens uses Kinect for Windows to improve nuclear plant employee safety
virtual 3-D human avatars  integrated to model motion-controlled actions input with a Kinect
5/19/2018
EEC492/693/793 - iPhone Application Development

12. Enabling Retailers to Drive Business
Benefits:
Capture a customer's attention
Educate customers about products
Move a customer through the decision-making cycle to close a sale
Ski Shop scenario:
Toy store scenario:
Retail clothing scenario
5/19/2018
EEC492/693/793 - iPhone Application Development

13. Styku: Apparel Shopping
Help find clothes that fit
Virtually try on clothes before they buy
To be offered by major retailers, such as Brooks Brothers, IM-Label, and others
5/19/2018
EEC492/693/793 - iPhone Application Development

14. Nissan Pathfinder Virtual Showroom
Shoppers who approach the Kinect-based showroom are guided through an array of natural movements to activate hotspots on the Pathfinder model, allowing them to inspect the car inside and out
5/19/2018
EEC492/693/793 - iPhone Application Development

15. EEC492/693/793 - iPhone Application Development
Education Tool
nsquared sky spelling
makes it fun for children to learn to spell in an interactive and collaborative environment
capture letters that complete the spelling of various words 
5/19/2018
EEC492/693/793 - iPhone Application Development

16. EEC492/693/793 - iPhone Application Development
Hospitality Tool
Nsquared Hotel Kiosk
augments the concierge function in a hotel by providing guidance to hotel guests through an intuitive, interactive experience
 use a “grip” gesture to drag content across the screen and a “push” gesture to select content
5/19/2018
EEC492/693/793 - iPhone Application Development

17. Ubi Interactive: Turn any Surface into a Touch Screen
Instruct classes in an interactive lecture hall
Access product information on a store’s window front
Recipes can be projected onto kitchen countertops without having to worry about getting anything dirty
Use wall to play interactive games like Angry Birds
Video:
5/19/2018
EEC492/693/793 - iPhone Application Development

18. EEC492/693/793 - iPhone Application Development
3D Scanning
Kinect fusion: perform 3D scanning with a single Kinect sensor!
Nice match with 3D printers
5/19/2018
EEC492/693/793 - iPhone Application Development

19. EEC492/693/793 - iPhone Application Development
New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch
A combined 2-D and 3-D camera from Intel will be built into laptops and tablets from a range of manufacturers, Intel announced at the 2014 International Consumer Electronics Show (CES)
5/19/2018
EEC492/693/793 - iPhone Application Development

20. EEC492/693/793 - iPhone Application Development
New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch
Toshiba also demonstrated a dual-camera module for smartphones, tablets, and laptops at 2014 CES
Gesture recognition, refocus images, etc.
5/19/2018
EEC492/693/793 - iPhone Application Development

21. New Development on Depth Cameras
SoftKinetic demonstrate a depth camera for 3D scanning
Need to move the camera around the subject
Apple will also come to the depth camera field
Acquired Primesense, the company invented the technology behind Kinect
Filed a patent in 3D camera for iOS devices
5/19/2018
EEC492/693/793 - iPhone Application Development

22. EEC492/693/793 - iPhone Application Development
Main Topics
Understand the Kinect sensor and how they work
Learn how to capture data streams from the Kinect sensor
Learn the uses of color, depth and IR data and how to leverage them in your applications
Use skeleton-tracking for interactive applications
Learn basic gesture recognition methods and develop gesture-enabled applications
Learn 3D game development with Unity3D for Kinect (virtual reality!)
Applied computer vision with OpenCV and Kinect
5/19/2018
EEC492/693/793 - iPhone Application Development

23. Assignments and Grading
In-class exercises
Term project
Reading several research papers on the application of Kinect
Develop a working prototype program using Kinect
Project presentations and demonstration
Grading policy
Class participation: 60% (any missed session will lead to credit reduction)
Project: 40%
No exams
5/19/2018
EEC492/693/793 - iPhone Application Development

24. Demos, Software Installation and Q/A
5/19/2018
EEC492/693/793 - iPhone Application Development