Download presentation
Presentation is loading. Please wait.
1
Cindy Song Sharena Paripatyadar
2
Use vision for HCI Determine steps necessary to incorporate vision in HCI applications Examine concerns & implications of such applications
3
Background and Trends In Today’s world: Many devices with integrated cameras Many personal webcams Our Goal: To understand how to take advantage of these one camera systems
4
Literature Research Idea Generation Media Player Wizard of Oz Media Player Implementation Re-evaluate Approach Literature Research Revised Implementation Learning Application User Study Evaluation
5
Freeman and Roth For hand posture analysis Creates histograms of local orientation using feature vectors from pixel intensity Recognizes 10 gestures in real time
6
Triesch and von der Malsburg Based on Elastic Graph Matching Extended for skin color feature detection Recognizes 12 gestures
7
Freeman Uses one open hand to control onscreen display Real time application Hand may not be prominent in image
8
Original Idea: Video control for WMPlayer Wizard of Oz for a media player Implementation of gesture recognition Shortcoming: real time issues New approach: Teaching Chinese hand gestures
9
5 participants, various technical backgrounds, age 20-27 Using computer with remote control Used alternate monitor to show user video captured
10
Small set of user-intuitive gestures are easy to remember, but need some menu reminder Show rationale behind gestures Visual feedback to show recognized command before execution Concerns with: Low-light condition Camera field of view Webcam configuration Responsiveness Accuracy
11
Pros Don’t have to search for remote Don’t have to touch remote while eating No battery to run down Cons Doesn’t have as many features as remote Doesn’t work in dark environments More ambiguous than remote, more errors possible – know what each button will do
13
Gesture Determination: Using surrounding frames Pattern Matching: Compare regions with gesture patterns Finger Region Detection: Find connected regions (fingers/palm) Hand Location: Create bounding box from skin color Calibration: Learn lighting and coloring Skin Color Training: With images of various lighting
14
Train based on skin color Locate hand in image Analyze hand gesture Finger region detection Pattern matching Not real-time
15
Skin Color Training Trained on 20+ images Different lighting & people Uses “Lab” color space
16
Calibration Short training based on person’s hand and lighting conditions - < 1 sec needed Determines correct lighting & with skin color data Learns specific hand features
17
Hand Location Determines hand position in image using skin color Fill in missing portions of hand Create bounding box
18
Finger Region Detection Examine bounding box Find connected regions Remove small regions
19
Pattern Recognition Created set patterns based on 10 gestures Counts number of finger regions for gestures 1-5 For gestures 6-10, based on number regions detected, looks at other patterns i.e. for 6 determine ratio of finger width to space between fingers
20
Gesture Determination 20 frames needed to recognize the gesture Avoids recognizing accidental gestures
23
Complex Backgrounds First skin color analysis Then find large connected regions of fingers and hand
24
Motion Static gestures & frame by frame analysis Allow for moving camera Gesture determination corrects obscurities or out of frame hand positioning
25
5 participants, various technical backgrounds, age 20-27 Taught users 2-4 gestures Quizzed users on gestures learned Ran gesture recognition algorithm to provide feedback Asked several follow up questions
26
Useful for learning sign language, teaching kids to count Instant feedback necessary Nice to know how to correct gesture Needs high accuracy Other applications Some said Media Player application more useful Or use as security system (hand gestures as a password)
27
Of implementation Real time is difficult Pattern recognition for specific gestures vs. technique for all types of gestures Complex/moving backgrounds important for real world applications Of user studies Video is valuable avenue for many applications Accuracy and responsiveness are important In one camera systems, there is a tradeoff between convenience and clarity
28
Real-time More user studies Mobile devices Gesture learning application i.e. Chinese cultural gestures Media Player plug-in application
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.