Hand Gesture Based User Interface For Generic Windows Applications –Computer Interface A 2D/3D input device based on real time hand trackingA 2D/3D input device based on real time hand tracking Translation of gestures to Windows commandsTranslation of gestures to Windows commands More natural interface for 3D design applicationsMore natural interface for 3D design applications –Aid for disabled and deaf people Sign language analysis, translationSign language analysis, translation Sign language teachingSign language teaching Using multimodality to enable communication between disabled peopleUsing multimodality to enable communication between disabled people –Entertainment Applications involving hand gesture recognition, from computer games to sound and image design applicationsApplications involving hand gesture recognition, from computer games to sound and image design applications LeftImage 3D Reconstruction 3D Kalman Filter Vector Quantizer Gesture Spotter Application RightImage 2D Kalman Filter Marker Segmentation Capabilities Camera calibration and gesture training toolsCamera calibration and gesture training tools Registration of new markers through the interfaceRegistration of new markers through the interface Recognition of 2D and 3D gestures in real time, modeled with HMMsRecognition of 2D and 3D gestures in real time, modeled with HMMs Real time 2D and 3D Kalman filtering of the trajectoriesReal time 2D and 3D Kalman filtering of the trajectories Support for two hands wearing different colored glovesSupport for two hands wearing different colored gloves Real time mouse simulation based on hand trackingReal time mouse simulation based on hand tracking Capability of triggering windows input device events – controlling third party applicationsCapability of triggering windows input device events – controlling third party applications Generic interface that can be linked to any application through configurationGeneric interface that can be linked to any application through configuration

Requirements Hand Gesture Interface is specifically designed to not require any special hardware Hardware Requirements Smooth colored hand glove as the markerSmooth colored hand glove as the marker Simple WDM or VFW based camera(s)Simple WDM or VFW based camera(s) Good lighting conditionsGood lighting conditions Mainboard should be able to handle two cameras for the 3D modeMainboard should be able to handle two cameras for the 3D mode Software Requirements Windows Operating System (Tested on XP and Me)Windows Operating System (Tested on XP and Me) Execution The user registers the marker by waving the hand to the camera(s)The user registers the marker by waving the hand to the camera(s) The user trains the system any 2D or 3D gesture (2D uses only x-y)The user trains the system any 2D or 3D gesture (2D uses only x-y) The user modifies or replaces the configuration file to link it to any application and mouse or keyboard eventsThe user modifies or replaces the configuration file to link it to any application and mouse or keyboard events The user starts controlling the interfaceThe user starts controlling the interfaceRemarks Gestures are only recognized when the hand is held verticallyGestures are only recognized when the hand is held vertically Keeping the hand horizontal simulates the left click of the mouseKeeping the hand horizontal simulates the left click of the mouse The ends of a gesture are automatically found by the system.The ends of a gesture are automatically found by the system. The system comes with 8 pre-trained gestures, which can be changed anytimeThe system comes with 8 pre-trained gestures, which can be changed anytime The system comes with a pair of calibration matrices, which should work with most camera setups. If they don’t, the calibration tool needs to be usedThe system comes with a pair of calibration matrices, which should work with most camera setups. If they don’t, the calibration tool needs to be used