Gesture Control interface

Slides:



Advertisements
Similar presentations
Ozupad Nov A single device that with many features: Air-space mouse Optical mouse Air-space Joystick JOG Scrolling Pointing.
Advertisements

Electronic and Computer Engineering Colin Grogan Final Year Project: Design and Build an Air Mouse for people with lower mobility.
TEMPLATE DESIGN © Bluetooth Login Key RIT Computer Engineering Senior Design Project Tim Wesley, Dan Chen Spring 2009.
Students: Asulin Ofir Heller Itai Supervisor: Mony Orbach In association with: June 16, summer 2006.
TOUCH SCREEN AND ZIGBEE BASED WIRELESS COMMUNICATION ASSISTANT
Progress Presentation Final Year Project Air-Mouse for Windows/Linux PC Colin Grogan
Final Year Project: Design and Build an alternative input device Air Mouse Colin Grogan.
GENERAL PRESENTATION SUBMITTED BY:- Neeraj Dhiman.
Home Theater Remote Date: May 16, 2008 Team: Bryan Follis, Mike Schmidt, Dan Grissom, Jesse Butler Advisor: Dr. David Klotzkin Final Presentation.
MODULE 1 Computing Essentials © Paradigm Publishing, Inc.1.
 Input Devices Input Devices  Examples of Input Devices Examples of Input Devices  Keyboard Keyboard  Pointing Devices Pointing Devices Mouse Joystick.
Win OS & Hardware. Input Getting data into the computer.
Interacting with your Computer Chapter 2 Learning Objectives Identify five key groups on standard computer keyboard Name six special purpose keys.
TOUCH SCREEN TECHNOLOGY
ThinkOptics Confidential Providing the Next Generation of Remote Controls.
By Noordiana Kasim. MODERN I/O DEVICES 1. PRINTER 2. MONITOR 3. KEYBOARD 4. AUDIO SPEAKER 5. DVD DRIVE.
October Test Beam DAQ. Framework sketch Only DAQs subprograms works during spills Each subprogram produces an output each spill Each dependant subprogram.
HARDWARE DEVICES TOBY WILSON. HARDWARE DEVICES InputOutputStorage KeyboardMonitorsExternal Hard Disk MicePrintersFlash Drives Graphics TabletSpeakers.
Mid Semester Presentation Glove Mouse. Jacob BassettChelsea MosserRobert BlackBryan Lee Electrical Engineer HID Protocol Enclosure Electrical Engineer.
Introduction to Computing Slides By ADEELA MUSTAFA.
Keyboard mode: First Touch Second Touch if2is2mf2ms2rf2rs2ps2 if1abcdfg[space] is1hejklm[backspace] mf1npiqrs[return] ms1tvwoxz[period] rf10123u4[sym]
Voice Controlled Home Automation System Group 13 Zhe Gong Hongchuan Li.
Magic Wand Battle Game Team 53 Shanoon Martin, Jialin Sun, Manfei Wu.
TOUCHLESS TOUCHSCREEN USER INTERFACE
THE RECENT ADVANCEMENT AND APPLICATIONS IN TOUCH SCREEN TECHNOLOGY.
Pasewark & Pasewark 1 Windows Vista Lesson 1 Windows Vista Basics Microsoft Office 2007: Introductory.
Lecture Input Devices Keyboard. Mouse Microphone Digital Camera Scanner.
Office 2016 and Windows 10: Essential Concepts and Skills
Voice Controlled Robot by Cell Phone with Android App
TOUCHLESS TOUCH SCREEN USER INTERFACE
Grado en Ingeniería de Tecnologías y Servicios de Telecomunicación
Introduction to Computers Mrs. Gambucci
When CSI Meets Public WiFi: Inferring Your Mobile Phone Password via WiFi Signals Adekemi Adedokun May 2, 2017.
Lesson 4 Alternative Methods Of Input.
Alternative Methods Of Input
Development Environment
Standard Methods of Input.
Aim Of The Project To implement the obstacle avoidance and Zigbee control functions for Omni directional mobile robot.
AT Commands Supports AT commands
Operating System Review
Clocks, I/O devices, Thin Clients, and Power Management
Methods of Computer Input and Output
Depth Analysis With Stereo Cameras
Chapter 2: Operating-System Structures
Pipeline Leak Detection Device
GESTURE RECOGNITION TECHNOLOGY
Accelerometer-Based Character Recognition Pen
High Performance Low Cost Low Lost Wireless DC Motor Speed Control
Programming Concepts (Part B) ENGR 10 Introduction to Engineering
TOUCHLESS TOUCHSCREEN USER INTERFACE
Wearable Keyboard Graduation Project Seminar 21/12/ Thursday
Zigbee Networks Using Xbee Modules
Lesson 4 Alternative Methods Of Input.
Introduction to Computers
Office 2010 and Windows 7: Essential Concepts and Skills
Operating System Review
Group 16 Tarek Kaddoura Jigar Nahar
Serial Data Hub (Proj Dec13-13).
Input devices EIT, ©Author Gay Robertson, 2016.
ECE477 Group 7 Success Criteria
Chapter 2: The Linux System Part 5
There are different types of translator.
Lesson 4 Alternative Methods Of Input.
docket Assignment 6: Medium-fi Prototyping
Accelerometer-Based Character Recognition Pen
Computing Essentials Module 1.
Computing Essentials Module 1.
Computer components is a programmable machine that receives input, stores and manipulates data, and provides output in a useful format. Computer The computer.
Interactive Mirror Display
♪ Embedded System Design: Synthesizing Music Using Programmable Logic
Presentation transcript:

Gesture Control interface Group 2 project

Who are we? Rajan Jassal Eric Smith Chris Chmilar Current lead in Linux side USB communication Future lead in microcontroller programming Eric Smith Current lead in DE2 side USB communication Future lead in IMU (Inertial Measurement Unit) operations Chris Chmilar Current lead in wireless communication Future lead of gesture recognition

The project A glove for executing navigational operations in a Linux environment by detecting a user’s movements Gestures can be used to scroll vertically in documents, and switch between windows or tabs. Touching any finger to the thumb will allow additional macros to be used or change the working mode of the glove.

Motivation Convenience Provide a way to do the little convenient things with your computer intuitively and from a distance: say you’re chopping onions and your computer is on the other side of the counter. Using this device, with just a wave of your hand, you can scroll down to the next step in your recipe. Innovation To explore different ways to wirelessly interact with a computer. Gesture control has been in the mind of innovators for some time, however a useful product has yet to be realized.

Top level design (module diagram) Glove (IMU) DE2 (Gesture Recognition) Computer (Custom Driver)

Glove components 9-degree of freedom inertial measurement unit (IMU) for tracking motions Metal contacts for detecting interaction between fingers Wireless transmitter for sending data to microcontroller Microprocessor for managing other components

De2 and FPGA components Xbee wireless receiver or receiving data from glove USB controller (ISP1362) for communication between board and computer

Computer and driver Custom driver for Linux to interpret data from the DE2 board Performing actions within the OS based on data received

Block diagram

Dataflow diagram

Gestures Data gathered is interpreted by a program run on the DE2 and compared against hard coded gesture templates. Each template will represent a fairly broad set of movement inputs so as to not be too difficult to use. These gestures correspond to signals to be sent over the USB connection that the custom driver will recognise as mouse or keyboard actions. We may add some learning algorithms to narrow the template boundaries to improve user specific performance

Challenges (so far) Numerous components makeup the project so finding parts that interface correctly took a lot of time and research Drivers are quite complicated for the functionality we are after and so are taking a lot of research

Design calculations Maximum actions taken per second: Based on a trial done on a touchscreen for number of actions when scrolling rapidly on a screen. 65 actions / 30 seconds = 2.166 Nyquist rate = 2*2.166 = 4.33 actions / second. ZigBee serial data rates set to 38400 baud = 4800 bytes/s. Maximum packet size possible is 1.07 KB

Code Example (Pseudocode) gesture gestureRecognition (packet) data = unpackData(packet); gestureSequence = convertToGestureSequence(data); gesture = matchTemplate(templateList, gestureSequence); sendToComputer(gesture, USB_CORE_ADDR); return gesture

Test plan All components will be tested separately and after being integrated Wireless Communication: Use indicators to see if transmission is received (ex. LEDs) IMU: Attach to a board directly to observe the types of data received USB Communication: Pre-set hardcoded commands to be interpreted by the driver Integrated Testing

Optional features Features to add: Features to remove: Other modes of operation, such as a media mode to play/pause and skip multimedia A mouse-like mode to add cursor functionality Features to remove: The metal contact (“touch”) operation

Questions?