CONTENT FOCUS FOCUS INTRODUCTION INTRODUCTION COMPONENTS COMPONENTS TYPES OF GESTURES TYPES OF GESTURES ADVANTAGES ADVANTAGES CHALLENGES CHALLENGES REFERENCE REFERENCE

FOCUS Integration of information Integration of information Providing “just-in-time” information Providing “just-in-time” information

What is Sixth Sense? Wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information Wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information

INTRODUCTION Augmented Reality. Augmented Reality. Wearable GesturalInterface. Wearable GesturalInterface. Converts intangible,digital information out into tangible world. Converts intangible,digital information out into tangible world.

Interaction with the projected information. Interaction with the projected information. No need of special glasses. No need of special glasses.

The Device……..

Overview of Sixth Sense Device Integration of information Integration of information Gestural interface Gestural interface Augmentation of information Augmentation of information Wearable interface Wearable interface

Technologies implemented Augmented Reality Augmented Reality Object Augmentation Object Augmentation Computer vision based techniques Computer vision based techniques

Augmented reality (AR) Augmented reality (AR) Term for a live direct or indirect view of a physical real-world environment whose elements are merged with virtual, computer generated images, creating a mixed reality Term for a live direct or indirect view of a physical real-world environment whose elements are merged with virtual, computer generated images, creating a mixed reality

Object augmentation Object augmentation the camera recognizes and tracks physical objects such as a newspaper, a coffee cup, or a book and the projector projects relevant visual information on them. the camera recognizes and tracks physical objects such as a newspaper, a coffee cup, or a book and the projector projects relevant visual information on them. Computer-vision techniques Computer-vision techniques track objects and align projected visual information by matching pre-printed static color markers (or patterns) on these objects with the markers projected by the projector. track objects and align projected visual information by matching pre-printed static color markers (or patterns) on these objects with the markers projected by the projector.

Components Camera Camera Projector Projector Mobile Phone Mobile Phone Coloured markers Coloured markers

How does the device work?

Camera-logitech quick cam Camera-logitech quick cam recognizes and tracks user’s handgestures and physical objects using computer vision based techniques recognizes and tracks user’s handgestures and physical objects using computer vision based techniques Projector-3MM Pro110 Projector-3MM Pro110 projects visual information enabling surfaces walls and physical objects around us to be used as interfaces projects visual information enabling surfaces walls and physical objects around us to be used as interfaces

Mobile phone projector and camera are connected to the mobile computing device in the user’s pocket projector and camera are connected to the mobile computing device in the user’s pocket Coloured markers visual tracking fiducials visual tracking fiducials

Software Requirements Software Requirements Microsoft Windows platform using C#,WPF and openCV Microsoft Windows platform using C#,WPF and openCV

Internet Connection Benefits of having a network Benefits of having a network Access to the Internet. Access to the Internet. Communication. Communication. Wireless network connection used:- reduces additional burden of wiring Wireless network connection used:- reduces additional burden of wiring

Gestural interaction 3 types of gestures Supported by multitouch systems Supported by multitouch systems Free hand gestures Free hand gestures Iconic gestures Iconic gestures

Features Implementation of augmented reality Implementation of augmented reality Allows the user to interact with the projected information using hand gestures Allows the user to interact with the projected information using hand gestures Informations projected on to any opacque surface Informations projected on to any opacque surface User does not need to wear any special glasses User does not need to wear any special glasses Multitouch and moltiuserinterface Multitouch and moltiuserinterface

Applications o Drawing application o Defines a new perspective for human machine relationship o Bridges the gap between digital and real world environment

VIDEO CLIPS SHOWING THE APPLICATIONS OF THE DEVICE

Advantages Portable. Portable. Uses commercially available components. Uses commercially available components. No physical interaction with any hardware No physical interaction with any hardware

Challenges Image Recognition. Image Recognition. Surface is required. Surface is required. Correctness and clarity of gestures. Correctness and clarity of gestures. User profiling. User profiling.

Summary Highly relevant to a unique user and their current focus of attention Highly relevant to a unique user and their current focus of attention Easily accessible Easily accessible May replace traditional keyboard or screen based interface May replace traditional keyboard or screen based interface

Thank You…….